Job Overview

job description:

• 5+ years of hands-on experience in data engineering/etl using databricks on aws / azure cloud infrastructure and functions.
• 3+ years of experience in pbi and data warehousing experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• experience with aws (e.g. s3, athena, glue, lambda, etc.) preferred.
• deep understanding of data warehousing concepts (dimensional (star-schema), scd2, data vault, denormalized, obt) implementing highly performant data ingestion pipelines from multiple sources
• strong proficiency in python and sql.
• deep understanding of databricks platform features (delta lake, databricks sql, mlflow)
• experience with ci/cd on databricks using tools such as bitbucket, github actions, and databricks cli
• integrating the end-to-end databricks pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data is always maintained.
• working within an agile delivery / devops methodology to deliver proof of concept and production implementation in iterative sprints.
• experience with delta lake, unity catalog, delta sharing, delta live tables (dlt), mlflow.
• basic working knowledge of api or stream based data extraction processes like salesforce api, bulk api.
• understanding of data management principles (quality, governance, security, privacy, life cycle management, cataloguing)
• nice to have: databricks certifications and aws solution architect certification.
• nice to have: experience with building data pipeline from various business applications like salesforce, marketo, netsuite, workday etc.