Data Engineer (Azure Databricks)

Responsibilities: Azure Databricks Platform Implementation

• Develop and maintain data solutions using Azure Databricks (notebooks, Delta Lake)

• Configure and optimize Databricks jobs, clusters, and workloads

• Integrate with Azure Data Lake (ADLS Gen2)

Databricks Genie Implementation Support

• Support implementation of Databricks Genie based on predefined architecture

• Prepare curated datasets for Genie consumption

• Optimize data structures for query performance

Data Pipeline Development

• Build ETL/ELT pipelines using Databricks and Azure tools

• Develop ingestion pipelines from APIs, databases, and external systems

• Ensure pipelines are reliable, monitored, and production-ready

Data Management & Governance

• Apply data quality checks and controls

• Implement access control and data organization practices

• Document pipelines and datasets

Production Readiness & Optimization

• Ensure scalability and performance

• Optimize queries and pipeline efficiency.

• Improve Genie performance to deliver faster, more efficient results

Skills & Experiences:

• Hands-on skills of Azure Databricks (Delta Lake, notebooks), Azure Data Lake (ADLS Gen2), Azure Data Factory or Synapse

• Solid SQL and ETL experience and medallion architecture familiarity

• Exposure to Databricks Genie, familiarity with Python/PySpark, Unity Catalog or data governance tools are advantageous

• Solid execution focus with the ability to deliver high-quality work independently

• High attention to details, particularly in data quality and accuracy

• Solid problem-solving skills, with the ability to troubleshoot data and pipeline issues

• Receptive to feedback and able to iterate quickly based on technical guidance

• Able to collaborate effectively within a technical team environment