Job Description

Responsibilities

• Lead the architecture design and implementation of advanced analytics solutions using Azure Databricks Fabric.
• The ideal candidate will have a deep understanding of big data technologies data engineering and cloud computing with a strong focus on Azure Databricks along with Strong SQL.
• Design and implement solutions on Microsoft Fabric including Fabric Pipeline Lakehouse Warehouse and RealTime Analytics EventHouse.
• Knowledge on Migrate existing workloads eg Synapse Databricks Data Mart into Fabric.
• Work closely with business stakeholders and other IT teams to understand requirements and deliver effective solutions.
• Oversee the end to end implementation of data solutions ensuring alignment with business requirements and best practices.
• Lead the development of data pipelines and ETL processes using Azure Databricks PySpark and other relevant tools.
• Integrate Azure Databricks with other Azure services eg Azure Data Lake Azure Synapse Azure Data Factory and onpremise systems.
• Provide technical leadership and mentorship to the data engineering team fostering a culture of continuous learning and improvement.
• Ensure proper documentation of architecture processes and data flows while ensuring compliance with security and governance standards.
• Ensure best practices are followed in terms of code quality data security and scalability.
• Stay updated with the latest developments in Databricks and associated technologies to drive innovation.

Essential Skills

• Strong experience with Azure Databricks including cluster management notebook development and Delta Lake
• Proficiency in big data technologies eg Hadoop Spark and data processing frameworks eg PySpark
• Deep understanding of Azure services like Azure Data Lake Azure Synapse and Azure Data Factory
• Experience with ETLELT processes data warehousing and building data lakes
• Strong SQL skills and familiarity with NoSQL databases
• Experience with CICD pipelines and version control systems like Git
• Knowledge of cloud security best practices

Job Tags

Similar Jobs