Job Title:  Technology & Transformation - Engineering - Manager - Azure Databricks - Bangalore

The Team 

Deloitte’s Technology & Transformation practice can help you uncover and unlock the value buried deep inside vast amounts of data. Our global network provides strategic guidance and implementation services to help companies manage data from disparate sources and convert it into accurate, actionable information that can support fact-driven decision-making and generate an insight-driven advantage. Our practice addresses the continuum of opportunities in business intelligence & visualization, data management, performance management and next-generation analytics and technologies, including big data, cloud, cognitive and machine learning.

We are seeking highly skilled Databricks Data Engineers to join our data modernization team. You will play a pivotal role in designing, developing, and maintaining robust data solutions on the Databricks platform. Your experience in data engineering, along with a deep understanding of Databricks, will be instrumental in building solutions to drive data-driven decision-making across a variety of customers.

Work Profile

Azure Databricks/dbt Data Engineer in the analytics domain designs, builds, and maintains high-performance ETL/ELT pipelines using Databricks (PySpark/SQL) and dbt to transform raw data into curated, analytics-ready models. This role requires 5-10 years' experience in Azure cloud ecosystems, strong dimensional modelling skills, and expertise in implementing data pipeline, data quality, and governance using modern data stack tools. 

Key Responsibilities

• Data Modelling & Transformation: Architect and develop advanced data transformation workflows using dbt (models, tests, macros) within the Databricks Medallion Architecture (Bronze/Silver/Gold).
• Pipeline Development: Build and optimize scalable, high-throughput, and low-latency data pipelines using Azure Databricks (PySpark,SQL), dbt, Airflow, and Azure Data Lake Storage (ADLS).
• Analytics Engineering: Translate complex business requirements into data models (Star/Snowflake schemas) to support BI reporting, ad-hoc analysis, and AI/ML initiatives.
• Code & Quality Control: Implement CI/CD for dbt using GitHub/Azure DevOps, ensuring rigorous testing, documentation, and data quality checks (using Unity Catalog).
• Performance Tuning: Optimize Spark jobs, SQL queries, and dbt models for maximum efficiency and cost-performance.
• Mentorship: Provide technical guidance and conduct code reviews for junior engineers. 

Required Technical Skills

• Core Tools: Expert-level Azure Databricks, PySpark, Spark SQL, Airflow and dbt.
• Cloud & ETL: Extensive experience with Azure Databricks or similar cloud environments in medallion architecture
• Languages: Strong Python and SQL skills.
• Engineering Practices: Git/GitHub, CI/CD, and data modeling (Kimball methodology).
• Data Governance: Experience with Unity Catalog or similar data cataloging/governance tools. 

Good to Have Qualifications

• Experience with creation of packages and working with SDK.
• Background in domain-specific analytics (e.g., Retail, Manufacturing).
• Certification in Databricks
• Anyone with graduation

Key Competencies

• Collaboration: Working with Data Scientists, Data Analysts, and business stakeholders.
• Problem-Solving: Ability to debug complex data pipeline failures and optimize performance.
• Communication: Strong verbal and written communication skills to articulate technical designs.