Job Title:  Consultant | Data analysis | Delhi | Technology Strategy & Transformation

Job Description – Data Engineer (Databricks, Data Vault, PySpark, SQL & Azure)

Position: Data Engineer

Experience: 3–5 Years

Location: As per business requirement

Employment Type: Full-Time

Job Summary

We are seeking a highly motivated Data Engineer with 3–5 years of experience in designing, developing, and optimizing modern data platforms using Databricks, Data Vault, PySpark, SQL, and Azure. The ideal candidate should have strong expertise in building scalable data pipelines, implementing Data Vault modeling, and supporting enterprise data warehousing and analytics initiatives.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using PySpark and Databricks.
• Implement and manage Data Vault 2.0 models, including Hubs, Links, and Satellites.
• Develop ETL/ELT processes for ingesting, transforming, and loading structured and unstructured data.
• Build and optimize complex SQL queries, stored procedures, and data transformations.
• Leverage Azure Data Services such as Azure Data Factory, Azure Data Lake Storage, Azure Synapse, and Azure Databricks.
• Ensure data quality, consistency, security, and governance across enterprise data platforms.
• Collaborate with business stakeholders, data architects, and analytics teams to understand data requirements.
• Monitor and optimize pipeline performance, scalability, and cost efficiency.
• Support CI/CD implementation and deployment of data engineering solutions.
• Troubleshoot and resolve data processing and integration issues.

Required Skills

• 3–5 years of hands-on experience in Data Engineering.
• Strong expertise in Databricks and PySpark.
• Experience implementing Data Vault 2.0 data modeling methodologies.
• Advanced knowledge of SQL and database performance tuning.
• Hands-on experience with Azure Cloud services:
• Azure Databricks
• Azure Data Factory (ADF)
• Azure Data Lake Storage (ADLS)
• Azure Synapse Analytics
• Experience with ETL/ELT frameworks and large-scale data processing.
• Knowledge of data warehousing concepts and dimensional modeling.
• Familiarity with Git, CI/CD pipelines, and Agile methodologies.

Preferred Skills

• Experience with Delta Lake and Lakehouse architecture.
• Knowledge of Azure DevOps.
• Exposure to real-time/streaming data processing.
• Understanding of data governance, data quality, and metadata management frameworks.
• Experience in BFSI, Retail, Manufacturing, or other enterprise environments.

Educational Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.

Key Competencies

• Strong analytical and problem-solving skills.
• Excellent communication and stakeholder management.
• Ability to work independently and collaboratively in a fast-paced environment.
• Strong attention to detail and commitment to delivering high-quality solutions.