Job Title:  T&T | EAD Engineering | Consultant | Azure

What impact will you make?

Every day, your work will make an impact that matters, while you thrive in a dynamic culture of inclusion, collaboration and high performance. As the undisputed leader in professional services, Deloitte is where you will find unrivaled opportunities to succeed and realize your full potential

Deloitte is where you will find unrivaled opportunities to succeed and realize your full potential.

The Team

Deloitte’s T&T practice can help you uncover and unlock the value buried deep inside vast amounts of data. Our global network provides strategic guidance and implementation services to help companies manage data from disparate sources and convert it into accurate, actionable information that can support fact-driven decision-making and generate an insight-driven advantage. Our practice addresses the continuum of opportunities in business intelligence & visualization, data management, performance management and next-generation analytics and technologies, including big data, cloud, cognitive and machine learning.

Work you’ll do

As a Data Engineer with expertise in PySpark, Databricks, and Microsoft Azure, SQL you will be responsible for designing, developing, and maintaining robust and scalable data pipelines and processing systems. You will work closely with data scientists, analysts, and other stakeholders to ensure our data solutions are efficient, reliable, and scalable.

Responsibilities:

• Design, develop, and optimize ETL pipelines using PySpark and Databricks to process large-scale data on the Azure cloud platform.

• Implement data ingestion processes from various data sources into Azure Data Lake and Azure SQL Data Warehouse.

• Develop and maintain data models, data schemas, and data transformation logic tailored for Azure.

• Collaborate with data scientists and analysts to understand data requirements and deliver high-quality datasets.

• Ensure data quality and integrity through robust testing, validation, and monitoring procedures.

• Optimize and tune PySpark jobs for performance and scalability within the Azure and Databricks environments.

• Implement data governance and security best practices in Azure.

• Monitor and troubleshoot data pipelines to ensure timely and reliable data delivery.

• Document data engineering processes, workflows, and best practices specific to Azure and Databricks.

Requirements:

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

• Proven experience as a Data Engineer with a strong focus on PySpark and Databricks.

• Proficiency in Python and PySpark for data processing and analysis.

• Strong experience with Azure data services, including Azure Data Lake, Azure Data Factory, Azure SQL Data Warehouse, and Azure Databricks.

• Strong SQL skills and experience with relational databases (e.g., MySQL, PostgreSQL) and NoSQL databases (e.g., MongoDB, Cassandra).

• Experience with big data technologies such as Hadoop, Spark, Hive, and Kafka.

• Strong understanding of data architecture, data modeling, and data integration techniques.

• Familiarity with Azure DevOps, version control systems (e.g., Git), and CI/CD pipelines.

• Excellent problem-solving skills and attention to detail.

• Strong communication and collaboration skills.

Preferred Qualifications:

• Experience with Delta Lake on Azure Databricks.

• Knowledge of data visualization tools (e.g., Power BI, Tableau).

• Experience with containerization and orchestration tools (e.g., Docker, Kubernetes).

• Understanding of machine learning concepts and experience working with data scientists.

 Skills

• Azure Data Factory: Experience in creating and orchestrating data pipelines, understanding of triggers and data flows.

• Databricks: Knowledge of Apache Spark, programming in Python, Scala or R, experience optimizing data processing and transformation jobs. Experience querying databases and tables in SQL.

• Azure Data Lake Storage: Experience working with ADLS Gen 1 and Gen 2, knowledge of hierarchy, file systems and security aspects.

• Azure DevOps: Experience working with repositories, pipelines, builds and releases, understanding CI/CD processes.

• Data integration: Knowledge of various data sources and data formats such as JSON, CSV, XML, Parquet and Delta. Also knowledge of databases such as Azure SQL, MySQL or PostgreSQL.

T

 Your role as a leader

At Deloitte India, we believe in the importance of leadership at all levels. We expect our people to embrace and live our purpose by challenging themselves to identify issues that are most important for our clients, our people, and for society and make an impact that matters.

In addition to living our purpose, Consultant across our organization:

• Develop high-performing people and teams through challenging and meaningful opportunities
• Deliver exceptional client service; maximize results and drive high performance from people while fostering collaboration across businesses and borders
• Influence clients, teams, and individuals positively, leading by example and establishing confident relationships with increasingly senior people
• Understand key objectives for clients and Deloitte; align people to objectives and set priorities and direction.
• Acts as a role model, embracing and living our purpose and values, and recognizing others for the impact they make

How you will grow

At Deloitte, our professional development plan focuses on helping people at every level of their career to identify and use their strengths to do their best work every day. From entry-level employees to senior leaders, we believe there is always room to learn. We offer opportunities to help build excellent skills in addition to hands-on experience in the global, fast-changing business world. From on-the-job learning experiences to formal development programs at Deloitte University, our professionals have a variety of opportunities to continue to grow throughout their career. Explore Deloitte University, The Leadership Centre.

Benefits

At Deloitte, we know that great people make a great organization. We value our people and offer employees a broad range of benefits. Learn more about what working at Deloitte can mean for you.

Our purpose                                                                                                                                                                         

Deloitte is led by a purpose: To make an impact that matters.

Every day, Deloitte people are making a real impact in the places they live and work. We pride ourselves on doing not only what is good for clients, but also what is good for our people and the

Communities in which we live and work—always striving to be an organization that is held up as a role model of quality, integrity, and positive change. Learn more about Deloitte's impact on the world

Recruiter tips

We want job seekers exploring opportunities at Deloitte to feel prepared and confident. To help you with your interview, we suggest that you do your research: know some background about the organization and the business area you are applying to. Check out recruiting tips from Deloitte professionals.