Job Title:  Assistant Manager | Python | Delhi | TTC - DOMESTIC

Your potential, unleashed.

India’s impact on the global economy has increased at an exponential rate and Deloitte presents an opportunity to unleash and realise your potential amongst cutting edge leaders, and organisations shaping the future of the region, and indeed, the world beyond.
At Deloitte, your whole self to work, every day. Combine that with our drive to propel with purpose and you have the perfect playground to collaborate, innovate, grow, and make an impact that matters.

The Team

Innovation, transformation and leadership occur in many ways. At Deloitte, our ability to help solve clients’ most complex issues is distinct. We deliver strategy and implementation, from a business and technology view, to help you lead in the markets where you compete. Learn more about our Tax Practice.

Job Summary

We are seeking a talented and experienced Deputy Manager - PySpark Developer with 4 to 6 years of experience to join our dynamic team. The ideal candidate will possess extensive expertise in PySpark, big data technologies, and data engineering practices. As a Deputy Manager, you will play a critical role in developing and optimizing data processing pipelines, ensuring data quality, and collaborating with cross-functional teams to deliver high-impact data solutions. 

Key Responsibilities

1. PySpark Development: 

• Design, develop, and maintain scalable and efficient PySpark applications to process large datasets. 

• Write clean, maintainable, and efficient code following best practices and coding standards. 

• Optimize and tune PySpark jobs to improve performance and reduce execution times. 

2. Big Data Technologies: 

• Work with big data tools and frameworks such as Hadoop, Hive, and Spark to manage and process large-scale data. 

• Implement and maintain data pipelines using Apache Airflow or similar orchestration tools. 

• Ensure data is ingested, processed, and stored in an optimal manner for further analysis and reporting. 

3. Data Engineering: 

• Design and implement ETL processes to extract, transform, and load data from various sources. 

• Ensure data quality and consistency through robust validation and testing procedures. 

• Collaborate with data scientists and analysts to understand data requirements and deliver solutions that meet their needs. 

4. Cloud Integration: 

• Deploy and manage big data solutions on cloud platforms such as AWS, Azure, or Google Cloud. 

• Leverage cloud services for storage, computing, and data processing to enhance scalability and performance. 

• Optimize cloud resource utilization to reduce costs and improve efficiency. 

5. Collaboration and Communication: 

• Work closely with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand requirements and deliver data solutions. 

• Communicate effectively with team members and stakeholders, providing regular updates on project status. 

• Document technical designs, processes, and procedures to ensure knowledge sharing and continuity.