Job Title: Technology & Transformation - Engineering - Senior Consultant/Manager - Azure Data Bricks (ADB)
Senior Data Engineer
Total Experience - 7 to 11 Years
Relevant Experience - 5+
Keywords – Python, Databricks, Pyspark, SQL, Live / Streaming data, batch processing data
Roles and Responsibilities:
The Data Engineer will work on data engineering projects for various business units, focusing on delivery of complex data management solutions by leveraging industry best practices. They work with the project team to build the most efficient data pipelines and data management solutions that make data easily available for consuming applications and analytical solutions. A Data engineer is expected to possess strong technical skills.
Key Characteristics
Technology champion who constantly pursues skill enhancement and has inherent curiosity to understand work from multiple dimensions.
Interest and passion in Big Data technologies and appreciates the value that can be brought in with an effective data management solution.
Has worked on real data challenges and handled high volume, velocity, and variety of data.
Excellent analytical & problem-solving skills, willingness to take ownership and resolve technical challenges.
Contributes to community building initiatives like CoE, CoP.
Knowledge of and skills in various programming languages primarily Python
• Must to have knowledge on Back-end frameworks
• Thorough understanding of containers and functions. Deployment experience with Kubernetes (K8s) or Functions is highly desirable.
• Experience using Cloud Native CI/CD tools (Azure Pipelines/Circle CI/Jenkins X).
Mandatory skills:
Python - Master
Azure - Master
ELT - Skill
Data Modeling - Skill
Data Integration & Ingestion - Skill
Data Manipulation and Processing - Skill
GITHUB, Action, Azure DevOps - Skill
Data factory, Databricks, SQL DB, Synapse, Stream Analytics, Glue, Airflow, Kinesis, Redshift, SonarQube, PyTest - Skill
Optional skills:
Experience in project management, running a scrum team.
Experience working with BPC, Planning.
Exposure to working with external technical ecosystem.