Job Title:  Senior Consultant | Site Reliability Engineering | Bengaluru | Engineering

A DevOps Engineer in the analytics and cloud ecosystem designs, automates, and optimizes end‑to‑end CI/CD pipelines to ensure reliable, scalable, and secure delivery of data and ML workloads. This role requires strong expertise in Azure DevOps, multi‑stage YAML pipelines, and Databricks deployment automation (DAB), along with deep understanding of cloud infrastructure, environment promotion, and operational best practices. With 5–10+ years of experience, the engineer drives release engineering, environment governance, and DevOps standards across Azure, enabling high‑quality, consistent, and repeatable deployments for modern data platforms.

Key Responsibilities

CI/CD & Automation

• Design, build, and maintain multi-stage Azure DevOps CI/CD pipelines using YAML.
• Develop and manage automated deployment pipelines for Azure Databricks using Databricks Asset Bundles (DAB).
• Automate environment promotions across dev → staging → production.
• Implement versioning, artifact management, and structured release flows.
• Integrate pipelines with Azure ecosystem components (identity, networking, storage).

Collaboration & Best Practices

• Work closely with Data Engineers, Data Platform teams, and ML Engineers to streamline delivery workflows.
• Implement DevOps best practices, including idempotent deployments, validation gates, and rollback strategies.
• Ensure compliance, governance, and consistency across environments.

Requirements (Must-Have)

• Strong experience building and managing Azure DevOps YAML pipelines.
• Deep understanding of CI/CD concepts (stages, gates, approvals, environments).
• Hands-on experience with Azure Databricks.
• Proficiency with Databricks CLI (v0.200+) and Databricks Asset Bundles (databricks.yml).
• Experience with artifact management and automated deployment systems.
• Strong scripting skills using Python and/or Bash.

Nice to Have

• Experience with Terraform (Azure + Databricks provider).
• Hands-on with Azure Key Vault for secrets management.
• Familiarity with Unity Catalog (governance, permissions).
• Knowledge of Workload Identity / Managed Identity for secretless deployments.
• Understanding of Azure networking (Private Endpoints, VNet, DNS).
• Exposure to monitoring tools (Grafana, Prometheus, Azure Monitor).
• Awareness of Databricks cost optimization (cluster policies, job clusters).

Tech Stack

• Azure: Databricks, Storage, Networking, Identity
• Azure DevOps: Repos, Pipelines, Artifacts
• Databricks Asset Bundles (DAB)
• Python / Bash
• Terraform (preferred)