Job Title:  Engineering | ADMM | Manager | Databricks

About the Role

We are looking for a highly skilled Data Engineer with strong expertise in real‑time streaming, distributed systems, and Java-based data processing frameworks. The ideal candidate will have hands‑on experience building high‑throughput, low‑latency streaming pipelines using Kafka Streams / Spring Kafka / Apache Flink.

You will work closely with platform architects, product teams, and data consumers to design and deliver resilient, scalable, real‑time data pipelines for enterprise-grade systems.

Key Responsibilities

1. Real-Time Streaming & Pipeline Development

• Design, develop, and optimize real-time streaming applications using:
• Kafka Streams
• Spring Kafka
• Apache Flink
• Build high‑performance and fault‑tolerant event-driven microservices using Java 21.

2. Data Serialization & Schema Management

• Implement and manage AVRO-based schemas for consistent data serialization.
• Integrate and enforce schema governance through the Confluent Schema Registry.
• Ensure backward/forward schema compatibility and schema evolution best practices.

3. Data Ingestion & Integration

• Build and enhance pipelines for ingesting data from diverse source systems.
• Maintain connectors, producers, and consumers in the Kafka ecosystem.
• Optimize streaming workflows for scalability, resiliency, and low latency.

4. System Architecture & Performance

• Collaborate with architects to design event-driven and streaming data architectures.
• Implement robust observability using logs, metrics, dashboards, and alerts.
• Identify bottlenecks and improve throughput, memory management, and state handling.

5. Code Quality & DevOps Collaboration

• Write clean, high-quality, testable code leveraging Java 21 features (Records, Virtual Threads, Enhanced Switch, Pattern Matching).
• Partner with DevOps teams to streamline CI/CD pipelines, deployment automation, and environment configuration.

Required Skills & Experience

Core Technical Skills

Eduucation - Any Tech Graduation Bachelors/ Masters Degree

• Java 21 – deep understanding of modern Java features, multithreading, and JVM performance tuning.
• Kafka Streams, Spring Kafka, or Apache Flink – must have hands-on production experience.
• Apache Kafka – topics, partitions, consumer groups, offsets, retention policies.
• AVRO – schema design, serialization/deserialization patterns.
• Confluent Schema Registry – schema management, compatibility rules, governance.

Additional Preferred Skills

• Understanding of distributed systems, event sourcing, and stream processing patterns.
• Experience with microservices architecture and containerization (Docker/Kubernetes).
• Knowledge of CI/CD pipelines (Jenkins/GitLab/GitHub Actions).
• Familiarity with cloud platforms (AWS/GCP/Azure) is an advantage.
• Experience with monitoring tools (Prometheus, Grafana, Splunk, Datadog).