Top Data Engineer Jobs in Mumbai

Design, build, optimize, and maintain high-performance Spark-based data pipelines using Scala/Java and Hive on Hadoop/CDP. Own full project lifecycle, enforce coding best practices, troubleshoot Spark/Hive/YARN performance, and collaborate with stakeholders to deliver scalable data solutions.

Design, implement, and optimize Apache DolphinScheduler workflows and ETL pipelines. Integrate with Hadoop, Spark, Flink, Hive, and Kafka. Develop custom plugins, troubleshoot performance and scalability issues, monitor job execution and resource allocation, and maintain CI/CD, version control, and infrastructure automation while collaborating with data engineers and DevOps teams.

Design, build, and optimize PySpark applications and ETL pipelines to process large-scale datasets from relational, NoSQL, file, and streaming sources. Ensure data quality, implement error handling, and collaborate with analysts, scientists, and architects to deliver performant data solutions.

Design, configure, and support backend software and APIs; manage configuration files and environments; test and document API endpoints; collaborate on system data flow; participate in large-scale live data migrations (billions of records) into FinCloud; produce documentation and troubleshooting guides; work in agile teams and support integration and observability efforts.

Design, develop, test, and deploy high-performance Spark/Scala data processing applications and ETL pipelines on Cloudera (CDH). Optimize Spark and Cloudera platform performance, integrate HDFS/Hive/Impala/HBase/Kafka, ensure data integrity/security, and implement version control and CI/CD for production pipelines.

Build and scale an AI-powered stock research platform: design data pipelines ingesting SEC and market data, develop backend APIs, implement semantic search with embeddings and vector DBs, improve production reliability and performance, and own features end-to-end from design through deployment and monitoring.

The role involves managing data operations, automating processes, monitoring data pipelines, and maintaining operational documentation within QRT.

Design and own end-to-end data platform architecture, build ingestion and transformation pipelines, model multi-tenant datasets, optimize Postgres and ClickHouse performance/cost, implement orchestration and observability, define data quality and reconciliation standards, and mentor data/product engineers.

Support electrical design, engineering, commissioning, and delivery of liquid-cooled data centre solutions: prepare layouts, SLDs, wiring diagrams, BOMs, perform load calculations, assist testing/FAT/SAT, troubleshoot commissioning issues, coordinate cross-functional teams, and ensure compliance with IEC/NEC/IEEE and company HSE standards.

Design, implement, and lead large-scale generative AI and ML systems (LLM fine-tuning, RAG, agents, Graph ML). Build REST APIs, deploy with Docker/Kubernetes, optimize big-data workflows, mentor engineers, and operate ML services on cloud platforms.

Build and maintain end-to-end data pipelines using PySpark and SQL on Databricks/Delta. Establish data modeling and pipeline best practices, create AI-ready analytical datasets, troubleshoot pipeline issues, and collaborate with stakeholders to embed business logic and optimize reliability, efficiency, and performance.

As a Data Engineer, you'll build and monitor large-scale data pipelines, develop the DBT setup for data transformation and solve customer problems using your SQL skills.

Manage and support AKS-based enterprise data platform infrastructure, administer Apache NiFi, Azure Event Hub and Functions, implement CI/CD with Azure DevOps, automate infrastructure with Terraform/Bicep/ARM, monitor with Azure Monitor and Log Analytics, troubleshoot production incidents, ensure security, capacity planning, and collaborate with data engineering teams.

Lead the architecture and implementation of a distributed data platform for microscopy, ensuring scalability and high-performance in data management.

As a key member of the Data Infrastructure team, you will develop and optimize software features for Apache Hudi's transactional engine, enhance database performance, and solve complex data processing challenges.

Design, build, and optimize high-volume data pipelines using Python, Databricks and Spark on Azure. Create and maintain automation test suites (Azure DevOps), perform manual/load/exploratory testing, collaborate with BAs and developers to meet sprint goals, and assist with sprint estimation and delivery.

Design, build, and operate scalable data engineering services and ingestion pipelines (CDC/streaming). Develop and optimize a centralized analytics warehouse, data models, and aggregation strategies. Build APIs, enforce multi-tenant security and data governance, monitor pipeline health, improve automation and observability, and ensure production reliability through strong engineering practices.