Design, build, and optimize high-throughput ETL/ELT pipelines (Glue/PySpark), orchestrate workflows (Airflow/Step Functions), manage and tune multi-terabyte Redshift clusters, lead migrations from Snowflake/RDBMS, integrate heterogeneous sources, implement monitoring and governance, and collaborate with stakeholders to deliver analytics-ready datasets.
Job Title: Data Engineer
Location: Mumbai
Experience: 3-5 Years
Employment Type: Full-time
Position Overview:We are looking for a highly skilled and hands-on Senior Data Engineer to join our growing data engineering practice in Mumbai. This role requires deep technical expertise in building and managing enterprise-grade data pipelines, with a primary focus on Amazon Redshift, AWS Glue, and data orchestration using Airflow or Step Functions. You will be responsible for building scalable, high-performance data workflows that ingest and process multi-terabyte-scale data across complex, concurrent environments.
The ideal candidate is someone who thrives in solving performance bottlenecks, has led or participated in data warehouse migrations (e.g., Snowflake to Redshift), and is confident interfacing with business stakeholders to translate requirements into robust data solutions.
Key Responsibilities:● Design, develop, and maintain high-throughput ETL/ELT pipelines using AWS Glue (PySpark), orchestrated via Apache Airflow or AWS Step Functions.
● Own and optimize large-scale Amazon Redshift clusters and managing high concurrency workloads for very large user base:
● Lead and contribute to migration projects from Snowflake or traditional RDBMS to Redshift, ensuring minimal downtime and robust validation.
● Integrate and normalize data from heterogeneous sources including REST APIs, AWS Aurora (MySQL/Postgres), streaming inputs, and flat files.
● Implement intelligent caching strategies, leverage EC2 and serverless compute (Lambda, Glue) for custom transformations and processing at scale.
● Write advanced SQL for analytics, data reconciliation, and validation, demonstrating strong SQL development and tuning experience.
● Implement comprehensive monitoring, alerting, and logging for all data pipelines to ensure reliability, availability, and cost optimization.
● Collaborate directly with product managers, analysts, and client-facing teams to gather requirements and deliver insights-ready datasets.
● Champion data governance, security, and lineage, ensuring data is auditable and well-documented across all environments.
● 3-5 years of core data engineering experience, especially focused in Amazon Redshift hands-on performance tuning and large-scale management capacity.
● Demonstrated experience handling multi-terabyte Redshift clusters, concurrent query loads, and managing complex workload segmentation and queue priorities.
● Strong experience with AWS Glue (PySpark) for large-scale ETL jobs.
● Solid understanding and implementation experience of workflow orchestration using Apache Airflow or AWS Step Functions.
● Strong proficiency in Python, advanced SQL, and data modeling concepts.
● Familiarity with CI/CD pipelines, Git, DevOps processes, and infrastructure-as-code concepts.
● Experience with Amazon Athena, Lake Formation, or S3-based data lakes.
● Hands-on participation in Snowflake, BigQuery, or Teradata migration projects.
● AWS Certifications such as:
○ AWS Certified Data Analytics – Specialty
○ AWS Certified Solutions Architect – Associate/Professional
● Exposure to real-time streaming architectures or Lambda architectures.
● Excellent communication skills — must be able to confidently engage with both technical and non-technical stakeholders, including clients.
● Strong problem-solving mindset and a keen attention to performance, scalability, and reliability.
● Demonstrated ability to work independently, lead tasks, and take ownership of large-scale systems.
● Comfortable working in a fast-paced, dynamic, and client-facing environment.
Oneture Technologies Mumbai, Maharashtra, IND Office
Mumbai, India
Similar Jobs
Big Data • Food • Hardware • Machine Learning • Retail • Automation • Manufacturing
Design and build scalable cloud-based data solutions, develop ETL pipelines for warehouses and data lakes, implement data quality and validation processes, optimize data storage and processing, and collaborate with data teams, product owners, modelers, analysts, and stakeholders. The role requires expertise in Python, PySpark, SQL, Databricks, AWS services, data modeling, medallion architecture, and pipeline scheduling.
Top Skills:
AecorsoftAlteryxAmazon Ec2Amazon RedshiftApache AirflowAws GlueAws LambdaAws S3DatabricksDatasphereExcelLakeflow JobsPysparkPythonRSQLTableau
Blockchain • Fintech • Payments • Consulting • Cryptocurrency • Cybersecurity • Quantum Computing
Build and maintain large-scale data platforms, databases, ETL pipelines, and data solutions. Develop optimized SQL, Spark, Kafka, and streaming processes; support data analysts and scientists; perform large-scale extraction and analysis; improve data quality, modeling, architecture, automation, security, and deployment. Collaborate with stakeholders on requirements, investigations, root-cause analysis, and production data artifacts.
Top Skills:
Amazon EmrAmazon S3Apache KafkaAWSClouderaDatabricksDatadogDelta LakeDynatraceIcebergJavaScriptJenkinsOzonePl/SqlPythonRegexRelational DatabasesScalaSparkSpark StreamingSplunkSQLTrino
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Designs, develops, and maintains scalable batch and streaming data pipelines using Python, Scala, Spark, and Databricks. Responsibilities include optimizing large-scale data workloads, building ETL/ELT processes, modeling data warehouses and lakes, implementing data quality and governance standards, troubleshooting production issues, and collaborating with architects, analysts, and data scientists.
Top Skills:
SparkAws DatabricksAzure DatabricksDatabricksDatabricks WorkflowsDataframesDelta LakeEtl/EltGcp DatabricksPythonScalaSpark SqlSQL
What you need to know about the Mumbai Tech Scene
From haggling for the best price at Chor Bazaar to the bustle of Crawford Market, the energy of Mumbai's traditional markets is a key part of the city's charm. And while these markets will always have their place, the city also boasts a thriving e-commerce scene, ranking among the largest in the region. Driven by online sales in everything from snacks to licensed sports merchandise to children's apparel, the local industry is worth billions, with companies actively recruiting to meet the demands of continued growth.
