The Data Engineer will design and optimize large-scale data pipelines and workflows, ensuring data quality and collaborating with data scientists and engineers for insights and reporting.
We are looking for a highly skilled Data Engineer. The ideal candidate will have hands-on expertise in Big Data technologies, with a strong foundation in distributed data processing, real-time pipelines, and large-scale data systems. You will design, build, and optimize data solutions that power insights, reporting, and decision-making for our advertising ecosystem.
Key Responsibilities- Design, develop, and maintain large-scale data pipelines to support Ads reporting, attribution, and analytics use cases.
- Work extensively with Hive, Spark, SQL, Scala, and Kafka to process and manage petabyte-scale datasets.
- Optimize data workflows for performance, scalability, and cost efficiency.
- Partner with data scientists, product managers, and platform engineers to deliver high-quality, reliable datasets and APIs.
- Ensure data quality, integrity, and consistency across multiple data sources.
Troubleshoot and resolve issues in real-time streaming pipelines and batch data jobs.
- Continuously evaluate new technologies to enhance the Ads Data platform.
Requirements
- Strong programming experience in Scala (preferred), Java, or Python.
- Hands-on experience with Apache Spark (batch & streaming) for large-scale data processing.
- Proficiency in Hive, SQL, and data modeling for analytical workloads.
- Experience working with Kafka for real-time event streaming.
- Solid understanding of Big Data ecosystems (S4, Hive, Presto, Delta etc.).
- Strong debugging, performance tuning, and problem-solving skills.
- Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
- Experience in AdTech, Attribution, or Campaign Analytics.
- Familiarity with cloud-based big data solutions (AWS EMR, GCP BigQuery, Databricks, etc.).
- Familiarity with scheduling services like AirFlow
- Knowledge of data governance, security, and compliance best practices.
Benefits
- Best in class salary: We hire only the best, and we pay accordingly.
- Proximity Talks: Meet other designers, engineers, and product geeks — and learn from experts in the field.
- Keep on learning with a world-class team: Work with the best in the field, challenge yourself constantly, and learn something new every day.
Top Skills
Airflow
Aws Emr
Big Data
Databricks
Gcp Bigquery
Hive
Kafka
Scala
Spark
SQL
Similar Jobs
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
The role involves testing and validating Basel 3.1 initiatives, producing reports, collaborating with stakeholders, and enhancing existing controls.
Top Skills:
Basel 3.1Design SolutionsReportingRwa
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
The role involves acting as a liaison between Finance Operations and implementation teams, supporting the business's requirements and ensuring correct configuration and reporting for a project in finance transformation.
Top Skills:
FcdpSaracenTm1
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
The role involves managing finance change, regulatory reporting, project management, and software testing, requiring extensive experience in financial services and team leadership.
Top Skills:
Automation Testing ToolsExcelSQL
What you need to know about the Mumbai Tech Scene
From haggling for the best price at Chor Bazaar to the bustle of Crawford Market, the energy of Mumbai's traditional markets is a key part of the city's charm. And while these markets will always have their place, the city also boasts a thriving e-commerce scene, ranking among the largest in the region. Driven by online sales in everything from snacks to licensed sports merchandise to children's apparel, the local industry is worth billions, with companies actively recruiting to meet the demands of continued growth.
