The Associate Process Manager will design and maintain ETL/ELT pipelines, optimize data workflows, and support data solutions for analytics in eCommerce, ensuring data quality and performance optimization.
Roles & Responsibility
- Design, develop, and maintain robust ETL/ELT pipelines using Databricks (Spark) and Python (PySpark).
- Develop and optimize complex transformations using SQL (joins, window functions, CTEs, query tuning).
- Build curated datasets and data models to support reporting, dashboards, and advanced analytics use cases.
- Implement pipeline reliability best practices: data quality checks, monitoring, alerting, and reconciliation.
- Optimize Databricks workloads for performance and cost (cluster sizing, partitioning strategies, caching, file formats).
- Work with structured and semi-structured data (JSON, CSV, Parquet/Delta) and handle schema evolution.
- Collaborate with stakeholders to understand business KPIs and deliver data solutions aligned to retail/eCommerce metrics (sales, orders, returns, inventory, customer cohorts).
- Follow engineering best practices for version control (Git), documentation, reusable code patterns, and testing.
Good to have: Support or migrate Alteryx workflows into Python/Databricks pipelines.
Must have skills
- 4-8 years of experience in Data Engineering / Data Warehousing / Big Data.
- Strong hands-on experience with Databricks (Jobs/Workflows, notebooks, cluster concepts, Spark tuning fundamentals).
- Strong programming skills in Python (PySpark preferred).
- Excellent SQL skills, including performance tuning and writing complex analytical queries.
- Experience building scalable pipelines and working with large datasets in distributed environments.
- Strong understanding of data engineering concepts: ETL/ELT, orchestration, data validation, and observability.
- Familiarity with modern data storage formats and practices (Delta/Parquet, partitioning, incremental loads).
Top Skills
Csv
Databricks
Delta
JSON
Parquet
Pyspark
Python
Spark
SQL
eClerx LLC Mumbai, Maharashtra, IND Office
Express Towers, 4th Floor, Ramnath Goenka Marg, Nariman Point, Mumbai, Maharashtra, India, 400021
Similar Jobs
Analytics
Responsible for managing processes related to big data engineering and analytics, ensuring efficient operation and quality control.
Top Skills:
Big Data
Analytics
Assist in managing and optimizing processes within the organization, focusing on efficiency and effectiveness.
Analytics
Develop and animate motion graphics for various media, collaborating with the creative team and managing multiple projects while utilizing industry-standard software.
Top Skills:
Adobe After EffectsIllustratorPhotoshop
What you need to know about the Mumbai Tech Scene
From haggling for the best price at Chor Bazaar to the bustle of Crawford Market, the energy of Mumbai's traditional markets is a key part of the city's charm. And while these markets will always have their place, the city also boasts a thriving e-commerce scene, ranking among the largest in the region. Driven by online sales in everything from snacks to licensed sports merchandise to children's apparel, the local industry is worth billions, with companies actively recruiting to meet the demands of continued growth.
