Design, develop, and maintain scalable ETL/ELT pipelines and data integration solutions using Azure Databricks, ADF, and SSIS. Optimize performance, ensure data quality and governance, support reporting/analytics, monitor production pipelines, and migrate legacy ETL to Azure platforms while collaborating with stakeholders.
This role is for one of the Weekday's clients
Salary range: Rs 500000 - Rs 2000000 (ie INR 5 - 20 LPA)
Min Experience: 5+ years
Location: Mumbai, Maharashtra, India
JobType: full-time
We are looking for an experienced Data Engineer to design, develop, and maintain scalable data pipelines and enterprise-grade data integration solutions. The ideal candidate will have strong expertise in Microsoft Azure data services, particularly Azure Databricks, Azure Data Factory (ADF), and SQL Server Integration Services (SSIS). You will work closely with data architects, analysts, and business stakeholders to build reliable data platforms that support reporting, analytics, and business intelligence initiatives.
This role requires hands-on experience in data transformation, ETL/ELT development, cloud-based data engineering, and performance optimization. The ideal candidate should be passionate about building high-quality, scalable, and efficient data solutions while ensuring data accuracy, security, and governance.
RequirementsKey Responsibilities
- Design, develop, and maintain robust ETL/ELT pipelines for ingesting, transforming, and loading data from multiple sources.
- Build and optimize scalable data processing solutions using Azure Databricks and Apache Spark.
- Develop and manage workflows using Azure Data Factory for orchestrating enterprise data movement and transformation.
- Create, maintain, and enhance SQL Server Integration Services (SSIS) packages for on-premises and hybrid data integration requirements.
- Develop optimized SQL queries, stored procedures, views, and database objects to support reporting and analytics.
- Collaborate with business teams to understand data requirements and translate them into technical solutions.
- Ensure data quality through validation, cleansing, reconciliation, and monitoring processes.
- Optimize pipeline performance, troubleshoot bottlenecks, and implement best practices for scalability and reliability.
- Integrate structured and semi-structured data from multiple enterprise systems.
- Monitor production data pipelines, resolve failures, and implement proactive monitoring mechanisms.
- Participate in code reviews, documentation, and knowledge-sharing initiatives.
- Follow data governance, security, compliance, and best practices throughout the data lifecycle.
- Support migration of legacy ETL processes to modern Azure-based data platforms where applicable.
- 5–8 years of experience in Data Engineering or ETL Development.
- Strong hands-on expertise in Azure Databricks.
- Extensive experience with Azure Data Factory (ADF).
- Proficiency in SQL Server Integration Services (SSIS).
- Strong SQL programming skills with experience in query optimization and performance tuning.
- Experience developing scalable ETL/ELT pipelines.
- Good understanding of Azure Data Lake Storage and cloud-based data architectures.
- Experience with Apache Spark using PySpark or Spark SQL.
- Strong knowledge of relational databases and data warehouse concepts.
- Familiarity with data modeling, data transformation, and data integration techniques.
- Experience with source control systems such as Git.
- Strong analytical, troubleshooting, and problem-solving skills.
- Experience with Azure Synapse Analytics.
- Knowledge of Azure SQL Database or SQL Server administration.
- Familiarity with Delta Lake architecture.
- Experience with CI/CD pipelines for Azure data solutions.
- Exposure to Power BI or other business intelligence tools.
- Understanding of DevOps practices for data engineering.
- Experience working with REST APIs and data ingestion from external systems.
- Knowledge of data governance, security, and compliance standards.
- Familiarity with Agile/Scrum development methodologies.
- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
- Microsoft Azure Data Engineer certification is an added advantage.
- Excellent communication and stakeholder management skills.
- Ability to work independently while collaborating effectively within cross-functional teams.
- Strong commitment to delivering high-quality, scalable, and maintainable data engineering solutions.
Similar Jobs
Fintech • Information Technology • Insurance • Financial Services • Big Data Analytics
Develop and maintain Azure Databricks data pipelines using PySpark, Spark SQL, SQL, and Delta Lake. Support Informatica-to-Databricks modernization and HR data integrations, build source-to-target mappings and dimensional models, perform profiling, testing, reconciliation, and defect resolution, and create technical documentation. Apply data governance and access controls through Unity Catalog, participate in agile ceremonies and code reviews, and escalate risks while working under senior direction.
Top Skills:
SparkAzure Data FactoryAzure Data Lake StorageAzure DatabricksAzure DevopsAzure SqlAzure SynapseCi/CdDelta LakeGitGithub CopilotInformaticaMicrosoft PurviewOracle Hcm CloudPysparkPythonSpark SqlSQLUnity Catalog
Fintech • Information Technology • Financial Services
Build, deploy, and support production-grade data pipelines and platforms for structured and semi-structured datasets. Develop Snowflake data models and advanced SQL ELT transformations, implement workflow orchestration and data-quality controls, and improve reliability, observability, lineage, and performance. Partner with product teams and data specialists, troubleshoot production issues, modernize legacy pipelines, and deliver maintainable solutions using Python and cloud technologies.
Top Skills:
Amazon Web Services (Aws)APIsAzureContinuous Integration And Continuous Delivery (Ci/Cd)Dag Workflow OrchestrationData ModelingData WarehousingDistributed Data Analytics EnginesEltEvent Streaming And MessagingGitPythonSnowflakeSQL
Fintech • Information Technology • Financial Services
Builds and maintains enterprise data platform frameworks, including automated pipelines, ingestion, processing, orchestration, data quality, testing, and performance improvements. The role uses Python, SQL, Airflow, Snowflake, dbt, Azure, and streaming technologies. Responsibilities include collaborating with stakeholders across the SDLC, supporting platform users, conducting root cause analysis, providing L2/L3 support, contributing to complex system design, and ensuring scalable, reliable production operations.
Top Skills:
AdlsAdoAhaApache AirflowAzureDbtKafkaPythonSnowflakeSnowpipe StreamingSQL
What you need to know about the Mumbai Tech Scene
From haggling for the best price at Chor Bazaar to the bustle of Crawford Market, the energy of Mumbai's traditional markets is a key part of the city's charm. And while these markets will always have their place, the city also boasts a thriving e-commerce scene, ranking among the largest in the region. Driven by online sales in everything from snacks to licensed sports merchandise to children's apparel, the local industry is worth billions, with companies actively recruiting to meet the demands of continued growth.
