Designs and maintains scalable AWS, Snowflake, and dbt data pipelines and transformation frameworks. Builds ETL/ELT workflows, dimensional models, data quality frameworks, and optimized SQL workloads for structured and semi-structured data. Responsibilities include pipeline orchestration, monitoring, troubleshooting, CI/CD, legacy ETL modernization, documentation, production support, and collaboration with technical teams. The role also contributes to architecture discussions, Agile delivery, code reviews, and mentoring junior engineers.
Senior Data Engineer; Must Have: Strong hands-on AWS, Snowflake, SQL and dbt experience. Nice to have – Python, PySpark, Airflow, Informatica
Requirements and Responsibilities
What You’ll Do
- Design, develop, and maintain scalable data pipelines and data transformation frameworks using AWS, Snowflake, and dbt.
- Develop robust ETL/ELT pipelines to ingest, transform, and integrate data from multiple source systems into Snowflake.
- Build and maintain data models in Snowflake supporting analytics, reporting, operational applications, and downstream data consumers.
- Develop reusable dbt models, macros, tests, snapshots, and incremental pipelines following established data engineering standards.
- Write complex and optimized SQL for data transformation, aggregation, validation, and analytical workloads.
- Design and implement scalable data ingestion solutions using AWS services such as S3, Glue, Lambda, and related data services.
- Develop data pipelines for both structured and semi-structured data and work with formats such as JSON, CSV, Parquet, and Avro.
- Implement incremental loading, CDC, partitioning, clustering, and other optimization techniques to efficiently process large datasets.
- Design and maintain appropriate data warehouse schemas and dimensional models, including fact tables, dimension tables, star schemas, and data marts.
- Implement data quality and validation frameworks using dbt tests, SQL-based validation, reconciliation, and data profiling.
- Monitor data pipelines and proactively identify failures, performance issues, data quality problems, and processing bottlenecks.
- Troubleshoot pipeline failures and data discrepancies and perform detailed root-cause analysis.
- Optimize Snowflake workloads through appropriate warehouse sizing, query optimization, clustering, partition pruning, and workload management.
- Implement appropriate logging, monitoring, error handling, alerting, and recovery mechanisms for production data pipelines.
- Develop automated data pipelines and workflows using orchestration tools such as AWS Glue, Airflow, or similar technologies.
- Implement version control, CI/CD, and deployment processes for dbt models, SQL code, and data pipelines.
- Work closely with Data Architects, Business Analysts, ETL Developers, SQL Developers, QA Engineers, and application teams to understand requirements and deliver scalable data solutions.
- Participate in data architecture and technical design discussions and contribute to data platform modernization initiatives.
- Support migration of legacy ETL processes to cloud-based AWS/Snowflake ELT architectures.
- Develop and maintain technical documentation covering data pipelines, data models, source-to-target mappings, dependencies, and operational procedures.
- Conduct code reviews and ensure adherence to development, security, performance, and data engineering standards.
- Support production releases, incident management, troubleshooting, and post-production validation.
- Work in an Agile/Scrum environment and actively participate in sprint planning, daily stand-ups, reviews, retrospectives, and technical discussions.
- Mentor junior data engineers and contribute to knowledge sharing and continuous improvement.
What You’ll Bring
- Bachelor’s degree or higher in Computer Science, Information Technology, Engineering, or a related field.
- 5+ years of relevant experience in data engineering, data integration, data warehousing, or cloud data platforms.
- Strong hands-on experience with Snowflake as an enterprise cloud data warehouse.
- Strong hands-on experience with AWS data services, particularly:
- Amazon S3
- AWS Glue
- AWS Lambda
- IAM and security concepts
- Other AWS data/analytics services as applicable
- Strong hands-on experience with dbt (data build tool), including:
- dbt models
- Incremental models
- Macros
- Sources
- Seeds
- Snapshots
- Tests
- Documentation
- Model dependencies
- Advanced proficiency in SQL, including complex joins, CTEs, subqueries, window functions, aggregations, and performance optimization.
- Strong understanding of ETL/ELT architecture and data pipeline development.
- Strong understanding of data warehousing and dimensional modeling concepts.
- Experience designing and implementing fact and dimension models, star schemas, snowflake schemas, and data marts.
- Experience working with large-volume datasets and optimizing data processing workloads.
- Experience with structured and semi-structured data such as JSON, CSV, Parquet, and Avro.
- Strong understanding of data quality, validation, reconciliation, and data governance practices.
- Experience implementing incremental loads and CDC processes.
- Experience with Git and CI/CD practices for data engineering projects.
- Experience with pipeline orchestration and scheduling tools such as Airflow, AWS Glue Workflows, or similar technologies.
- Strong troubleshooting and root-cause-analysis skills.
- Experience working in Agile/Scrum environments.
- Banking or financial-services domain experience is preferred.
- Experience with Python for data engineering, automation, or scripting is a plus.
- Experience with PySpark / Apache Spark is a plus.
- Experience with Informatica or other enterprise ETL tools is a plus, particularly for legacy ETL modernization.
- Experience with data migration from traditional data warehouses to AWS/Snowflake is a plus.
- Knowledge of Snowflake security, RBAC, data sharing, streams/tasks, and stored procedures is a plus.
- Strong analytical and problem-solving skills.
- Ability to work independently and collaborate effectively with cross-functional teams.
- Strong communication skills in English, both written and verbal.
Recommended External Screening Criteria
For Senior Data Engineer – AWS/Snowflake/dbt, I would keep the following as the core mandatory screening criteria:
Must Have
- 5+ years of Data Engineering / Data Warehousing experience
- Strong hands-on Snowflake
- Strong hands-on SQL
- Hands-on dbt development
- AWS data services experience, particularly S3 and Glue
- ETL/ELT pipeline development
- Data Warehousing & Dimensional Modeling
- Experience with large-volume data
- Data quality and validation
- Git / CI/CD experience
Preferred
- Python
- PySpark / Spark
- Airflow
- Snowflake Streams & Tasks
- Snowflake performance tuning
- AWS Lambda
- Informatica
- CDC
- Legacy ETL modernization
- Banking / Financial Services experience
Ness Digital Engineering Andheri East, Maharashtra, IND Office
9th Floor, Times Square, Andheri- Kurla, Marol, Mumbai., Andheri East, India, 400059
Similar Jobs
Security • Cybersecurity
Own global revenue recognition under ASC 606 and accounting processes including commissions, COGS, and leases. Review tax provisions, returns, transfer pricing, and subsidiary filings; support external audits and cross-functional reporting. Lead accounting projects, process improvements, and AI-driven workflow automation. Requires a CA, Big 4 high-tech experience, strong US GAAP and revenue recognition knowledge, ERP experience, advanced Excel skills, and excellent communication.
Top Skills:
Ai Automation ToolsAsc 340-40Asc 606ExcelMS OfficeNetSuiteUs Gaap
Artificial Intelligence • Big Data • Cloud • Information Technology • Software • Big Data Analytics • Automation
Designs and deploys production AI assistants, copilots, agents, RAG systems, and intelligent automations. Integrates LLMs with Snowflake, AWS, enterprise applications, and workflow platforms to automate business processes. Builds backend services and serverless workflows, implements observability and responsible AI controls, and scales validated prototypes into reliable production solutions. Partners with technical and business teams to improve decision-making, efficiency, quality, and cost performance.
Top Skills:
AirflowAmazon Api GatewayAmazon BedrockAmazon EventbridgeAmazon S3Amazon SagemakerAnthropicAws LambdaAws Step FunctionsAzure OpenaiDbtGeminiGoMcpMicrosoft Power AutomateMlflowN8NOauthOpenaiPythonRbacSalesforceServicenowSnowflakeSnowflake Cortex SearchSnowflake Ml RegistrySnowparkSsoTypescriptUipathWeights & BiasesWorkato
Aerospace • Artificial Intelligence • Cloud • Machine Learning • Software • Cybersecurity • Defense
Perform software verification and validation for embedded aerospace platform components. Develop test plans, conduct requirement- and scenario-based testing, execute change impact analyses, resolve technical issues, support audits, and collaborate with stakeholders. The role requires embedded C/C++ and RTOS expertise, emulator and debugger experience, testing tools, technical leadership, and stakeholder management.
Top Skills:
BitbucketC++DeosDoors NgEmbedded CIcdJIRALauterbachLdraLinuxRtosVectorcast/Cover
What you need to know about the Mumbai Tech Scene
From haggling for the best price at Chor Bazaar to the bustle of Crawford Market, the energy of Mumbai's traditional markets is a key part of the city's charm. And while these markets will always have their place, the city also boasts a thriving e-commerce scene, ranking among the largest in the region. Driven by online sales in everything from snacks to licensed sports merchandise to children's apparel, the local industry is worth billions, with companies actively recruiting to meet the demands of continued growth.
