Yadimen Consulting Limited Logo

Yadimen Consulting Limited

Data Engineer — PySpark + AWS Glue

Posted 2 Days Ago
Be an Early Applicant
In-Office
Pune, Mahārāshtra
Senior level
In-Office
Pune, Mahārāshtra
Senior level
Design, develop, optimize, and maintain scalable ETL pipelines using PySpark and AWS Glue. Build data ingestion and transformation workflows, orchestrate processes with AWS Step Functions, and develop Python-based AWS Lambda functions. Support AWS data lake architectures, analytical use cases, monitoring, reliability, and performance optimization. The role also involves SQL and may include Java microservices, REST APIs, and backend integrations.
The summary above was generated by AI

Data Engineer — PySpark + AWS Glue

ETL & Data Pipelines | 2 Openings

 

 

Location: Remote (UK)

Employment Type: Contract / Permanent

Experience Level: 4–8 Years

Openings: 2

 

About the Role

We are looking for a skilled Data Engineer with solid hands-on experience in PySpark, AWS Glue, and ETL development to build and maintain scalable, production-grade data pipelines on AWS. You will be part of a delivery-focused team working on complex data engineering challenges, contributing to data lake architecture and end-to-end pipeline development.



Requirements

Key Responsibilities

       Design, develop, and maintain scalable ETL pipelines using PySpark and AWS Glue

       Build and optimise data ingestion, transformation, and loading workflows

       Orchestrate data workflows using AWS Step Functions

       Develop serverless functions using AWS Lambda (Python)

       Work with data lake architectures on AWS to support analytical use cases

       Ensure pipeline reliability, monitoring, and performance optimisation

 

Required Skills & Experience

       Strong hands-on experience with PySpark and AWS Glue

       Proven track record in ETL pipeline development and optimisation

       Experience orchestrating workflows with AWS Step Functions

       Proficiency in serverless development using AWS Lambda (Python)

       Good SQL skills and a solid understanding of data processing principles

 

Nice to Have

       Exposure to Java-based microservices

       Understanding of REST APIs and backend service integrations

       Experience working with AWS-based data lakes



Similar Jobs

39 Minutes Ago
Hybrid
Senior level
Senior level
Big Data • Fintech • Information Technology • Business Intelligence • Financial Services • Cybersecurity • Big Data Analytics
Design, develop and deliver features for TransUnion UK consumer products using modern engineering practices. Collaborate with global and cross-functional teams, ensure code quality, testability and CI/CD, follow architectural standards, mentor junior engineers, and participate in QA and production readiness.
Top Skills: Asp.NetAsp.Net CoreC#Ci/CdGitNunitOctopus DeploySeleniumSpecflowTfs
An Hour Ago
Remote or Hybrid
2 Locations
Junior
Junior
Financial Services
Develops secure, scalable full-stack software for JPMorgan Chase’s Asset and Wealth Management Data and Analytics team. Responsibilities include coding with Java, React, TypeScript, Python, and AWS; designing, testing, debugging, and troubleshooting applications; analyzing data; supporting operational stability; and applying agile, CI/CD, security, resiliency, and AI-assisted development practices.
Top Skills: AgileAi-Assisted Development ToolsAWSCi/CdJavaPythonReactSpring BootSpring JpaTypescript
9 Hours Ago
In-Office
Senior level
Senior level
Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
Leads end-to-end product design for complex web and mobile experiences, from discovery through delivery and continuous improvement. Partners with product, engineering, research, and business teams to translate insights into user-centered solutions. Creates flows, wireframes, prototypes, and high-fidelity designs; advocates for accessibility and inclusive design; evolves design systems; influences product strategy; supports implementation quality; and mentors other designers.
Top Skills: AgileAi-Enabled Design ToolsDesign SystemsFigmaMobile ApplicationsResponsive Web Applications

What you need to know about the Mumbai Tech Scene

From haggling for the best price at Chor Bazaar to the bustle of Crawford Market, the energy of Mumbai's traditional markets is a key part of the city's charm. And while these markets will always have their place, the city also boasts a thriving e-commerce scene, ranking among the largest in the region. Driven by online sales in everything from snacks to licensed sports merchandise to children's apparel, the local industry is worth billions, with companies actively recruiting to meet the demands of continued growth.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account