Lead healthcare data engineering projects by designing and implementing real-time data pipelines, ensuring compliance, and mentoring junior engineers.
Job Title: SSE – Pyspark, Python, and Azure Databricks (Healthcare Data Project)
Experience: 5 to 8 years
Role Overview:
We are looking for a highly skilled with expertise in Kafka, Python, and Azure Databricks (preferred) to drive our healthcare data engineering projects. The ideal candidate will have deep experience in real-time data streaming, cloud-based data platforms, and large-scale data processing. This role requires strong technical leadership, problem-solving abilities, and the ability to collaborate with cross-functional teams.
Key Responsibilities:
- Lead the design, development, and implementation of real-time data pipelines using Kafka, Python, and Azure Databricks.
- Architect scalable data streaming and processing solutions to support healthcare data workflows.
- Develop, optimize, and maintain ETL/ELT pipelines for structured and unstructured healthcare data.
- Ensure data integrity, security, and compliance with healthcare regulations (HIPAA, HITRUST, etc.).
- Collaborate with data engineers, analysts, and business stakeholders to understand requirements and translate them into technical solutions.
- Troubleshoot and optimize Kafka streaming applications, Python scripts, and Databricks workflows.
- Mentor junior engineers, conduct code reviews, and ensure best practices in data engineering.
- Stay updated with the latest cloud technologies, big data frameworks, and industry trends.
Required Skills & Qualifications:
- 4+ years of experience in data engineering, with strong proficiency in Kafka and Python.
- Expertise in Kafka Streams, Kafka Connect, and Schema Registry for real-time data processing.
- Experience with Azure Databricks (or willingness to learn and adopt it quickly).
- Hands-on experience with cloud platforms (Azure preferred, AWS or GCP is a plus).
- Proficiency in SQL, NoSQL databases, and data modeling for big data processing.
- Knowledge of containerization (Docker, Kubernetes) and CI/CD pipelines for data applications.
- Experience working with healthcare data (EHR, claims, HL7, FHIR, etc.) is a plus.
- Strong analytical skills, problem-solving mindset, and ability to lead complex data projects.
- Excellent communication and stakeholder management skills.
Top Skills
Azure Databricks
Docker
Kafka
Kubernetes
NoSQL
Python
SQL
Similar Jobs
Cloud • Information Technology • Security • Software
The Staff Quality Engineer will lead quality initiatives, engage in SDLC processes, architect testing strategies, and mentor teams in quality standards and automation.
Top Skills:
AWSCypressGherkinPytestPython
.jpeg)
Artificial Intelligence • Hardware • Information Technology • Machine Learning
Responsible for maintaining, calibrating, and troubleshooting lab equipment in a semiconductor environment. Lead preventive maintenance and implement global initiatives.
Top Skills:
EsdIso/Iec 17025KaizenLean Six SigmaLinuxWindows
Artificial Intelligence • Hardware • Information Technology • Machine Learning
The GEL Engineer leads maintenance and calibration of lab equipment, troubleshooting problems, conducting failure analysis, and coordinating with teams to ensure compliance with safety regulations and quality standards.
Top Skills:
Iso/Iec 17025LinuxWindows
What you need to know about the Mumbai Tech Scene
From haggling for the best price at Chor Bazaar to the bustle of Crawford Market, the energy of Mumbai's traditional markets is a key part of the city's charm. And while these markets will always have their place, the city also boasts a thriving e-commerce scene, ranking among the largest in the region. Driven by online sales in everything from snacks to licensed sports merchandise to children's apparel, the local industry is worth billions, with companies actively recruiting to meet the demands of continued growth.
