Design, build, optimize, and maintain high-performance Spark-based data pipelines using Scala/Java and Hive on Hadoop/CDP. Own full project lifecycle, enforce coding best practices, troubleshoot Spark/Hive/YARN performance, and collaborate with stakeholders to deliver scalable data solutions.
We need a Senior Data Engineer with 10+ years exp proficient in Spark, Scala/Java, and Hive, with extensive hands-on development experience in the Big Data Ecosystem.
Key Responsibilities:
- Design, implement, and optimize highly performant data pipelines using Spark, Scala/Java, and Hive on platforms like Cloudera Data Platform (CDP) or other Hadoop echo systems.
- Take complete ownership of complex data engineering projects within the big data ecosystem, covering the entire lifecycle from initial design and development to deployment and ongoing maintenance.
- Develop robust and efficient Hive queries for extensive data analysis and reporting.
- Champion and enforce best practices and coding standards for new and existing data flows to ensure they are robust, scalable, secure, and maintainable using Spark, Scala/Java, and Hive within the big data ecosystem.
- Diagnose, troubleshoot, and resolve complex issues related to Spark, Scala/Java, and Hive applications and YARN resource management, implementing performance optimization solutions.
- Proactively collaborate with stakeholders, working closely to develop solutions with full commitment and accountability.
Technical Skills & Experience:
- Proven hands-on development expertise with Apache Spark
- Strong programming proficiency in Scala and/or Java
- In-depth knowledge and practical experience with Hive, including query optimization and data analysis.
- Experience with data platforms such as Cloudera Data Platform (CDP) is highly desirable.
Education:
- Bachelor’s / Master's degree/University degree or equivalent experience
Similar Jobs
Artificial Intelligence • Consumer Web • Edtech • Enterprise Web • HR Tech • Social Impact • Generative AI
Own enterprise AI customer deployments from discovery and product scoping through delivery, adoption, and value realization. Translate customer workflows into product requirements, technical solutions, deployment roadmaps, and measurable outcomes. Partner with engineering, sales, customer success, finance, legal, security, and operations while evaluating AI trade-offs, business value, and production readiness. Identify reusable customer needs and help establish scalable product, operating, and deployment playbooks.
Top Skills:
AIAPIsClaudeCloud EnvironmentsCursorIdentity And Access ManagementInfrastructureLlmsLovableRagRetool
Financial Services
Owns delivery and continuous improvement of credit monitoring products within wholesale lending services. Partners with business and technology stakeholders to gather requirements, design solutions, manage the product backlog, coordinate user acceptance testing, support production incidents, and oversee change readiness, risk controls, and regulatory adherence. Develops automated regression testing, resolves operational issues, and drives process and product transformations across cross-functional teams.
Top Skills:
Agile MethodologyAlteryxJIRAExcelMicrosoft PowerpointMicrosoft VisioMicrosoft WordTableau
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Business Analyst supporting retail banking and digital transformation initiatives. Responsibilities include mapping customer onboarding and digital account servicing journeys, documenting business requirements, supporting end-to-end delivery, and collaborating with Product Owners, Technology, Operations, and global stakeholders. The role also involves Agile delivery, including creation of epics and user stories and backlog management using Jira and Confluence.
Top Skills:
ConfluenceJIRA
What you need to know about the Mumbai Tech Scene
From haggling for the best price at Chor Bazaar to the bustle of Crawford Market, the energy of Mumbai's traditional markets is a key part of the city's charm. And while these markets will always have their place, the city also boasts a thriving e-commerce scene, ranking among the largest in the region. Driven by online sales in everything from snacks to licensed sports merchandise to children's apparel, the local industry is worth billions, with companies actively recruiting to meet the demands of continued growth.
