The Big Data Lead will design and develop data pipelines, improve EY.ai Data Marketplace's services, and ensure their robustness while collaborating across teams.
Check the JD for details however the primary skills are - Spark expert who have used PDF unstructured data and mapped them into tables, ADB, ADF, strong SQL, Python / Pyspark Job description EYDM is seeking a proactive, dynamic, and adaptable member to join our global, diverse, and inclusive team. Given the continuous growth and evolution of our products, we require candidates who not only embrace change but also possess the experience and capability to hit the ground running. Ideal team members will thrive in ambiguous environments, demonstrating curiosity and proactiveness while effectively contributing from day one. Reporting into the Technical Lead, the new member will play a pivotal role in supporting the Data Ingestion team, Business and Technical Product Owners, to expand and promote our offering throughout EY’s Service Lines, Products, Functions and engagement teams. This role covers a broad area of activity, including technical design and development of near real time data processing, consumer interaction, business documentation and reporting. Your key responsibilities You will be responsible for: Collaboratively identify and ideate opportunities to continuously improve EY.ai Data Marketplace (EYDM) data asset and the services it provides to consumers. Engage in design and development sessions to further EYDM’s data asset creation and ingestion pipelines with a focus on stability, optimisation and traceability. Develop, test and maintain EYDM’s data pipelines & processes. Developing and testing resilient, fault tolerant, modular code, to ensure EYDM’s processes, pipelines and services are robust and highly available. Research, explore and if feasible implement new methods and processes across EYDM to build world class data products and associated services. Actively participate in cross-team collaboration to ensure smooth transition through Ingestion, Consumption and Activation
Top Skills
Adb
Adf
Pyspark
Python
Spark
SQL
Hexaware Navi Mumbai, Maharashtra, IND Office
#1, Sector III, Millennium Business Park,, 'A' Block , TTC Industrial Estate, Mahape, Navi Mumbai, Maharashtra , India, 400710
Similar Jobs
Fintech • Information Technology • Insurance • Financial Services • Big Data Analytics
Lead development and maintenance of Big Data solutions for Disability & Absence products, ensuring high-quality, efficient, and scalable applications.
Top Skills:
AzureGCPHadoopHbaseHiveIn-Memory Data ProcessingKafkaNifiNoSQLPigPythonScalaShell ScriptSolrSpark
Information Technology • Consulting
Lead development of big data solutions, focusing on text data processing, data pipelines, and cloud services, primarily in AWS and GCP.
Top Skills:
AWSDockerElasticsearchGCPHadoopHbaseKubernetesMongoDBPythonSpark
Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
As Associate Manager - SES, you will oversee maintenance engineering tasks, support project execution, ensure compliance with GMP, and collaborate on safety initiatives. You will also perform Root Cause Failure Analysis and manage work orders.
Top Skills:
ExcelProcess Automation Systems
What you need to know about the Mumbai Tech Scene
From haggling for the best price at Chor Bazaar to the bustle of Crawford Market, the energy of Mumbai's traditional markets is a key part of the city's charm. And while these markets will always have their place, the city also boasts a thriving e-commerce scene, ranking among the largest in the region. Driven by online sales in everything from snacks to licensed sports merchandise to children's apparel, the local industry is worth billions, with companies actively recruiting to meet the demands of continued growth.
