NielsenIQ Logo

NielsenIQ

Principal Data Engineer

Posted Yesterday
Be an Early Applicant
Hybrid
Pune, Mahārāshtra
Expert/Leader
Hybrid
Pune, Mahārāshtra
Expert/Leader
Provides technical leadership across data engineering, data science, and software teams. Designs scalable Azure and Databricks data platforms, pipelines, transformations, and ML workflows using Spark, Delta Lake, and Airflow. Drives architecture, governance, security, CI/CD, testing, DevOps, and cloud-native practices. Leads hands-on development, mentors engineers, manages performance and career growth, supports hiring, and coordinates with stakeholders to deliver maintainable, production-grade data solutions.
The summary above was generated by AI
Company Description

NIQ is the world’s leading consumer intelligence company, delivering the most complete understanding of consumer buying behavior and revealing new pathways to growth. In 2023, NIQ combined with GfK, bringing together the two industry leaders with unparalleled global reach. With a holistic retail read and the most comprehensive consumer insights—delivered with advanced analytics through state-of-the-art platforms—NIQ delivers the Full View™. NIQ is an Advent International portfolio company with operations in 100+ markets, covering more than 90% of the world’s population.

Job Description

We are looking for a Principal Software Engineer to provide technical leadership within our development teams, ensuring best practices, architectural coherence, and effective collaboration across projects. This role is ideal for a highly experienced engineer who can bridge the gap between data engineering, data science, and software engineering, helping teams build scalable, maintainable, and well-structured data solutions. 

As a Principal Software Engineer, you will play a hands-on role in designing and implementing solutions while mentoring developers, influencing technical direction, and driving best practices in software and data engineering. This role includes line management responsibilities, ensuring the growth and development of team members. The role will be working within an Azure and Databricks environment, leveraging the power of cloud-native technologies and modern data platforms. 

Key Responsibilities 

Technical Leadership & Architecture 

  • Act as a technical architect, ensuring alignment between the work of multiple development teams in data engineering and data science. 
  • Design scalable, high-performance data processing solutions within Azure and Databricks, considering factors such as governance, security, and maintainability. 
  • Drive the adoption of best practices in software development, including CI/CD, testing strategies, and cloud-native architecture. 
  • Work closely with Product Owners and Engineering Managers to translate business needs into technical solutions. 

Hands-on Development & Technical Excellence 

  • Lead by example through high-quality coding, code reviews, and proof-of-concept development. 
  • Solve complex engineering problems and contribute to critical design decisions. 
  • Ensure effective use of Azure services, Databricks, and data processing frameworks such as Apache Spark, Delta Lake, and Airflow. 
  • Develop and optimise data pipelines, data transformations, and ML workflows in a cloud environment. 

Line Management & Team Development 

  • Provide line management to engineers, ensuring their professional growth and development. 
  • Conduct performance reviews, set development goals, and mentor team members to enhance their skills. 
  • Foster a collaborative and high-performing engineering culture, promoting knowledge sharing and continuous improvement. 
  • Support hiring, onboarding, and career development initiatives within the engineering team. 

Collaboration & Cross-Team Coordination 

  • Act as the technical glue between data engineers, data scientists, and software developers, ensuring smooth integration of different components. 
  • Provide mentorship and guidance to developers, helping them level up their skills and technical understanding. 
  • Work with DevOps teams to improve deployment pipelines, observability, and infrastructure as code. 
  • Engage with stakeholders across the business, translating technical concepts into business-relevant insights. 

Governance, Security & Data Best Practices 

  • Champion data governance, lineage, and security across the platform. 
  • Advocate for and implement scalable data architecture patterns, such as Data Mesh, Lakehouse, or event-driven pipelines. 
  • Ensure compliance with industry standards, internal policies, and regulatory requirements. 

Qualifications

Requirements & Experience 

  • 10+ years of strong software development experience required
  • Strong software engineering background with experience in designing and building production-grade applications in Python, Scala, Java, or similar languages. 
  • Proven experience with Azure-based data platforms, specifically Databricks, Azure Data Lake, Azure Synapse, and Azure Functions. 
  • Expertise in Apache Spark and Delta Lake, with experience building large-scale distributed data pipelines. 
  • Experience with workflow orchestration tools like Apache Airflow. 
  • Cloud experience in Azure, including containerisation (Docker, Kubernetes) and infrastructure as code (Terraform, Bicep). 
  • Strong knowledge of modern software architecture, including microservices, event-driven systems, and distributed computing. 
  • Experience leading teams in an agile environment, with a strong understanding of CI/CD pipelines, automated testing, and DevOps practices. 
  • Excellent problem-solving and communication skills, with the ability to engage with both technical and non-technical stakeholders. 
  • Previous experience in data engineering, data science, or analytics-heavy environments is highly desirable. 
  • Proven line management experience, including mentoring, career development, and performance management of engineering teams. 

Additional Information

Our Benefits

  • Flexible working environment
  • Volunteer time off
  • LinkedIn Learning
  • Employee-Assistance-Program (EAP)

NIQ may utilize artificial intelligence (AI) tools at various stages of the recruitment process, including résumé screening, candidate assessments, interview scheduling, job matching, communication support, and certain administrative tasks that help streamline workflows. These tools are intended to improve efficiency and support fair and consistent evaluation based on job-related criteria. All use of AI is governed by NIQ’s principles of fairness, transparency, human oversight, and inclusion. Final hiring decisions are made exclusively by humans. NIQ regularly reviews its AI tools to help mitigate bias and ensure compliance with applicable laws and regulations. If you have questions, require accommodations, or wish to request human review were permitted by law, please contact your local HR representative. For more information, please visit NIQ’s AI Safety Policies and Guiding Principles: https://nielseniq.com/global/en/info/niqs-ai-safety-policies/

About NIQ

NIQ is the world’s leading consumer intelligence company, delivering the most complete understanding of consumer buying behavior and revealing new pathways to growth. In 2023, NIQ combined with GfK, bringing together the two industry leaders with unparalleled global reach. With a holistic retail read and the most comprehensive consumer insights—delivered with advanced analytics through state-of-the-art platforms—NIQ delivers the Full View™. NIQ is an Advent International portfolio company with operations in 100+ markets, covering more than 90% of the world’s population.

For more information, visit NIQ.com

Want to keep up with our latest updates?

Follow us on: LinkedIn | Instagram | Twitter | Facebook

Our commitment to Diversity, Equity, and Inclusion

At NIQ, we are steadfast in our commitment to fostering an inclusive workplace that mirrors the rich diversity of the communities and markets we serve. We believe that embracing a wide range of perspectives drives innovation and excellence.  All employment decisions at NIQ are made without regard to race, color, religion, sex (including pregnancy, sexual orientation, or gender identity), national origin, age, disability, genetic information, marital status, veteran status, or any other characteristic protected by applicable laws. We invite individuals who share our dedication to inclusivity and equity to join us in making a meaningful impact. To learn more about our ongoing efforts in diversity and inclusion, please visit the https://nielseniq.com/global/en/news-center/diversity-inclusion

Similar Jobs

One Month Ago
In-Office
Expert/Leader
Expert/Leader
Logistics • Transportation • Design • Automation • Manufacturing
The Principal Data Engineer oversees the design and development of data pipelines, optimizing data engineering practices, leading teams, and ensuring high-quality data products using cloud technologies.
Top Skills: AirflowApache FlinkApache KafkaAws AthenaAws EmrAws GlueDatabricksDbtGitGitlabPythonScalaSparkSQLTerraformUnity Catalog
2 Months Ago
Hybrid
Senior level
Senior level
Information Technology
The Lead Data Engineer will focus on Azure Data Engineering, utilizing skills in AKS, Event Hub, Cosmos DB, and Azure DevOps to design and implement data solutions.
Top Skills: AksAzureAzure DevopsAzure FunctionsCosmos DbEvent HubMs Fabric
Yesterday
Remote or Hybrid
India
Expert/Leader
Expert/Leader
Fintech • Legal Tech • Software • Financial Services • Cybersecurity • Data Privacy
Oversee private equity fund accounting, including NAV calculations, capital accounts, waterfalls, carried interest, fees, expenses, transactions, valuations, and financial reporting. Review statements, investor packages, audits, tax-related information, and regulatory compliance. Lead and develop fund accounting teams, manage stakeholders, improve controls and processes, and support Investran or Geneva implementations.
Top Skills: ExcelGenevaIfrsInvestranLux GaapUs Gaap

What you need to know about the Mumbai Tech Scene

From haggling for the best price at Chor Bazaar to the bustle of Crawford Market, the energy of Mumbai's traditional markets is a key part of the city's charm. And while these markets will always have their place, the city also boasts a thriving e-commerce scene, ranking among the largest in the region. Driven by online sales in everything from snacks to licensed sports merchandise to children's apparel, the local industry is worth billions, with companies actively recruiting to meet the demands of continued growth.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account