Senior Data Engineer

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.
If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!

Must have skills:

• 3+ years of hands-on experience in building large-scale data pipelines.

• Strong experience with Spark Streaming, AWS Glue, and EMR for processing both real-time and batch workloads

• Proficiency in Pyspark/Python, including building Kafka producers for data ingestion

• Experience working with Confluent Kafka/spark streaming for streaming ingestion from on-premise sources.

• Solid understanding of AWS services including S3, Redshift, Glue, CloudWatch, and Secrets Manager

• Experience working with Medallion architecture and hybrid data destinations (e.g., Redshift and on-premise Oracle)

• Ability to implement monitoring dashboards and observability for pipelines using tools like CloudWatch or Datadog

• Strong SQL skills for validating data accuracy and developing job-level metrics

• Experience with building alerting mechanisms for pipeline failures and performance issues

• Strong collaboration and communication skills

• Proven ability to take ownership of technical deliverables, driving execution from design to deployment

• Experience in guiding junior developers or data engineers, providing technical direction and code reviews to uphold best practices

• AWS Data Engineering - Associate Certified.

Good to have skills : 

• Experience with orchestration tools like Apache Airflow or AWS Step Functions

• Exposure to Bigdata stacks like Sqoop,HDFS,Hive,Nifi

• Exposure to Terraform for infrastructure automation

• Familiarity with CI/CD pipelines for data workflows