Quantiphi Logo

Quantiphi

Senior Data Engineer

Posted 10 Days Ago
Be an Early Applicant
3 Locations
Senior level
3 Locations
Senior level
Design, implement, and test scalable AWS-based big data ETL/ELT pipelines. Utilize advanced SQL and Python for data warehousing and database optimization while ensuring adherence to architecture requirements.
The summary above was generated by AI

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.
If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!

Requirement Description:

  • Experienced in designing, implementation & testing of highly scalable and cost-efficient AWS based big data ETL/ELT pipeline using PySpark and SQL

  • Well-experienced in data warehousing

  • Well versed in working with transactional databases

Must have

  • Big data engineering technologies like Spark (PySpark)

  • Good knowledge of working with Python Programming

  • Good knowledge of advanced SQL concepts, Optimization techniques

  • Good knowledge of data warehousing, data modeling and concepts like OLAP and OLTP

  • Working knowledge with SQL databases like  Postgresql, etc. is mandatory

  • Good knowledge of Data Migration like bringing on premise database to cloud or one cloud to another

  • Understand Architecture Requirements and ensure effective design, development, validation, and support activities.

  • Understanding of data based AWS services like Glue, EMR, Lambda, DMS, RDS, S3 etc and basic AWS architecture best practices.

  • Knowledge of data pipeline orchestration using Step function or Airflow

  • Ability to use the AWS service APIs, AWS CLI, and SDKs to build applications

Good to have

  •  Knowledge of NoSQL databases like Dynamo can be add on

  • Knowledge of BI tools

  • Knowledge of working with streaming pipelines 

If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!

Top Skills

Airflow
AWS
Dms
Emr
Glue
Lambda
Postgres
Pyspark
Python
Rds
S3
SQL
Step Functions

Similar Jobs

12 Hours Ago
Hybrid
Mumbai, Maharashtra, IND
Senior level
Senior level
Big Data • Food • Hardware • Machine Learning • Retail • Automation • Manufacturing
As a Senior Data Engineer, you'll design cloud-based data solutions, manage data pipelines, ensure data quality, and collaborate with teams to implement analytics strategies.
Top Skills: Ab-InitioAecorsoftAirflowAtaccama DqAzure Data FactoryAzure DatabricksAzure Datalake Gen2Azure Stream AnalyticsAzure Synapse AnalyticsBig QueryCloud FunctionCloudwatchCollibraDatabricksDataflowDataplexDataprocDbtEmrErwinFivetranGcsGlueGoGraph DatabaseInformaticaJavaKafkaKinesisLambdaNeo4JPentahoPl/SqlPubsubPysparkPythonRedshiftS3SnsSQLTalend
9 Days Ago
Hybrid
Pune, Maharashtra, IND
Senior level
Senior level
Artificial Intelligence • Cloud • Sales • Security • Software • Cybersecurity • Data Privacy
As a Senior Data Engineer, you will develop and maintain a robust data platform, collaborating with peers and producing high-quality code with a focus on data ingestion and architecture evolution.
Top Skills: AirflowAWSDbtJavaPythonSnowflakeSparkSQL
11 Days Ago
Remote
Hybrid
2 Locations
Senior level
Senior level
Artificial Intelligence • Cloud • Sales • Security • Software • Cybersecurity • Data Privacy
Seeking a Senior Data/Software Engineer to design and implement ELT processes, maintain data pipelines, and leverage AWS services for data integration.
Top Skills: Apache AirflowAWSDbtDockerFlinkHelmJenkinsKafkaKubernetesKustomizeSnowflakeSparkTerraform

What you need to know about the Mumbai Tech Scene

From haggling for the best price at Chor Bazaar to the bustle of Crawford Market, the energy of Mumbai's traditional markets is a key part of the city's charm. And while these markets will always have their place, the city also boasts a thriving e-commerce scene, ranking among the largest in the region. Driven by online sales in everything from snacks to licensed sports merchandise to children's apparel, the local industry is worth billions, with companies actively recruiting to meet the demands of continued growth.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account