The Senior Data Engineer will lead the design and development of data pipelines, mentor junior engineers, and enhance data engineering practices using frameworks and best practices.
Organization: At CommBank, we never lose sight of the role we play in other people’s financial wellbeing. Our focus is to help people and businesses move forward to progress. To make the right financial decisions and achieve their dreams, targets, and aspirations. Regardless of where you work within our organisation, your initiative, talent, ideas, and energy all contribute to the impact that we can make with our work. Together we can achieve great things
Job Title: Senior Data Engineer
Location: Bangalore-Manyata Tech Park (Hybrid)
Business & Team: Retail Technology
Impact & contribution
- Contribute to internal online discussion around engineering, delivery and technology e.g. blog posts, knowledge base articles.
- Deliver high quality, sustainable solutions to meet business requirements, leveraging approved delivery frameworks and by applying industry best practice.
- Develop reusable frameworks and tooling to accelerate data engineering maturity.
- Develop strategic data programs and data products.
- Identify and drive opportunities for continuous improvement within the team and in delivery of products.
- Lead the design and development of highly reliable and scalable data pipelines and data platforms with comprehensive test coverage.
- Participate in domain planning, providing guidance to ensure that technical deliveries are aligned to engineering direction and strategy.
- Partner with teams to engineer improvements to the scaling and reliability of our services.
- Provide data engineering practice consultancy across platforms, products and delivery teams.
- Provide thought leadership, mentorship and up-skilling to other engineering teams.
- Take responsibility for nominated technical assets related to areas of expertise including roadmaps and technical direction.
- Actively participate in On-Call rosters from time to time to own resolution of any critical issues, ensuring the seamless operation of our systems, products and platforms.
- Architect and implement scalable, secure, and high-performance data pipelines using technologies such as Spark, Python, SQL, and AWS
- Lead the development and optimisation of ETL/ELT processes for structured and unstructured data sources
- Collaborate with data scientists, analysts, and product teams to understand data requirements and translate them into technical solutions
- Ensure data quality, lineage, and governance across all stages of the data lifecycle.
- Drive adoption of engineering best practices including CI/CD, DevSecOps, and observability tooling
- Provide technical leadership and mentorship to junior engineers, fostering a culture of continuous learning and improvement
- Participate in hiring, onboarding, and bar-raiser interviews for senior and staff-level engineering roles
- Contribute to the development of semantic data models and data product lifecycle frameworks
Roles & Responsibilities:
- Participate in on-call rotations to provide 24/7 support for critical systems
- Maintain and enhance the productivity of data delivery and its processes.
- Build strategy to migrate ETL Load from legacy systems to cloud.
- Manage the hyper care period
- Support platform engineer to improve platform resilience, reduce incident rates, and optimize solution performance as well as reduce operations cost
Essential Skills:
- 10+ years of experience in data engineering, with at least 2 years in a staff or lead role
- Proficiency in distributed computing frameworks (e.g., Spark, Hadoop), cloud platforms (AWS preferred), and data orchestration tools.
- Strong programming skills in Python, Scala, or Java.
- Experience with data modelling, data warehousing, and advanced storage technologies (e.g., NoSQL, HDFS, Graph Stores)
- Experience in AWS cloud (Certification Preferred)
- Familiarity with observability tools like CloudWatch, PagerDuty, and Tableau
- Familiarity with implementation of Amazon RDS services
- Excellent communication and stakeholder management skills.
- Sound experience in building data pipelines & ETL tools - AWS Data Stack (Ab Initio is good to have)
- Experience in designing and developing ETL test scripts for structured and unstructured data
- Sound skills testing solutions with large data volumes and complex data transformation rules
- Sound Leadership and behavioural skills including Active Listening, Constructive Challenge, Problem Solving and Risk management
- Sound in AI and Data Literacy with an experience and ability to use AI Tools into development lifecycle
Education Qualifications
- Bachelor's or master’s degree in computer science, Engineering, or related field
If you're already part of the Commonwealth Bank Group (including Bankwest, x15ventures), you'll need to apply through Sidekick to submit a valid application. We’re keen to support you with the next step in your career.
We're aware of some accessibility issues on this site, particularly for screen reader users. We want to make finding your dream job as easy as possible, so if you require additional support please contact HR Direct on 1800 989 696.
Top Skills
Ab Initio
Amazon Rds
AWS
Cloudwatch
Graph Stores
Hadoop
Hdfs
Java
NoSQL
Pagerduty
Python
Scala
Spark
SQL
Tableau
Similar Jobs
Enterprise Web • Fintech • Financial Services
As a Senior Data Engineer, you'll design and maintain data pipelines, implement data models, and ensure data governance while collaborating across teams to deliver insights and improve data quality.
Top Skills:
AirflowDockerKafkaPower BIPythonSnowflakeSQLTableau
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
The Senior Data Engineer designs and develops complex data pipelines, ensures data reliability, and follows engineering best practices across enterprise-scale Big Data environments.
Top Skills:
HadoopHiveJavaNetezzaOraclePythonScalaSparkSQL
Artificial Intelligence • Cloud • Sales • Security • Software • Cybersecurity • Data Privacy
Seeking a Senior Data Engineer to design and implement ELT processes, build reliable data pipelines, and utilize AWS for data solutions.
Top Skills:
Apache AirflowAWSDbtDockerFlinkHelmJenkinsKafkaKubernetesKustomizeSnowflakeSparkTerraform
What you need to know about the Mumbai Tech Scene
From haggling for the best price at Chor Bazaar to the bustle of Crawford Market, the energy of Mumbai's traditional markets is a key part of the city's charm. And while these markets will always have their place, the city also boasts a thriving e-commerce scene, ranking among the largest in the region. Driven by online sales in everything from snacks to licensed sports merchandise to children's apparel, the local industry is worth billions, with companies actively recruiting to meet the demands of continued growth.
