Similar Jobs
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
The Principal Data Engineer leads technical strategy and architecture for enterprise data programs, ensuring solution leadership, engineering standards, and scalability.
Top Skills:
HadoopJavaPythonScalaSparkSQL
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
The role involves validating the integrity of data pipelines in Big Data environments, using SQL, Spark testing, and automation to ensure high data quality.
Top Skills:
HadoopHiveNifiOraclePythonScalaShell ScriptingSparkSQLUnix
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
The Data Engineer will build complex data pipelines, implement DW projects in a Big Data environment, and work with a variety of databases. Strong skills in Python, PySpark, and Agile methodologies are essential.
Top Skills:
AirflowClouderaHadoopJavaNetezzaOraclePysparkPythonScalaSQL
Organization: At CommBank, we never lose sight of the role we play in other people’s financial wellbeing. Our focus is to help people and businesses move forward to progress. To make the right financial decisions and achieve their dreams, targets, and aspirations. Regardless of where you work within our organisation, your initiative, talent, ideas, and energy all contribute to the impact that we can make with our work. Together we can achieve great things
Job Title: Data Engineer
Location: Bangalore-Manyata Tech Park (Hybrid)
Business & Team: Financial Crime Technology – Enterprise Data Platform
Impact & contribution:
You will work with Product Owner /lead engineer in the squad working towards impact of Enterprise Transformation Program across key applications.
Roles & Responsibilities:
- In your role as a Data Engineer, you will be reporting to a Data Engineering Manager
- Utilizing Big Data technologies to solve our customer data centric problems.
- Working closely with the Data engineer, Solution Designers and Business Analysts to understand and perform Data Ingestion, Enrichment and Egression
- Working across the Group Data Warehouse (GDW), Big Data Platform (Omnia) & AWS Cloud Platform
- Perform testing activities involved with End-to-End testing, System Testing and UAT testing.
- Analyse current state of testing within a specific platform and identify opportunities for initiating or increasing automation.
- To drive and optimize the Continuous Integration framework, using test automation against builds across multiple development, test, staging and production environments.
- Develop Automation feasibility reports for different applications and systems based on the technical analysis, complexity and reusability.
- Participate in the full lifecycle of agile projects (ideation through implementation).
- Building group data products by integrating diverse data from hundreds of internal and external sources
- Adopting tools, programming languages and templates to improve our data quality and efficiency
- Building and optimizing Big Data pipelines, architecture and data sets
- End to end ownership and accountability of the business requirements, identify technical / performance issues and solve them.
- Learn and adapt to new technologies to solve problems and improve existing solutions
- Required to maintain and build excellent relationships with CDAO peers and stakeholders as well as actively contribute to business outcomes and team culture
Essential Skills:
- 5+ Years of Experience in ETL/Bigdata Automation testing.
- In-depth knowledge of Testing principle, methodologies, and best practices.
- Sound understanding of Automation Framework - Using Python or JAVA.
- Minimum of 3+ years of hands-on experience in Software Engineering or development to build solution in Hadoop, Hive and Spark technologies.
- Minimum 3+ years of hands-on experience in any of the language (Scala/Python) with spark ETL development.
- Minimum 3+ years of hands-on experience in any of them (Teradata / Oracle / SQL Server / MySQL) with strong SQL expertise
- Minimum 3+ years of hands-on experience in any of the CI/CD tools like Intellij teamcity & Github cloud action.
- Minimum 3+ years of hands-on experience in any of the code maintenance repository like Github & bit bucket
- Minimum of 3+ years of experience on Shell Scripting.
- Good Understanding of AWS services.
- Good to have hands-on experience in any ETL Tool like Ab initio GDE/BRE
- Good Understanding of Distributed system and computing
- Good to have hands on experience on Performance Tuning
- Should be aware of AGILE processes and methodologies
- A natural ability to effectively communicate, educate and influence the different stakeholders/ peers, to solve challenging problems, while working in collaboration with other cross-functional engineering and solution design teams
- A positive desire to contribute towards initiatives, along with a strong ‘can- do’ attitude in juggling between constantly changing priorities
- Flexible to cross trained to organisational adopted tools & technologies
- Good to have hands-on experience in any ETL Tool Ab initio GDE/BRE
- Prior experience on Banking Domain / Financial Services related projects would be a great value add
- Hands on experience in any cloud platform like AWS/Azure/GCP is a great value add
Educational Skills:
- A degree in Engineering / Computer Science or a related discipline
If you're already part of the Commonwealth Bank Group (including Bankwest, x15ventures), you'll need to apply through Sidekick to submit a valid application. We’re keen to support you with the next step in your career.
We're aware of some accessibility issues on this site, particularly for screen reader users. We want to make finding your dream job as easy as possible, so if you require additional support please contact HR Direct on 1800 989 696.
What you need to know about the Mumbai Tech Scene
From haggling for the best price at Chor Bazaar to the bustle of Crawford Market, the energy of Mumbai's traditional markets is a key part of the city's charm. And while these markets will always have their place, the city also boasts a thriving e-commerce scene, ranking among the largest in the region. Driven by online sales in everything from snacks to licensed sports merchandise to children's apparel, the local industry is worth billions, with companies actively recruiting to meet the demands of continued growth.
