Address
Form of workSnr Data Engineer | Long Term Contract | Great Flexible team culture | Opportunity to make a difference
About Bi3 Technologies
Bi3 has been recognised for being one of the fastest-growing companies in Australia by Australian Financial Review (FAST 100). We specialise in Data, BI and analytics. Our team has delivered substantial & complex projects for some of the largest organisations around Australia and the globe. We’re quickly building a brand that is well known for superior delivery excellence & impressive track record of creative and efficient solutions.
We have a culture of high-performance in our workplaces and celebrate the diversity of our employees who contribute to the success of our and our client’s organisation.
Background
This role is part of the Data Engineering team that is responsible for building and maintaining the data platform for the personal banking business. The data platform is currently running on AWS and provides data ingestion, transformation, quality, governance and analytics capabilities for various use cases.
The role involves working on a multi-cloud compliance project that aims to ensure the data platform can operate on Azure as well as AWS, in case of any disruptions or outages that affect one of the cloud providers. The candidate will need to design, develop, test and document a solution that can seamlessly failover from AWS to Azure, using metadata driven ingestion frameworks, DBT, CICD and other tools and technologies. The candidate will also need to collaborate with the compliance teams within the bank to demonstrate the functionality and reliability of the solution and obtain the necessary approvals and sign offs. This is a challenging and exciting opportunity to work on a high-impact project that will enhance the resilience and scalability of the data platform and support the business objectives of the personal banking team.
Existing Technology Stack
- A metadata driven ingestion framework built in Python and Pyspark.
- DBT
- AWS S3
- AWS Redshift
- CICD
- Power BI
We are looking for a cloud engineer with Data Engineering skills to join our Personal Banking team and help us migrate our Operational Data Store (ODS) from AWS to Azure platform. You will be responsible for designing, developing, testing and deploying a cloud-based data solution that can handle large volumes of data and support business intelligence and analytics needs. You will also be working independently and collaborating with other engineers, analysts and stakeholders to deliver high-quality outcomes.
- You have at least 3 years of experience in cloud engineering.
- Knowledge of Data Engineering.
- You have strong knowledge and hands-on experience with Azure, Python, Pyspark, Postgres database.
- You have experience with migrating data solutions from AWS to Azure platform, including using Azure Blob to replace S3 and Postgres to replace Redshift.
- You have experience with metadata driven ingestion frameworks, DBT, CICD and multi-cloud failover scenarios.
- You have excellent communication, problem-solving and analytical skills.
- You are self-motivated, proactive, and able to work under pressure and tight deadlines.
