Data Engineer

• Sydney or Brisbane (Hybrid 3 days WFO 2 days WFH)
• Salary: $140k - $160k + super 
• Social good Data & AI

Ever considered the advantages of working in a company where every key decision-maker, from top executives to the company's founders, brings a wealth of experience in data? In such an environment, the ability to scale rapidly becomes second nature and the Data Engineering team is the backbone of everything they build.

Having grown from 8 - 100 people in the last few years, this business needs x2 Mid-Senior Data Engineers as they continue to grow. 

“We’re a social good, Data & AI platform that acts as a profit center for our customers. As you can imagine, customer retention and advocacy is high” 

That’s how the CTO introduced the business to us.

From architecting robust data pipelines to optimising performance and scalability, you'll play a pivotal role in shaping the future of their platform.

What you’ll be doing…

• Create and maintain optimal data pipeline architecture
• Assemble large, complex data sets that meet functional/ non-functional business requirements
• Identify, design and implement internal process improvements: automating manual processes, optimising data delivery, re-designing infrastructure for greater scalability
• Build the infrastructure required for optimal ETL of data from a variety of data sources using SQL, Spark and other AWS 'big data' technologies
• Build analytics tools that utilise the data pipeline to provide actionable insights

What you’ll need:

• Bachelor's or Master's degree in Mathematics, Software Engineering, or Computer Engineering, providing a solid foundation in mathematical principles, software development methodologies, and computer architecture.
• 3-5+ years of python experience for data manipulation, automation, ensuring code readability and efficiency
• Hands-on experience with AWS services: Glue, Lambda, S3, SNS, Step Functions, CloudWatch, EMR, Batch
• Proven track record of designing and implementing robust data pipelines capable of handling large volumes of data with scalability and efficiency.
• Experience with big data technologies such as Hadoop ecosystem (HDFS, MapReduce, HBase), Spark, Hive, and Presto, for processing and analyzing large volumes of structured and unstructured data efficiently.
• Proficiency in web scraping techniques to extract data from diverse sources across the internet, including websites, APIs, and other online repositories (desirable). 

You will need full working rights in Australia. 

If that sounds like you hit apply or get in touch with Ailbhe at The Onset for a confidential chat - ******@theonset.com.au