AddressExperis have the pleasure to announce a partnership with a global consultancy in their search for an experienced Data Engineer. This consultancy focuses on accelerating their customers migration to cloud and aims to digitalize and automate their operations. You will have the opportunity to work within a dynamic team focused on developing large-scale data warehouse solutions in Azure platforms. On offer is a contract role with a competitive daily rate, located in Eastern Melbourne.
As a Data Engineer you will be responsible for designing and developing scalable data pipelines using Azure Services such as Azure Data Factory, Azure Synapse Analytics and Databricks. Additionally, you will have the opportunity to collaborate with data scientists and analysts to deploy machine learning models and enable advanced analytics capabilities using DataBricks.
Your other responsibilities will also include;
- Implement data ingestion processes from various sources including structured and unstructured data, APIs, and streaming data sources.
- Optimize data processing and transformation workflows for performance and efficiency.
- Ensure data quality and integrity through data validation, cleansing, and enrichment processes.
- Implement and enforce data governance policies and security controls to protect sensitive data assets.
- Monitor and troubleshoot data pipelines to identify and resolve issues in a timely manner.
To be successful in this position you will require the below skills;
- Proficiency in programming languages such as Python (Pyspark) and Spark SQL for data processing and manipulation.
- Experience with data modelling, schema design, Databricks Delta Lake and Delta Live Tables, and optimization techniques.
- Familiarity with DevOps practices and tools for CI/CD pipeline automation is a plus.
- Excellent problem-solving skills and ability to work effectively in a dynamic environment with tight deadlines.
- Strong experience with Azure Data Factory, Azure Synapse Analytics (formerly SQL Data Warehouse), and Databricks.
If you're interested in applying for this role or would like to find out more information please contact Marcin Seder at or click the apply button below.
Marcin Seder
Senior Consultant
************@au.experis.com
ManpowerGroup is committed to being a Diversity Confident Recruiter and encourages applications from people from a diverse range of backgrounds, including people with a disability. Please indicate your preferred method of communication in your resume and please let us know if you require any reasonable adjustments should you be contacted for an interview.
Aboriginal and Torres Strait Islander people are encouraged to apply.
By submitting your resume and other personal information with this application you are consenting to this information being collected in line with our privacy policy. Follow the link to learn more - www.manpowergroup.com.au/privacy-policy
Experis Pty Ltd is a wholly owned subsidiary of ManpowerGroup
State: QLD, licensee/s Manpower Services (Australia) Pty Ltd, LHL-02026-D5L4Q. State: QLD, licensee/s Experis Pty Ltd, LHL-02014-Y5F6D. State: SA, licensee/s Manpower Services (Australia) Pty Ltd, LHS 288856
