Address
Form of work- Location: Hollywell , Queensland
- Type: Permanent
- Job #16181
- Salary: A$140,000
Our commercial client is looking to engage a Data Engineer on a permanent basis.
Key Responsibilities- Design and implement data pipelines that extract data from multiple sources and load it into a central data store.
- Work with stakeholders to understand business requirements and design data models to meet those requirements.
- Optimize and improve existing data pipelines to ensure high performance and reliability.
- Develop and maintain data quality checks to ensure the accuracy and completeness of data.
- Continuously monitor and maintain data pipelines to ensure they are running smoothly and identify areas for improvement.
- Participate in code reviews and collaborate with other engineers to maintain high-quality code.
- Create visualizations with tools like Power BI.
- Stay current with industry trends and recommend new tools and technologies to improve the data infrastructure.
- Involved in discovering workshops and solutions design activities with customers to elicit requirements and scope projects
- Strong experience with Databricks, or other technologies such as Snowflake, Redshift, and Synapse.
- Experience with AWS and/or Azure.
- Extensive knowledge of ETL processes and data integration.
- Experience with data warehousing, data modeling, data lake, and lake house architecture.
- Strong programming skills in Python, SQL, and Spark.
- Excellent communication and collaboration skills.
- Hands-on experience of Python – development within Pyspark, Lambda, Batch, Glue, DataFlow, etc.
- Experience working with customers to elicit requirements and scope projects
- Data transformation and data migration experience. This includes knowledge of source systems like relational databases, log files, flat files, event streams, etc
- Experience in data manipulation, cleansing, and blending from multiple data sources and exposure to various data types and storage paradigms
- Understanding of Data Governance concepts like MDM, Metadata management, Data Security, Data Quality, etc.
For a confidential chat, please contact Austin on 0429 524 *** or via email at ******@exclaim-it.com.au and we would be more than happy to provide you with additional information and assist in the application process.
Attach a resume file. Accepted file types are DOC, DOCX, PDF, HTML, and TXT.
