AddressNuage is on the lookout for a mid-senior Data Engineer to work with an esteemed Global Consultancy. We're seeking an individual with a passion for crafting robust Data Infrastructure, orchestrating streamlined Data pipelines, and refining Data workflows. You will be responsible for designing, developing, and delivering cutting-edge Data Platform. You will be working across a variety of technologies such as AWS, Kafka/Spark, Snowflake, Airflow, Python, and ETL.
Responsibilities:
- Lead the design, development, and deployment of scalable solutions utilising Big Data technologies, Analytics, and Business Intelligence platforms.
- Creating Airflow Directed Acyclic Graphs (DAGs) from scratch for orchestrating data workflows.
- Utilise Python and SQL proficiently to manipulate and analyse data effectively. Additional experience in languages such as .NET, TypeScript, etc., is a plus.
- Implement and manage data technologies for both batch and streaming processes, such as Kafka, Spark, and data lake or warehouse technologies like Databricks/Snowflake.
- Collaborate with cross-functional teams to integrate solutions with cloud platforms, with a preference for AWS, and DevOps toolchains like GitHub Actions, Terraform, Kubernetes
- Collaborate with stakeholders to understand and address business requirements, ensuring the alignment of solutions with organisational goals and objectives.
Skills:
- Demonstrated expertise on batch and streaming technologies, data lake or warehouse technologies like Databricks/Snowflake, and cloud platforms (preferably AWS).
- Experience in delivering solutions on Big Data technologies, Analytics, and Business Intelligence platforms.
- Advanced skills in SQL for querying and manipulating large datasets.
- Experience with cross-cutting concerns in Data Platforms such as tools like Airflow/Dagster, Datahub/Apache Atlas,
- Strong understanding of modern software delivery practices and processes, including test automation, CI/CD, and incremental and iterative delivery.
- Experience working with ELT tools such as DBT is a plus.
Must have full working rights for Australia (Minimum of Australian Permanent Residency or Australian Citizenship) VISA will not be considered.
If you are a motivated and skilled Data Engineer with a passion for cutting-edge technologies and a desire to contribute to groundbreaking projects, we invite you to apply for this exciting opportunity with Nuage.
For more information on the Australian Technology market, feel free to follow us on LinkedIn: Nuage Technology Group or visit our website www.nuagetg.io
