Address
Form of workData Engineer required to build data pipelines to ingest data from source systems as part of a migration to Azure Databricks
Responsibilities:
- partner with Data Analysts and BI Developers to understand requirements
- build, test and implement data pipelines
- gather and transform structured and unstructured data from various systems, presenting it in an easy-to-use way for its relevant audience
- carry out design, code and test reviews
- extensive experience working in an on-prem and Azure Cloud data warehousing environment, with proven background developing ETL solutions
- strong Data Engineering background working across Databricks Lakehouse and Delta Lakes
- expertise developing SQL scripts and stored procedures, and coding in Pyspark
