Data Engineer Lead - Databricks

About the Company: We are excited to be partnering with a leading player in the mining sector, dedicated to providing innovative solutions that enhance safety and productivity in mining operations. They are currently seeking a highly skilled and motivated individual to join their team as a Data Engineering Lead. This role is pivotal in shaping their Cloud Data Platform and driving excellence in data management to empower thier client's data-driven decision-making.
Responsibilities:
Design and Develop Cloud Data Platform:

• Architect and develop our Azure-based Cloud Data Platform, adhering to industry best practices.
• Implement robust and scalable data solutions using tools such as Data Factory, Data Bricks, Synapse, and Data Lake.

Lead Data Platform Design and Delivery:

• Coordinate and lead the design and delivery activities for the Azure Data Platform.
• Collaborate with stakeholders to understand business requirements and translate them into technical specifications.

Hands-On Cloud Data Platform Management:

• Lead efforts to build and maintain our client's cloud data platform, including hands-on involvement in information models and solutions.

Vendor and Partner Collaboration:

• Work with external vendors and partners as needed to implement and maintain cloud-based software solutions.
• Ensure third-party solutions align with regulatory requirements and industry standards.

Architecture and Integration Solutions:

• Establish robust and scalable architecture for data ingestion and data integration solutions.
• Oversee the execution of Data Engineering and Data Modeling activities within a modern, agile framework.

Compliance and Standards:

• Ensure that cloud-based software solutions comply with relevant regulatory requirements and industry standards.

Agile Framework Management:

• Manage Data Engineering and Data Modeling activities following a modern, agile framework.
• Ensure alignment with the Data & Analytics team sprint plans.

Requirements:
Education:

• University Degree or equivalent with a relevant focus on Data Engineering.

Technical Skills:

• Significant experience in developing data ingestion solutions in the Azure ecosystem (Data Factory, Data Bricks, Synapse, Data Lake).
• Experience designing and developing data models across various functional areas.
• Proficiency in data profiling, facilitating business-led data cleansing, and implementing Data Quality frameworks.
• Familiarity with CI/CD processes and tools, along with a sound knowledge of software engineering principles.

Methodologies:

• Experience using agile delivery frameworks (e.g., SCRUM / SAFe).

Communication and Leadership:

• Comfortable and confident leading and facilitating data-focused workshops with the business.