Data Engineer - DatabricksDo you thrive in building large-scale data pipelines that power game-changing analytics? Are you passionate about utilizing Databricks to unlock the potential of big data? If so, we want to hear from you!We're seeking a talented Data Engineer to join our growing team and play a key role in designing, developing, and deploying robust data pipelines on the Databricks platform. You'll work closely with data scientists and analysts to transform raw data into actionable insights, enabling data-driven decision making across the organization.Responsibilities:
- Design, develop, and deploy scalable data pipelines using Apache Spark and Databricks
- Work with various data sources (structured, semi-structured, unstructured) to ingest, cleanse, and transform data
- Optimize data pipelines for performance and efficiency
- Implement best practices for version control, code documentation, and unit testing
- Collaborate with data scientists and analysts to understand data requirements and translate them into technical solutions
- Monitor and maintain data pipelines to ensure data quality and availability
- Stay up-to-date on the latest advancements in big data technologies and Databricks
- 3+ years of experience as a Data Engineer or similar role
- Proven experience with Apache Spark and Databricks
- Strong understanding of Data Engineering principles and best practices
- Experience with data warehousing concepts (e.g., data modeling, dimensional modeling)
- Familiarity with cloud platforms (AWS, Azure, GCP) a plus
- Excellent written and verbal communication skills
- Ability to work independently and as part of a cross-functional team