Data Engineer

We are looking for a curious and creative Data Engineer to join our growing team of data experts. You will be responsible for our day to day data operations, rebuild and optimization of data pipelines on premise and migration to cloud, data supplier refreshes, and will ensure our data architecture is consistent throughout ongoing projects.
You must be self-directed and comfortable supporting the data needs of multiple teams, systems and products. We need someone who will be excited by the prospect of supporting and optimizing our company's data architecture to support our next generation of products and data initiatives.
What you'll need to bring to the party

• 
  Expert level SQL and T-SQL scripting and experience working with Microsoft SQL Server relational databases, as well as working familiarity with a variety of databases
  
• 
  Familiarity with the Census databases and public records such as DoNotCall.
  
• 
  Knowledge of the Privacy Act guidelines for Australia/New Zealand
  
• 
  Familiarity with at least one of these languages: Python or Java/Scala
  
• 
  Experience performing root cause analysis on data issues and processes to answer specific business questions and identify opportunities for improvement
  
• 
  Build processes supporting data transformation, data structures, metadata, dependency and workload management
  
• 
  A successful history of manipulating, processing and extracting value from disconnected datasets
  
• 
  Experience supporting and working with cross-functional teams in a dynamic environment
  

What you'll be doing

• 
  Maintaining existing data pipelines in SSIS and T-SQL scripts
  
• 
  Assemble large, complex data sets and assisting us in our cloud migration and technology innovation projects
  
• 
  Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure and data models for greater scalability, etc.
  
• 
  Maintain the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL or Python
  
• 
  Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs
  
• 
  Keep our data separated and secure across national boundaries through multiple data centers and AWS regions
  
• 
  Create data structures for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader
  
• 
  Work with data and analytics experts to strive for greater functionality in our data systems
  

More about you

• 
  Ideally, you will have 5+ years of experience in a Data Engineer/Data Analyst role , who has a strong foundation in MS SQL, having obtained a Graduate degree in any technology area or another quantitative field
  
• 
  It will be a differentiator if you have experience with these software/tools: Big data tools - Spark, Kafka, etc. Data pipeline and workflow management tools - Glue, Azkaban, Luigi, Airflow, etc.
  
• 
  AWS knowledge and Amazon Redshift would be a 'nice to have'
  
• 
  Object-oriented/object function scripting language experience: Python, Julia, Java, C++, Scala, etc.
  

Why this role is critical to us

• 
  Experian maintains the most comprehensive demographical database about consumer behavior in Australia and New Zealand. We enable Australian businesses to be more efficient and make intelligent decisions critical to their success in a daily basis
  
• 
  You will operate the most important data and technology assets of our Marketing Services division, ensuring that data used by several clients and consultants remains up to date, accurate and in compliance with regulations