Direct experience of building a data pipeline using Azure Data Factory and (preferably Databricks).
Ideally a Microsoft Azure Big Data Architecture certification.
Hands-on experience designing and delivering solutions using the Azure Data Analytics platform (Cortana Intelligence Platform) including Azure Storage, Azure SQL Data Warehouse, Azure Data Lake, Azure Cosmos DB, Azure Stream Analytics
- Proven track record of using Azure data solution tools and implementation principles.
- Experienced engineer with Data Factory/DataBricks/PySpark skillsets
- An understanding of the development lifecycle (e.g. dev, unit test, system test, UAT, etc.)
- Experience of using code repositories and version control (ideally Git)
- Knowledge of ETL development suites e.g. Microsoft SSIS or Informatica
- Strong analytical and problem-solving abilities
- Strong attention to detail and accuracy
Understanding of the DevOps philosophy and knowledge of database schemas and 1 or more flavours of SQL - scripts, stored procedures, functions, indexes, constraints etc. would be a bonus.