Senior Data Engineer

Job Title: Senior Data Engineer
Contract Type: Permanent
Location: City of London, London
Salary: Up to £80000 per annum + Public Sector Benefits
Start Date: ASAP
Reference: DMS291019002_1572366152
Contact Name: Damian Speirs
Contact Email:
Job Published: October 29, 2019 16:22

Job Description

Senior Data Engineer / Data Engineer / Data Quality / Data Modelling / Data Cleansing / Python / SQL / Linux / Permanent / Public Sector Benefits

£80,000 & Flexible Working Conditions

Canary Wharf

We are looking for a Senior Data Engineer to join the Data Team for this public sector organisation. This is an innovative and forward-thinking team, leading the modernisation of data management for this organisation through the development of an Enterprise data platform. The data provided will be used to make a difference for all citizens across the UK.

The team is comprised of product leads, data scientists, software engineers, user researchers, QA testers, business and data analysts. By using Agile approaches to delivery, the team 'work out loud' encouraging open and inclusive discussions to drive development forward.

The Senior Data Engineer will be responsible for the construction of data products for the business and delivering new functionality to the Data Services platform. You will engage in discussion with technical design leads and other technical staff, recommending options for implementing technical infrastructure and future data products. You will facilitate the end-to-end data handling approach of data pipelines using the Data Services platform. Defining and communicating the technical needs for data collection, ingestion, transformation and calculation.

Senior Data Engineer Specification:

  • Advanced knowledge of software development:
  • The software development lifecycle and specific applications to data engineering, including Test Driven Development and quality assurance methods
  • An understand of software design and architecture best practice
  • Ability to create a code base with a high-test coverage and ensure test coverage remains high in development through automated test processes (for example continuous deployment / continuous integration best practice) Knowledge of data management, handling and processing
  • Data Quality
  • Data Preparation
  • Data Cleansing
  • Data Modelling
  • Experience working with and integrating into Business Intelligence tool
  • Hands on experience in platforms such as Azure and the Hadoop ecosystem (HIVE, HDFS, Apache Spark, Airflow)
  • Advanced competency with Python and SQL. Knowledge of Linux
  • Senior Team member
  • Ability to provide effective knowledge transfer / retention and support others accordingly
  • Proven experience of innovation and improving best practice
  • Experience of collaboration and partnering within a team, and implementing effective data engineering working practices in agile teams
  • Experience working with privacy and information accreditation or assurance teams
  • Experience in the delivery of data solutions capable of:
  • being certified to an information security standard such as ISO27001 or equivalent
  • holding UK Government OFFICIAL-sensitive or equivalent information
  • meeting legal and regulatory requirements such as DPA or GDPR
  • SQL Databases, such as MS SQL and/or Postgress
  • Cloud/Big Data: HDInsight, S3, EMR
  • NoSQL Databases, such as Neo4j

If you have the relevant experience and are up for a new challenge - apply today. I look forward to receiving your application and talking you through this role in more detail. All applications will be dealt with in the strictest confidence. My client is open to Tier 2 applications but they are not able to sponsor at this time.

Senior Data Engineer / Data Engineer / Data Quality / Data Modelling / Data Cleansing / Python / SQL / Linux / Permanent / Public Sector Benefits