Senior Data Engineer

Job Title: Senior Data Engineer
Contract Type: Permanent
Location: City of London, London
Salary: £23 - £650000.00 per annum
Reference: BBBH20133_1566912289
Contact Name: Katherine Clarke
Contact Email:
Job Published: August 27, 2019 14:24

Job Description

Role: Senior Data Engineer

Salary: £45,000 - £65,000 per annum

Contract type: Permanent

Location: London

I am looking for a Senior Data Engineer to join our team in London, leading the modernisation of the organisations data management services, through the development of an Enterprise data platform.

You will be working in a multi-disciplinary Agile Team. By using Agile approaches to delivery, the team 'work out loud' encouraging open and inclusive discussions to drive development forward.

The Senior Data Engineer will responsible for the construction of data products for the business delivering new functionality to the Data Services platform. You will engage in discussion with technical design leads and other technical staff, recommending options for implementing technical infrastructure and future data products.

Key responsibilities

  • Follow Agile methodologies
  • Assure that your work and the work of colleagues meets governing standards
  • Ensure principles of privacy and security are assessed and designed into your data solutions
  • Facilitate the end-to-end data handling approach of Data Pipelines through the Data Services platform (defining and communicating technical needs for data collection, ingestion, transformation and calculation)
  • Encourage teamwork and communication, especially between Data Engineers, Quality Assurance and Subject Matter Experts
  • Be knowledgeable about the portfolio of data journeys
  • Provide succinct documentation to support knowledge retention and training
  • Be supportive and open to junior members of the team

Key outputs and deliverable

  • The data software engineering meets the requirements of each data product design
  • The data engineering solutions comply with privacy and security requirements
  • Work in an agile way, collaborating with colleagues
  • Provide advice and assurance about the work of junior colleagues
  • Deliver new functionality to the Data Service platform
  • Provide documentation to ensure knowledge retention
  • Ensure that governance rules, particularly security standards are always upheld

Key stakeholder relationships

  • The Data Services Analyst community, who will set the scope of your data engineering work through their research of user and business needs and by their designing data pipelines through writing user stories
  • The Data privacy and information security teams, who will need to work with you to assure that security and privacy requirements are identified and addressed
  • Fellow members of the Data Engineering community and the Lead Data Engineer, who you will mentor and collaborate with to set the direction of the data service technology and its functionality and promote Agile working
  • The Product Owner, who you will share your work with and discuss strategy for how the service is best able to support business needs

Specific skills and capabilities


  • Advanced knowledge of software design principles
  • Data development process
  • Data integration design
  • Programming and build
  • Technical understanding
  • Knowledge of data management, handling and processing
  • Data Quality
  • Data Preparation
  • Data Cleansing
  • Data Modelling
  • Experience working with and integrating into Business Intelligence tools

  • Hands on experience in platforms such as Azure and the Hadoop ecosystem (HIVE, HDFS and Apache spark)
  • Advanced competency with Python and SQL
  • Knowledge of Linux
  • Senior Team member
  • Ability to provide effective knowledge transfer / retention and support others accordingly
  • Proven experience of innovation and improving best practice
  • Experience of collaboration and partnering within a team, and implementing effective data engineering working practices in agile teams


  • Experience working with privacy and information accreditation or assurance teams
  • Experience in the delivery of data solutions capable of:
  • being certified to an information security standard such as ISO27001 or equivalent
  • Holding UK Government OFFICIAL-sensitive or equivalent information meeting legal and regulatory requirements such as DPA or GDPR
  • SQL Databases, such as MS SQL and/or Postgress
  • Cloud/Big Data: HDInsight, S3, EMR
  • NoSQL Databases, such as Neo4j
  • Data Integration and ETL approaches
  • Competency with Scala and/or Java
  • Experience of metadata management

Benefits include:

  • 25 days holiday, rising to 30 after 2 years of service
  • Competitive Pension
  • Childcare Vouchers
  • Subsidised gym membership
  • Season Ticket Loan Scheme
  • Cycle to Work Scheme
  • Flexible working hours and flexi-time

If this role sounds like you then please apply ASAP or give me Katherine Clarke a call on 01173320813 or email me on Katherine. Clarke @