Python (PySpark) Developer

Job Title: Python (PySpark) Developer
Contract Type: Contract
Location: England
Salary: £550 - £600 per day + Inside IR35
Start Date: ASAP
Reference: BBBH30779_1657723302
Contact Name: James Norval
Contact Email:
Job Published: July 13, 2022 15:41

Job Description

Python (PySpark) Developer

  • Security Clearance required SC clearance - ideally held currently
  • Remote working anywhere in the UK
  • 3 months
  • £550/£600 per day Inside IR35
  • Deadline 18/07/22 12 Noon
  • Central Governement

Description of Requirement:

The coronavirus (COVID-19) Infection Survey (CIS) provides statistical analysis on the COVID-19 pandemic for Government and academic research purposes. The CIS Data Processing Pipeline (DPP) receives data from multiple sources and engineer these data to provide cleaned, linked datasets for analysis.

This role is responsible for the development and maintenance of data processing pipelines. You will work with a small development team and wider analysis team to meet requirements for analysis. You will need experience of data-driven projects.

We are looking for a senior developer to join the team for a hands-on coding role with elements of team leadership. We are looking for strong development skills coupled with wider awareness and the ability to contribute fully to the role of the development team.

Please highlight specific experience that applies the skills described below.


You will be responsible for the development of sophisticated data processing pipelines, whilst supporting team members and others to apply agile principles to deliverables. In doing so, you will:
* Contribute to the solution design and development approach for the pipeline
* Work hands-on to produce code solutions in Python / PySpark / SQL Server / Hive
* Work with other developers, through pair programming, to implement data processing pipelines.
* Liaise with analysis teams to ensure that requirements for outputs are understood and incorporated into the pipeline.
* Address technical blockers, actively seeking solutions to remove them and proposing alternative routes to delivery.
* Routinely test the developed pipelines to ensure that they are fit for purpose.
* Peer review work of others in the team, to ensure quality and consistency.

Relevant Skills and Experience: maximum 6
Desirable or Essential

* Python - proficiency in Python development, including PySpark. E
* Testing - writing unit tests for PySpark routines using pytest. E
* Agile - applying Agile software development practices and tools. E
* DevOps - applying DevOps practices, including continuous integration and continuous deployment. D
* Analysis - identifying the root cause of data quality issues and communicating these with data providers. D

Role requires SC security Clearance for access to secure data and environments