linkedin

Connecting...

SC Cleared Data Engineer x2 (Hadoop/Spark/Python)

Job Title: SC Cleared Data Engineer x2 (Hadoop/Spark/Python)
Contract Type: Contract
Location: England
Industry:
Salary: £650 - £750 per day + inside IR35
Start Date: ASAP
Reference: BBBH29007_1639207063
Contact Name: James Norval
Contact Email: james.norval@talentinternational.com
Job Published: December 11, 2021 07:17

Job Description

*** You must have been SC Cleared within the last 12 months to be eligible for these contracts ***

Data Engineer x2 - Central Government

  • Active SC Clearance required (Must have been in a position that required SC Clearance within the last 12 months)
  • Remote working anywhere in the UK
  • Day rates, Inside IR35 - negotiable upon discussion
  • Until 31/05/22 initially
  • Deadline: 12 noon 15th December 2021

SERVICE REQUIREMENT


Software developers required to lead on design and implementation of data interface to move census data products through the processing tech stack as part of the team. Outputs to be delivered to various QA teams to validate return quality, enabling outputs in a year publication data to be met. Candidate will work as one of a team of engineers using agile methodologies to implement these interfaces and provide first line support through critical operational periods.
Software developers required to work on building data pipelines on Cloudera using Hadoop, Spark, Scala, Hive with an understanding and focus on sourcing data, transforming, establishing governance and building flexible frameworks to transform, aggregate data for aggregation.


Responsibilities:

Hands on Data engineer to work on building data interfaces and data pipelines on Cloudera using Hadoop, Spark, Scala, Hive with an understanding and focus on sourcing data, transforming, establishing governance and building flexible frameworks to transform, aggregate and facilitate data for aggregation.

Relevant Skills and Experience:

  • Familiarity with Cloudera toolset desirable (HUE, Hive, Impala, Data Science Workbench, HDFS, Avro, Parquet). Essential
  • Python, Spark, Gitlab development Essential
  • Agile - to align to team current way of working and promote iterative development Essential
  • Confluence, JIRA, Sharepoint - agile project tooling Desirable