Fareham or Newport
Active SC Clearance is Required
This will involve developing operational and organisational data to form the basis of a series of products as well as applying established data engineering and data modelling methods to the data for the main stage of the product build.
Skills and Experience
- Extensive proven experience of data engineering and architectural techniques, including data wrangling, data profiling, data preparation, metadata development, and data upload/download.
- Proven experience of 'big data' environments, including the Hadoop Stack (Cloudera), including data ingestion, processing and storage using HDFS, Spark, Hive and Impala.
- Extensive hands-on experience of developing ETL functionality in a cloud or on-premise environment.
- Experience of using tools such as python and SQL (Spark) to profile, query and structure large-volume data.
- Proven experience of using Cloud Services particularly in the context of Hadoop.
- Experience of developing/utilising programming and query languages e.g. SQL (Hive Impala specifically), Python (through Spark), Scala.
- Understanding of data bases and applying data models in relational database formats.
If you would like to find out abit more please get in touch on 0117 332 0834