DevOps Engineer (AWS, Data, Python)
Must be eligible for SC
Deadline: Tuesday 2nd November, 12:00
Seeking an AWS accredited DevOps engineer with hands on experience of designing and building data centric solutions with AWS services and components; services such as: Lambda, Glue, S3, Athena, Step Functions, and API Gateway.
Capable of working across multiple software tools and systems, using a range of programming languages and software libraries, including: Python, SQL, Pandas and Spark. Some knowledge of R and Java would also be desirable.
Our solution utilises data virtualisation, with a Denodo virtual data platform at its core, so experience or knowledge of data virtualisation concepts would also be advantageous.
The solution is an analytical platform, so any experience in working with Notebook technologies such as Jupyter and Zeppelin, or R Studio would also be beneficial.
Contribute towards a data centric solution for a cloud-based Integrated Data Platform; strong experience of working with cloud-based data storage and data query technologies.
The emphasis of the role is building data services for the Data Access and Management Product.
Relevant Skills and Experience:
- Strong knowledge and experience of using serverless AWS services and components: Lambda, Glue, S3, Athena, Step Functions, API Gateway and EventBridge. Apache Airflow experience would be desirable.
- Experience of working on data centric solutions with an emphasis on data processing; designing, developing and testing data services written in Python, SQL, Pandas and Spark.
- Experience of AWS database technologies and services: Athena, Redshift, DynamoDB and RDS; including designing cloud storage solutions, query performance optimisation and troubleshooting issues.
- Any experience of data virtualisation technologies would also be beneficial. Essential
- Infrastructure as Code (IAC) experience using Terraform and CloudFormation technologies to provision and manage AWS components.
- Continuous Integration and Delivery (CI/CD) experience using AWS toolset: CodePipeline, CodeBuild and CodeDeploy.
- Also experience of Concourse would be beneficial.
- Experience of working with Notebook technologies such as Jupyter and Zeppelin, or R Studio would also be beneficial.