Principal Cloud Data Engineer

ZipRecruiter; London; On-site; GBP 50,000 - 90,000

Job description

Job Description

AWS Data DevOps Engineer

Location: London

Employment Type: Permanent

Salary: Competitive based on experience

Security Clearance: Eligible or holds SC Clearance

About Valcon

At Valcon, we specialise in delivering cutting-edge data and cloud solutions to help businesses maximise their potential. We are seeking an AWS Data DevOps Engineer to join our team and play a key role in building, maintaining, and optimising our cloud-based data infrastructure. If you thrive in a fast-paced, tech-driven environment and have a passion for AWS, data pipelines, and automation, we'd love to hear from you!

Key Responsibilities

• Design, implement, and maintain AWS-based data infrastructure to support scalable and secure data operations.
• Develop and optimise data pipelines for processing, storing, and retrieving structured and unstructured data.
• Work with Amazon OpenSearch to enable real-time search and analytics on large-scale datasets.
• Implement and manage Lake Formation and AWS Security Lake, ensuring data governance, access control, and security compliance.
• Optimise file formats (e.g., Parquet, ORC, Avro) for S3 storage, ensuring efficient querying and cost-effectiveness.
• Automate infrastructure deployment using Infrastructure as Code (IaC) tools such as Terraform or AWS CloudFormation.
• Monitor and troubleshoot data workflows, ensuring high availability and performance.
• Collaborate with data engineers, security teams, and DevOps professionals to build a resilient data ecosystem.

Key Skills & Experience

• AWS Expertise: Strong experience with AWS services, including S3, Lambda, IAM, OpenSearch, Security Lake, and Lake Formation.
• Data Engineering: Proficiency in building and optimising data pipelines and working with large-scale datasets.
• File Formats & Storage: Hands-on experience with Parquet, ORC, Avro, and efficient S3 storage solutions.
• DevOps & Automation: Experience with Terraform, CloudFormation, or CDK to automate infrastructure deployment.
• Security & Compliance: Familiarity with AWS Security Lake, IAM policies, and access control best practices.
• Programming & Scripting: Proficiency in Python, Bash, or similar scripting languages for automation.
• SC Clearance: Having or being eligible for Security Clearance (SC) is needed.

Desirable Skills

• Team Leadership: Experience leading teams or mentoring engineers in data/DevOps projects.
• Solution Leadership: Strong ability to architect and drive technical solutions, influencing key design decisions.