Data Engineer

Different Technologies Pty Ltd.; London; Hybrid; GBP 40,000 - 80,000

Job description

Daintta are a rapidly growing, values-driven team of specialists who work with government clients across Cyber, Telecommunications and Data. We are seeking a talented and motivated Data Engineer to join our team and contribute to our mission of protecting the UK through data-driven insights and solutions. As a Data Engineer, you will work closely with our public sector clients and project teams to develop and implement data engineering pipelines to support data science and business/mission intelligence use cases.

Key Responsibilities

• Assessing your clients’ technical needs and understanding how their needs are different to wants and managing clients’ stakeholders relationship appropriately
• Identifying data sources, data extraction, transformation, and loading (ETL/ELT) concepts and methods
• Designing, evaluating, and implementing on-premise, cloud-based and hybrid data engineering solutions (including providing review and guidance on testing aspects related to data engineering, identification of risks and proposing and implementing their mitigations)
• Modelling, structuring and storing data along with their data flows for uses including — but not limited to — analytics, machine learning, data mining, compliance, business intelligence, sharing with applications and organisations
• Harvesting and ingesting structured and unstructured data in different formats and standards
• Integrating, consolidating and cleansing data
• Migrating and converting data
• Applying ethical principles in handling data
• Ensuring appropriate storage of data in line with relevant legislation
• Building in security, compliance, scalability, efficiency, reliability, fidelity, flexibility and portability
• Accurately delivering high quality work to agreed timelines and taking the initiative and knowing how to ‘jump straight in’
• Supporting client engagements, including pitches and presentations
• Helping to support & grow Daintta by actively inputting into the company strategy and helping to shape our future
• Representing us and our core values: Transparent, fair and daring

Skills/Knowledge

• You have 2+ years of degree level industry experience in data engineering, preferably in a consultancy or industry setting
• You have worked in client delivery across a range of projects, including data analysis, extraction, transformation, and loading, data intelligence, data security and proven experience in their technologies (e.g. Spark, cloud-based ETL services, Python, Kafka, SQL, Airflow)
• You have experience in assessing the relevant data quality issues based on data sources & uses cases, and can integrate the relevant data quality checks into data pipelines
• You have experience in designing schemas for data warehouses/data marts to support the relevant ingest and queries
• You have experience working on cloud-based infrastructure (e.g. AWS, Azure, GCP)
• You have demonstrable continuous personal development with relevant data certifications and accreditations
• You have strong interpersonal skills
• You are security cleared or capable and willing to undergo security clearance
• You have experience with using CI/CD tooling to analyse, build, test and deploy your pipelines and proven experience in their technologies
• You have experience in database technologies including writing complex queries against their (relational and non-relational) data stores (e.g. Postgres, Hadoop, Elasticsearch, Graph databases), and designing the database schemas to support those queries
• You have a good understanding of coding best practices and design patterns and experience with code and data versioning, dependency management, code quality and optimisation, error handling, logging, monitoring, validation and alerting

Location

Hybrid, with 2-3 days working from Daintta office (London or Cheltenham) or on client site as required.

What’s in it for you?

You will be joining the company at Daintta "Consultant" grade. In addition to being rewarded fairly for your contribution to the business, you get to work in a dynamic organisation that is agile and responsive. A business that is growing fast and where you get to drive and shape the future. A place where you are respected by everyone and your voice is important. Somewhere where you can be innovative and creative. A place where you have the opportunity to learn about all aspects of business from marketing to sales, to delivery and business operations.

Security Information

Due to the nature of this position, you must be willing and eligible to achieve a minimum of SC clearance. To qualify, you must be a British Citizen and have resided in the UK for the last 5 years. For more information about clearance eligibility, please see https://www.gov.uk/government/organisations/united-kingdom-security-vetting