Senior Data Engineer

Manchester Digital; Manchester; On-site; GBP 50,000 - 90,000

Job description

This year marks the go-live for our major tech transformation. This end-to-end system will give us some amazing benefits and opportunities, including access to more and ‘better’ data. With this context in mind, we are looking for two Senior Data Engineers to play a pivotal role in designing, developing, and maintaining a cutting-edge data platform on Azure using Databricks. Your primary responsibility will be to architect and implement scalable data pipelines, ensuring efficient data processing, storage, and retrieval. You will collaborate closely with analysts and other stakeholders to deliver high-quality data solutions that drive business insights and decision-making.

Duties include (but are not limited to):

1. Provide logical and physical data designs/models.
2. Design, develop, and maintain scalable data pipelines using Azure Databricks.
3. Ensure efficient data ingestion, transformation, and storage processes.
4. Work closely with business stakeholders, architects, analysts, and other stakeholders to understand data requirements.
5. Implement and enforce data security measures to protect sensitive information.
6. Ensure compliance with data governance policies and regulatory requirements.
7. Maintain comprehensive documentation of data pipelines, processes, and architectures.

You!

We are looking for a seasoned Senior Data Engineer.

If you have a combination of the following skills and experience, we are keen to chat!

1. Strong skills in designing and building ETL/ELT pipelines. ETL vs ELT and when to use them.
2. DataBricks and Azure.
3. Proficiency in programming languages such as Python, SQL, and PowerShell.
4. Familiarity with cloud architecture and best practices, specifically Azure.
5. Experience in data modeling, schema design, and data warehousing concepts.
6. OLTP and OLAP modeling.
7. Understanding of data quality frameworks and data governance principles.
8. DevOps experience and tooling, specifically Microsoft Azure DevOps.

Knowledge of Microsoft SQL Server architecture and features.