Lead Data Engineer

Mustard Systems; London; On-site; GBP 60,000 - 100,000; Full time

Job description

Mustard Systems is a team of roughly 65 people dedicated to developing market leading sports models and trading strategies. Applications of our research are widespread across multiple sectors including betting, insurance, finance and the media, as well as giving advice to coaches and owners of sporting teams. Our aim is to predict sport as accurately as possible, and we achieve this by combining large, rapidly changing datasets with home grown software and statistical models.

We enjoy a fast-paced, ever-changing environment, offering fresh challenges which are both rewarding and enjoyable. Our ambitions are high, aiming to create best-in-class systems in a highly competitive field. In the coming year, we will be working on expanding our trading strategies and data feeds.

We are looking for a Lead Data Engineer to start our Data Engineering team. This is an exciting opportunity, as this team will transform how our quants interact with the data we have. This role in particular will be at the forefront of setting up our Data Team and building out our capabilities in this new chapter for Mustard.

Requirements
About the Team

As the first hire you'll have a unique opportunity to build processes from the ground up with a view to future scalability. Whether it's video footage of cricket matches, or terabyte scale relational databases, you'll work with the quants to ensure they can seamlessly access the combination of sporting event data and betting data they need to build more powerful models that directly drive our profit via trading.

Our current pipelines included both scheduled batch jobs and real-time consumption from our applications, writing to both files and SQL databases.

The new Data Engineering team will own processes to ensure our analysts can reliably and performantly get the data they need, in a way that scales to our future ambitions. They'll work directly with consumers of the data, from quants and sports traders to the company board, in order to understand their needs, and will support other development teams to integrate producers.

As Data Engineering Team Lead, you will:

• Recruit to help build the team
• Manage the team
• Lead the design and implementation of our long-term scalable data architectures ensuring both efficient storage and access
• Collaborate closely with the quants and other consumers to understand and meet their needs
• Build and maintain our data pipelines, to integrate, process and store large (terabyte scale) data sets across various sources (including our own internal databases)
• Actively monitor our data pipelines for errors, and resolve them to ensure the needed data reaches our end users

You will have the freedom to suggest and bring your own tools and ideas to solve our data problems.

Our Core Tech Stack

• Languages: Python (Python 3.10+)
• Tools: PostgreSQL (with TimescaleDB) for data storage; RabbitMQ and ZeroMQ for messaging
• Environment: Linux servers
• Observability: Prometheus, Grafana, Zabbix

What We're Looking For:

• At least 5 years' experience in a Data Engineering role, with:
• Solid Python expertise and a track record of building robust, high-quality solutions - including experience writing scrapers and doing data manipulation (e.g. Pandas / Numpy)
• Advanced SQL skills, with a deep understanding of relational databases including performance optimization
• At least 1 year in a leadership position, with a history of successfully guiding teams and projects to impactful outcomes.
• Experience in designing scalable and extensible data solutions that pragmatically address end user needs
• Clear and impactful communication skills, capable of translating intricate technical topics and business challenges into actionable insights.
• Ideally you will have a breadth of knowledge about existing tools, allowing you to bring ideas about how to solve Mustard's problems.

Benefits

Why join Mustard Systems?

• Work on cutting-edge systems in a competitive and innovative field.
• Collaborate with a smart, driven team, where your contributions directly impact business performance.
• Opportunity to drive the company's technical direction and double its revenue in the next three years.

Comprehensive benefits, including:

• Competitive salary and significant bonus potential (up to 30%).
• Enhanced pension match with salary sacrifice option.
• Health insurance and life assurance.
• Sabbatical leave after five years.
• 33 days of annual leave (including bank holidays).