Senior Python Software Engineer, Data Platform

YouGov is a global online research company, offering insight into what the world thinks.

We speak daily to our panel of over 24 million registered members to understand opinion and behaviors around the world. We have a strong reputation as a source of accurate data and we’re trusted by the world’s biggest brands to get it right, making us the most quoted market research source in the world.

With operations in the UK, North America, Mainland Europe, the Nordics, the Middle East, India and Asia Pacific, YouGov has one of the world’s largest research networks.

About the Crunch Team

The Senior Software Engineer (Data Platform) role will work within the Crunch team. Crunch is the core web application for our syndicated data products, custom research, and first-party research hub. The Crunch team is responsible for creating innovative, intuitive, accessible, and engaging experiences for the world’s leading brands. This role will be a vital part of the evolution of one of our premier data products, such as YouGov Brand Index and Profiles.

We are inviting you to join our fully remote team of developers and operators helping make our platform faster, more secure, and more reliable. You would help us deliver new features and build the future for how we deploy our products faster and safer, while increasing the availability and reliability of our systems. You will explore different ways of problem-solving and discover, explain, and defend your choices in implementation. You will be self-motivated and disciplined in order to work with our fully distributed team. As Crunch’s employees are completely distributed, it’s crucial that you can work well independently, and keep yourself motivated and focused. We move fast but carefully, with an earnest commitment to not breaking things… well, not too much.

About the role

The Data Platform Team is responsible for ensuring that Crunch can reliably import customer survey data via file uploads, streaming, 3rd-party API data providers and custom build data pipelines. You’ll work on the design, development, and maintenance of reliable, efficient and scalable data systems, APIs and pipelines to support Crunch’s and YouGov’s growing business.

WHAT YOU WILL DO

1. Develop effective ways to store, query, and interactively analyze large datasets that contain millions of rows and hundreds of thousands of columns.
2. Work closely with product managers, sales, and customer success teams to understand the system’s functional and non-functional requirements.
3. Contribute to code quality through unit testing, integration testing, code review, and system design using Python.
4. Establish realistic estimates for timelines and ensure that projects remain on target to meet deadlines.
5. Assist in diagnosing and fixing system failures quickly when they occur in your area of expertise. This is limited to when the on-call rotation needs a subject-matter expert to help troubleshoot an issue.
6. Design and implement RESTful API endpoints using the Python programming language.
7. Break down complex problems to identify key variables and make informed decisions based on thorough analysis.

WHAT YOU NEED TO KNOW (TECHNICAL & BEHAVIOURAL SKILLS & QUALIFICATIONS)

1. Extensive enterprise experience solving complex problems using multiple database systems, data lake architectures, and query engines.
2. Proven previous experience with open data tools such as Apache Arrow, Pandas, Polars.
3. A record of successful delivery of SaaS and cloud-based applications.
4. Strong understanding of the software development lifecycle.
5. Extensive programming experience using Python as a programming language.
6. A commitment to producing robust, testable code.
7. Results-driven, self-motivated and enthusiastic.
8. Excellent communication skills – verbal, written and presentation.

PREFERRED QUALIFICATIONS

1. Experience with data locality problems and caching issues.
2. Proven expertise and experience with database technologies including NoSQL databases like MongoDB, RDBMS such as Postgres and MySQL, Apache Iceberg, Trino / Presto, Clickhouse, Snowflake, BigQuery.
3. Pytest testing experience.
4. Relevant experience in the maintenance of data APIs and data lake architectures.
5. Bachelor’s Degree in Programming, Computer Science, or Engineering-related field.