Experimentation Principal Lead

You will need to login before you can apply for a job.

Job Band
Job Reference: 18222
Job Band: D
Contract type: Permanent
Location: Flexible (Glasgow, London or Newcastle)
Salary: Up to £79,000 depending on relevant skills, knowledge and experience. The expected salary range for this role reflects internal benchmarking and external market insights.

Introduction

We are looking for a Principal Experimentation Specialist to join the BBC Experimentation CoE team.
As a Principal Experimentation Specialist, you will drive a culture of data-informed decision-making within product teams by leading the design, execution, and analysis of high-impact experiments. This role will act as the key advisor on experimentation strategy, working closely with cross-functional teams to optimize product performance and customer experiences.
The Principal Experimentation Specialist will not only focus on running A/B tests and other experimental designs but also shape best practices and build scalable frameworks that enable product teams to test hypotheses and learn rapidly.

Key Responsibilities

This role is crucial in creating a culture of experimentation through consistent, evolving processes that inform decision making. You will work with product teams across the BBC to create elegant solutions and templates that streamline operations and improve the ease of applying experimentation methodologies.

1. Highly skilled in interpreting data and using insights to guide strategic decisions. Assess experiment results objectively and drive the team to focus on evidence-based improvements.
2. Develop an understanding of our product, operations, systems, and processes by interacting with product managers, engineering, and business teams.
3. Work with the Product, Design, Engineering, and experimentation teams to improve ways to identify, prioritize, design, implement, and own the backlog of experiments.
4. Establish experimentation standards that promote consistency and best practices across the business, and communicate these standards effectively to the broader data team.
5. Work with the experimentation team to develop methods and tools that increase the rigor and efficiency of our experimentation platform and analyses constructed using causal inference techniques.
6. Partner with Design and Product to identify key questions and opportunities, and construct pragmatic agendas to guide hypothesis-based learning strategies.
7. Work with the Data Platforms team to improve the processes, knowledge, and tools to support growing business demand for high-quality experimentation planning and execution.
8. Advocate for experimentation depth, rigor, and consistency across the data and business teams, and coach data team members on the principles of effective experimentation and inference.

Are you the right candidate?

Maintain a high-level view of the experimentation activities across all products: You will be responsible for identifying, reviewing, and communicating the key problems the business is trying to solve through experimentation.

1. Champion hypotheses definition and experimental design process across the product development lifecycle to capture all assumptions and decisions as hypotheses to test.
2. Collaborate with cross-functional teams: Work closely with designers, product managers, data analysts, and engineers to define experiment goals, set up tracking and measurement, and interpret results.
3. Ensure high-quality experiments: Responsible for ensuring that experiments are properly designed, executed, and analyzed, with appropriate statistical rigor and validity.
4. Analyze and interpret results: Analyze experimental results, interpret their implications, and present them to stakeholders.
5. Communicate insights: Responsible for communicating insights and recommendations to stakeholders, including senior leadership, in a clear and actionable manner.
6. Stay up-to-date with experimentation best practices: Responsible for staying updated with the latest experimentation best practices and tools, and sharing knowledge with the broader team.
7. Drive a culture of experimentation: Encourage testing assumptions and learning from experiments.

Overall, your role as an experimentation specialist will be critical in helping the product teams drive growth, optimize user experiences, and achieve business objectives through data-driven experimentation.

Experience and Skills

1. 5+ years experience with statistical analysis and tools like Excel, SQL, R, or Python.
2. 3+ years experience in designing experiments, including A/B testing and randomized controlled trials, and an understanding of experimental design principles.
3. Ability to formulate hypotheses and test them using statistical methods and testing tools.
4. Excellent communication skills, both written and verbal.
5. Strong problem-solving skills and ability to develop innovative solutions to complex problems.
6. Experience in project management and ability to manage multiple experiments simultaneously.
7. Understanding of business objectives and ability to align experimentation goals with these objectives.
8. Detail-oriented with a strong focus on accuracy when analyzing data and designing experiments.

Company

Learn more about this company

Visit this company’s hub to learn about their values, culture, and latest jobs.