Content Safety Testing: Design, develop, and execute comprehensive test plans to ensure the security and reliability of content generated by large language models.
LLM Integration: Work with LLM APIs (e.g., OpenAI's API) to integrate AI models into automated testing workflows, ensuring consistent and thorough content evaluation.
Prompt Engineering: Leverage prompt engineering skills to enhance testing strategies and optimize LLM behaviour for safe and reliable output.
Defect Identification & Resolution: Identify and document content safety issues or model behaviour concerns, and collaborate with AI developers to resolve them.
Automation & Scripting: Develop automated testing scripts using Python to streamline LLM testing processes and ensure scalable and efficient content safety evaluations.
The Successful Applicant
Master's or Bachelor's degree in Computer Science, Engineering, or a related field.
Proficient in Python programming and comfortable working with LLM APIs (e.g., OpenAI).
Experienced in testing the safety, security, and reliability of AI-generated content.
Knowledgeable in prompt engineering techniques and how they can enhance content safety.
Passionate about AI technologies and committed to ensuring the ethical and secure use of AI-generated content.
At least 1 year of hands-on experience with LLM APIs.
What's on Offer
Opportunity to work with cutting edge technologies and support the development of ethical and secure AI use cases.
Excellent overall compensation package and benefits.
Desired Candidate Profile
Attention to Detail: Precision in reviewing processes, products, and documentation to identify potential safety and quality issues.
Regulatory Knowledge: Understanding of industry-specific regulations and standards (e.g., OSHA, ISO, FDA) to ensure compliance and safety.
Analytical Skills: Ability to analyze data and metrics to assess quality performance and identify areas for improvement.
Risk Management: Skills in identifying, assessing, and mitigating risks related to safety and quality in products and processes.
Communication Skills: Strong verbal and written communication abilities for effectively reporting findings, conducting training, and collaborating with teams.
Problem-Solving Skills: Capability to develop and implement solutions to address quality issues and enhance safety protocols.
Process Improvement: Familiarity with quality management methodologies (e.g., Six Sigma, Lean) to drive continuous improvement initiatives.
Training and Development: Experience in designing and delivering training programs to educate employees on safety and quality practices.
Project Management: Skills to manage multiple initiatives, timelines, and teams while ensuring adherence to safety and quality standards.
Interpersonal Skills: Strong ability to build relationships and work collaboratively with various stakeholders, including management, staff, and external partners.
* The salary benchmark is based on the target salaries of market leaders in their relevant sectors. It is intended to serve as a guide to help Premium Members assess open positions and to help in salary negotiations. The salary benchmark is not provided directly by the company, which could be significantly higher or lower.