Member of Technical Staff, Head of Quality
Fractal Analytics
Posted 1 day agoDiscoveredMatch locked
full time1+ years exp
About the role
We're supporting a client with the recruitment requirement outlined below. If your recruiting expertise and networks align with the role, please 'apply' to express your interest.
Once approved to work on the opportunity, you'll receive the full brief, including the client name, role details, and commercial terms.
Role Details
Company: US-based applied AI research startup
Location: San Francisco, California (SoMa)
Employment Type: Full-time
Work Arrangement: On-site, five days a week
Salary: $180,000-$280,000
Experience: 1-5 years
Visa Sponsorship: Open to visa transfers, including OPT and H-1B transfers
About the Company
An early-stage AI research company developing infrastructure to train specialised AI agents using real-world data and high-fidelity simulated environments. Its technology helps generate training data and evaluation environments that improve the capabilities of advanced AI models.
The company works on complex AI training challenges alongside leading AI research organizations and technology companies.
About the Role
The company is hiring a Head of Quality to build automated quality assurance and evaluation systems for AI training data, agents, and simulated environments.
This is a hands-on technical role focused on ensuring data quality, evaluating model outputs, identifying failures, and building tools that make quality assessment more reliable and scalable.
The ideal candidate combines practical experience reviewing AI outputs with strong engineering skills and the ability to build automated evaluation systems in a fast-moving startup environment.
Responsibilities
- Build automated QA and evaluation systems for AI models, agents, and training data.
- Review model outputs and datasets to identify quality issues and failure patterns.
- Develop automated graders, evaluation tools, and quality monitoring systems.
- Detect reward hacking, verification failures, and data inconsistencies.
- Improve evaluation frameworks and data quality pipelines.
- Establish quality standards and help scale the quality function.
Requirements
- 1-5 years of experience in AI quality engineering, LLM evaluation, or technical QA.
- Hands-on experience building automated evaluation, QA, or data quality systems for ML training data, agents, or large-scale pipelines.
- Strong understanding of machine learning training and what makes data or simulated environments effective for model learning.
- Experience manually reviewing LLM outputs, agent traces, evaluation results, or annotation pipelines.
- Ability to build automated graders, judge models, QA agents, and quality monitoring tools.
- Strong attention to detail and willingness to perform hands-on data review.
- Ability to work independently in an ambiguous, early-stage environment.
- Technical background.
Benefits
- Competitive equity package.
- Opportunity to work on cutting-edge AI research.
- High ownership and the opportunity to build quality systems from the ground up.
Interview Process
- Candidate submission and hiring manager review.
- Introductory call.
- First-round interview covering AI quality experience.
- Technical interview assessing evaluation skills and problem-solving.
- Offer and hiring.
Skills
artificial-intelligenceqallmmachine-learningexpress
Who can apply
The employer didn't state any visa, work authorization, citizenship or clearance requirements in this posting. Confirm with the employer before applying.
Read automatically from the employer's posting text. Always confirm with the employer — requirements can change after a job is published.