Job
Full source details
✓ 100% verification score · Source: Clera (ashby) · Always confirm final requirements on the original source.
Complete source information imported
The available role or programme description, requirements, benefits and source facts were imported from the public official endpoint and formatted for reading.
EmploymentFull-time
Overview
ABOUT THE ROLE
Full job description
ABOUT THE ROLE Lead the data quality team at an early-stage AI evaluation company, building the strategy and systems used to assess and improve training data for AI agents. Your work will help ensure data is realistic, reliable, and useful for training. WHAT YOU'LL DO
- Lead development of systems to evaluate tasks across reinforcement learning environments, synthetic data, benchmarks, and domain-specific workflows.
- Define quality standards, metrics, experiments, and processes for assessing agent outputs.
- Develop scalable synthetic data validation methods, including failure analysis, task mutation checks, and trajectory audits.
- Partner with research engineers, domain experts, and data vendors to diagnose issues and improve data generation workflows.
- Turn research insights into production tools, dashboards, validation pipelines, and feedback loops.
- Mentor research engineers and promote technical rigor and clear communication. WHAT WE'RE LOOKING FOR
- At least 5 years of research or engineering experience building AI or machine learning data evaluation and quality systems.
- Experience leading technical teams or projects from problem definition through implementation and iteration.
- Advanced proficiency in Python, Docker, and Linux, with a technical education or background.
- Strong understanding of AI evaluations, post-training, and the qualities that make agent training data realistic, learnable, diverse, and reliable.
- Experience building evaluation infrastructure, benchmarks, synthetic data pipelines, or validation workflows.
- Ability to translate domain expertise and research insights into scalable review systems and production data pipelines.
- Strong written communication and comfort working independently in an early-stage startup environment. COMPENSATION & BENEFITS Salary range: $150,000 to $180,000 annually. Visa sponsorship is available. LOCATION On-site in San Francisco, California, United States.
Tips for this job
Practical JobOpportunity guidance. These tips do not replace official rules or create new eligibility requirements.
- Tailor the CV and application to the responsibilities and required skills stated on the official employer page.
- Use concrete evidence of relevant work, projects and measurable results rather than generic claims.
- Confirm location, work authorization, remote restrictions and sponsorship terms before applying.
- Apply through the original employer or official recruitment destination shown on this page.
Verification notes
laptop-ats-crawler v3
Original authoritative source
JobOpportunity is the discovery and verification layer. Confirm eligibility, dates, salary/funding and application instructions on the original source before submitting anything.
Apply through JobOpportunity →