Overview
ABOUT THE ROLE
Full job description
ABOUT THE ROLE This is a senior technical leadership role on the data quality team at an early-stage AI infrastructure company focused on building and scaling reinforcement learning environments for frontier model training. You will own the strategy and systems that measure and improve training data quality, shaping research culture around what makes agent data genuinely useful rather than just superficially correct. WHAT YOU'LL DO
- Lead the data quality team in building systems that evaluate thousands of tasks across RL environments, synthetic data, benchmarks, and domain-specific workflows.
- Define the data quality strategy by building QC systems, enforcing standards, and designing experiments to grade agent outputs.
- Develop new methods for validating synthetic data at scale, including failure-mode analysis, task mutation checks, and trajectory auditing.
- Partner with research engineers, domain experts, and data vendors to diagnose quality issues and improve data generation workflows.
- Turn qualitative research insights into production systems: internal tools, dashboards, validation pipelines, and feedback loops.
- Mentor research engineers to maintain a high bar for technical rigor, clarity, and execution speed. WHAT WE'RE LOOKING FOR
- 5+ years of relevant engineering or research experience, specifically building systems for AI/ML data evaluation or data quality.
- Proven track record leading technical teams on ambiguous projects from problem definition through implementation and iteration.
- Advanced proficiency in Python, Docker, and Linux environments.
- Experience building QC systems, evals, benchmarks, synthetic data pipelines, validation workflows, or model evaluation infrastructure.
- Deep intuition for what makes training tasks realistic, learnable, diverse, reliable, and useful for AI agents.
- Research-oriented understanding of AI evals and post-training, beyond surface-level agent tooling projects.
- Comfort designing metrics, experiments, and QA/QC processes, not just executing them.
- Strong written communication skills, with the ability to explain methodology clearly to diverse audiences.
- Experience working with subject-matter experts to capture domain judgment and convert it into scalable review or generation systems.
- Early-stage startup experience and the ability to work independently in fast-paced environments. COMPENSATION & BENEFITS Salary range: $150,000 to $180,000 USD annually. Visa sponsorship is available. LOCATION On-site in San Francisco, CA, USA. The team also has a presence in Singapore.
Tips for this job
Practical JobOpportunity guidance. These tips do not replace official rules or create new eligibility requirements.
- Tailor the CV and application to the responsibilities and required skills stated on the official employer page.
- Use concrete evidence of relevant work, projects and measurable results rather than generic claims.
- Confirm location, work authorization, remote restrictions and sponsorship terms before applying.
- Apply through the original employer or official recruitment destination shown on this page.
Verification notes
laptop-ats-crawler v3
JobOpportunity is the discovery and verification layer. Confirm eligibility, dates, salary/funding and application instructions on the original source before submitting anything.
Apply through JobOpportunity →