Job
Full source details
✓ 100% verification score · Source: Clera (ashby) · Always confirm final requirements on the original source.
Complete source information imported
The available role or programme description, requirements, benefits and source facts were imported from the public official endpoint and formatted for reading.
EmploymentFull-time
Overview
ABOUT THE ROLE
Full job description
ABOUT THE ROLE Own complex data and evaluation programs for AI research teams and internal partners at an early-stage AI data infrastructure startup. You will bring structure to ambiguous technical requests and build reliable processes that help improve LLM and agent evaluations. WHAT YOU'LL DO
- Lead data and evaluation programs from scoping and requirements gathering through production, quality assurance, delivery, and retrospectives.
- Turn evolving technical requests into clear specifications with milestones, owners, dependencies, and acceptance criteria.
- Assess and improve data quality using quantitative metrics and qualitative review, including task coverage, difficulty, and reward distributions.
- Manage external vendors across the delivery lifecycle and scale delivery pipelines.
- Improve workflows, documentation, automation, and cross-functional handoffs for data production.
- Work with research, platform, and marketplace teams to turn operational learnings into product improvements. WHAT WE'RE LOOKING FOR
- At least 3 years of experience owning complex technical programs in AI/ML, data operations, support engineering, applied research, forward-deployed engineering, or a similar cross-functional area.
- Experience as a Technical Program Manager or Strategic Program Lead at a startup, including supporting AI research teams, technical enterprise customers, or other partners with urgent, changing needs.
- Strong quantitative and qualitative judgment, with the ability to assess both aggregate dataset metrics and individual examples.
- Experience managing vendors and improving data quality processes, workflows, and delivery pipelines.
- Familiarity with RLHF, reinforcement-learning environments, agent evaluations, human-in-the-loop data pipelines, annotation, or expert data collection is valuable. COMPENSATION & BENEFITS Visa sponsorship and relocation support are available for strong full-time candidates. LOCATION On-site in San Francisco, California. Remote candidates may be considered if they can maintain substantial time zone overlap with San Francisco or Singapore.
Tips for this job
Practical JobOpportunity guidance. These tips do not replace official rules or create new eligibility requirements.
- Tailor the CV and application to the responsibilities and required skills stated on the official employer page.
- Use concrete evidence of relevant work, projects and measurable results rather than generic claims.
- Confirm location, work authorization, remote restrictions and sponsorship terms before applying.
- Apply through the original employer or official recruitment destination shown on this page.
Verification notes
laptop-ats-crawler v3
Original authoritative source
JobOpportunity is the discovery and verification layer. Confirm eligibility, dates, salary/funding and application instructions on the original source before submitting anything.
Apply through JobOpportunity →