Job
Full source details
✓ 100% verification score · Source: Clera (ashby) · Always confirm final requirements on the original source.
Complete source information imported
The available role or programme description, requirements, benefits and source facts were imported from the public official endpoint and formatted for reading.
EmploymentFull-time
Overview
ABOUT THE ROLE
Full job description
ABOUT THE ROLE Join a small technical research and engineering team building benchmarks for evaluating AI agents on realistic, domain-specific workflows. You will own the design and implementation of rigorous evaluations that help technical teams understand how agents perform in real-world tasks. WHAT YOU'LL DO
- Design, implement, and maintain benchmarks for evaluating AI agents on domain-specific tasks.
- Work with subject-matter experts to translate real workflows into benchmark tasks and evaluation criteria.
- Build and operate reliable infrastructure for running models and agents against tasks at scale.
- Develop metrics and analyses to assess benchmark difficulty, reliability, and failure modes.
- Validate whether benchmark results align with real-world performance and technical user needs.
- Write clear documentation and reports for research and engineering audiences. WHAT WE'RE LOOKING FOR
- Two to four years of experience in research engineering, machine learning engineering, or a related technical role, including at least two years building AI benchmarks, evaluation infrastructure, or agent environments.
- Strong Python skills and practical experience with Docker and Linux.
- Experience designing, implementing, and running benchmarks or evaluation environments for AI agents or large language models.
- Experience collaborating with subject-matter experts and analyzing workflows across technical or business domains.
- Experience developing metrics, statistical analyses, or validation studies for evaluations.
- Strong technical writing, attention to detail, and ability to work independently in an early-stage environment.
- A technical educational background; experience with reinforcement learning pipelines, published evaluation work, or widely used benchmarks is a plus. COMPENSATION & BENEFITS Salary range: USD 150,000 to 250,000 annually. Visa sponsorship is available. LOCATION On-site in Singapore, Singapore.
Tips for this job
Practical JobOpportunity guidance. These tips do not replace official rules or create new eligibility requirements.
- Tailor the CV and application to the responsibilities and required skills stated on the official employer page.
- Use concrete evidence of relevant work, projects and measurable results rather than generic claims.
- Confirm location, work authorization, remote restrictions and sponsorship terms before applying.
- Apply through the original employer or official recruitment destination shown on this page.
Verification notes
laptop-ats-crawler v3
Original authoritative source
JobOpportunity is the discovery and verification layer. Confirm eligibility, dates, salary/funding and application instructions on the original source before submitting anything.
Apply through JobOpportunity →