Job
Full source details
✓ 100% verification score · Source: Clera (ashby) · Always confirm final requirements on the original source.
Complete source information imported
The available role or programme description, requirements, benefits and source facts were imported from the public official endpoint and formatted for reading.
EmploymentFull-time
Overview
ABOUT THE ROLE
Full job description
ABOUT THE ROLE Join a small, technical research and engineering team building rigorous benchmarks for evaluating AI agents on realistic, domain-specific workflows. You will own benchmark design and implementation, helping ensure evaluation results are reliable and useful to research and industry teams. WHAT YOU'LL DO
- Design, implement, and maintain benchmarks for evaluating AI agents on domain-specific tasks.
- Collaborate with subject-matter experts to turn real workflows into realistic tasks and evaluation criteria.
- Build reliable infrastructure to run models and agents against evaluation tasks at scale.
- Develop metrics and analyses to assess benchmark difficulty, reliability, and failure modes.
- Validate how benchmark results relate to real-world performance and evaluation needs.
- Write clear technical documentation and reports for research and engineering audiences. WHAT WE'RE LOOKING FOR
- Two to four years of experience in software engineering, machine learning engineering, or research, including at least two years focused on AI benchmarks, evaluations, or agent environments.
- Hands-on experience designing, implementing, and operating benchmarks or evaluation infrastructure for AI agents or large language models.
- Proficiency with Python, Docker, and Linux environments.
- Experience working with subject-matter experts to model workflows across technical or business domains and define evaluation criteria.
- Experience developing metrics, statistical analyses, or validation studies for benchmark quality and real-world relevance.
- Strong technical writing, attention to detail, and ability to work independently in an early-stage environment.
- Experience with reinforcement learning pipelines, data generation, or agent evaluation is useful. Published work or technical writing on AI evaluation is also valued. COMPENSATION & BENEFITS Annual salary range: $100,000 to $170,000 USD. Visa sponsorship is available. LOCATION On-site in Singapore, Singapore.
Tips for this job
Practical JobOpportunity guidance. These tips do not replace official rules or create new eligibility requirements.
- Tailor the CV and application to the responsibilities and required skills stated on the official employer page.
- Use concrete evidence of relevant work, projects and measurable results rather than generic claims.
- Confirm location, work authorization, remote restrictions and sponsorship terms before applying.
- Apply through the original employer or official recruitment destination shown on this page.
Verification notes
laptop-ats-crawler v3
Original authoritative source
JobOpportunity is the discovery and verification layer. Confirm eligibility, dates, salary/funding and application instructions on the original source before submitting anything.
Apply through JobOpportunity →