Verified current Job

AI Benchmarking Specialist, Performance Benchmarking Evaluation

At Amazon, we're evolving how to support sellers, customers in solving their query resolved faster using AI. That is where our team steps in, we support multiple AI models in benchmarking their responses for it relevancy, correctn...

Job Full source details
ADCI HYD 13 SEZ Hyderabad, Telangana, India Verified 10 hours ago
✓ 88% verification score · Source: Amazon Com Opportunities · Always confirm final requirements on the original source.
Complete source information imported The available role or programme description, requirements, benefits and source facts were imported from the public official endpoint and formatted for reading.
Employmentfull-time
CountryIndia

Overview

At Amazon, we're evolving how to support sellers, customers in solving their query resolved faster using AI. That is where our team steps in, we support multiple AI models in benchmarking their responses for it relevancy, correctness and completeness. You perform straightforward annotation related tasks across ML data process areas. You are able to determine the right task or procedure (or when a slight deviation is needed) to achieve desired outcome. You act as point of contact for one or more simple annotation tasks in a given ML data process area. You are responsible for the quality of execution for these tasks. You understand the dependencies of tasks across ML data workflows and are able to articulate customer impact. You use the existing annotation methods and modify the SOPs. You document the changes in SOP and once approved, you share the knowledge with the team, audit SOP adopti

Full job description

Full Job Description

At Amazon, we're evolving how to support sellers, customers in solving their query resolved faster using AI. That is where our team steps in, we support multiple AI models in benchmarking their responses for it relevancy, correctness and completeness.

  • You perform straightforward annotation related tasks across ML data process areas. You are able to determine the right task or procedure (or when a slight deviation is needed) to achieve desired outcome.
  • You act as point of contact for one or more simple annotation tasks in a given ML data process area. You are responsible for the quality of execution for these tasks.
  • You understand the dependencies of tasks across ML data workflows and are able to articulate customer impact. You use the existing annotation methods and modify the SOPs. You document the changes in SOP and once approved, you share the knowledge with the team, audit SOP adoption and execution.
  • You test new SOP/tools and share feedback on their quality and provide inputs for improvement to support on boarding of new tools.

Key job responsibilities

  • You structure data collection, analyse results and share inputs for SOP changes.
  • You collate, track, and report progress on key metrics agreed to with respective stakeholders (e.g., Program managers, Applied Scientist) specific to your functional area.
  • You identify operational issues related to process and tooling and recommend suggestions to improve key project metrics such as productivity and quality.

Basic Qualifications

  • Bachelor's degree
  • Speak, write, and read fluently in English
  • Experience with Microsoft Office products and applications
  • Experience with Excel

Preferred Qualifications

  • 0 to 1 years of equivalent experience
  • Performed annotation related tasks across ML data process areas.
  • Technical proficiency in SQL querying and Python programming for data analysis

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.

Requirements & qualifications

Bachelor's degree Speak, write, and read fluently in English Experience with Microsoft Office products and applications Experience with Excel

Tips for this job

Practical JobOpportunity guidance. These tips do not replace official rules or create new eligibility requirements.

  1. Tailor the CV and application to the responsibilities and required skills stated on the official employer page.
  2. Use concrete evidence of relevant work, projects and measurable results rather than generic claims.
  3. Confirm location, work authorization, remote restrictions and sponsorship terms before applying.
  4. Apply through the original employer or official recruitment destination shown on this page.

Verification notes

Public Amazon careers search record with employer description and structured locations.

Original authoritative source

JobOpportunity is the discovery and verification layer. Confirm eligibility, dates, salary/funding and application instructions on the original source before submitting anything.

Apply through JobOpportunity →

Browse current JobOpportunity listings from Amazon Com Opportunities →

More ways to save

Discover deals, coupons and free courses on our sister site.

Explore DealVorio
Save more with DealVorio: deals, coupons, free courses, apps and books