Overview
Evaluation & Quality — Primary Focus Own the evaluation stage end to end: the eval suite and its baseline, measurement across quality, accuracy, latency, cost, trust, DLP behavior, and source scoping, failure triage, and the advisory-board quality gate. Make evaluation continuous rather than a milestone, keep each agent's eval suite in correspondence with its declared success criteria, and run A/B testing with BizChat plus post-release drift detection. Protect the trust boundary: gate sign-off, accepting a known failure, and admin consent stay accountable human decisions, and every gate decision stays auditable. Ensure APIs are composable, predictable, and production‑ready for enterprise workloads Partner with security, privacy, and compliance teams to define safe defaults and guardrails Avoid over‑inference and ambiguity that could erode enterprise trust Changing the Way We Work: apply
Full job description
Full Job Description
Evaluation & Quality — Primary Focus Own the evaluation stage end to end: the eval suite and its baseline, measurement across quality, accuracy, latency, cost, trust, DLP behavior, and source scoping, failure triage, and the advisory-board quality gate. Make evaluation continuous rather than a milestone, keep each agent's eval suite in correspondence with its declared success criteria, and run A/B testing with BizChat plus post-release drift detection. Protect the trust boundary: gate sign-off, accepting a known failure, and admin consent stay accountable human decisions, and every gate decision stays auditable. Ensure APIs are composable, predictable, and production‑ready for enterprise workloads Partner with security, privacy, and compliance teams to define safe defaults and guardrails Avoid over‑inference and ambiguity that could erode enterprise trust Changing the Way We Work: apply an agentic loop approach to evaluation so evidence assembly, failure triage, and drift detection run zero-touch — while irreversible or customer-visible decisions stay human. Leading with New AI Products: dogfood Work IQ's own surfaces to build those loops, and turn that developer experience into platform investments. Lead with outcomes and a published point of view: less human effort per agent shipped, faster resolution of eval failures, drift caught before customers see it — and never a false gate pass. Design agentic usage patterns, and enable various scenarios Shape developer experience across SDKs, CLI, MCP integrations, and documentation Establish strong collaboration partnership and trust with local and global engineering stakeholders. Bachelor's Degree AND 8+ years experience in product/service/program management or software development These requirements include but are not limited to the following specialized security screenings: Bachelor's Degree AND 12+ years experience in product/service/program management or software development OR equivalent experience. 4+ years experience taking a product, feature, or experience to market (e.g., design, addressing product market fit, and launch, internal tool/framework). 6+ years experience improving product metrics for a product, feature, or experience in a market (e.g., growing customer base, expanding customer usage, avoiding customer churn). 6+ years experience disrupting a market for a product, feature, or experience (e.g., competitive disruption, taking the place of an established competing product). Experience owning or evolving developer‑facing APIs or platforms.
Tips for this job
Practical JobOpportunity guidance. These tips do not replace official rules or create new eligibility requirements.
- Tailor the CV and application to the responsibilities and required skills stated on the official employer page.
- Use concrete evidence of relevant work, projects and measurable results rather than generic claims.
- Confirm location, work authorization, remote restrictions and sponsorship terms before applying.
- Apply through the original employer or official recruitment destination shown on this page.
Verification notes
Verified from public schema.org JobPosting structured data on the official source page. The complete published description, responsibilities, requirements and benefits were normalized when present; unstated facts were not inferred.
JobOpportunity is the discovery and verification layer. Confirm eligibility, dates, salary/funding and application instructions on the original source before submitting anything.
Apply through JobOpportunity →Browse current JobOpportunity listings from Microsoft Careers →
