Overview
An agent platform - an extensible, Model Context Protocol (MCP)-based system of AI diagnostic skills integrated with enterprise data services, run inside engineers' existing tools (CLI, VS Code, and chat). - Production engineering rails - CI-enforced quality gates, evaluation regression testing, comprehensive data science telemetry, and supply-chain security that let us ship AI safely and at scale. Every system is instrumented to surface performance, adoption, and quality signals in real time. - Language-model pipelines - training, fine-tuning (including RLHF), and automated evaluation of models that reason over and assess support cases. - Quality-evaluation systems - rubric-based, LLM-as-judge evaluation that scores case handling and surfaces coaching and compliance signals. - Intelligent case-routing services - classification and ranking models that route cases to the right team and ca
Full job description
Full Job Description
- An agent platform - an extensible, Model Context Protocol (MCP)-based system of AI diagnostic skills integrated with enterprise data services, run inside engineers' existing tools (CLI, VS Code, and chat). - Production engineering rails - CI-enforced quality gates, evaluation regression testing, comprehensive data science telemetry, and supply-chain security that let us ship AI safely and at scale. Every system is instrumented to surface performance, adoption, and quality signals in real time. - Language-model pipelines - training, fine-tuning (including RLHF), and automated evaluation of models that reason over and assess support cases. - Quality-evaluation systems - rubric-based, LLM-as-judge evaluation that scores case handling and surfaces coaching and compliance signals. - Intelligent case-routing services - classification and ranking models that route cases to the right team and catch misrouted cases early. - Privacy-preserving data tooling - a redaction pipeline (pattern-matching → transformer models → LLM) that protects PII and secrets across every data flow, forming the trusted substrate the rest of the platform depends on. Bachelor's Degree in Computer Science, Information Technology (IT), or related field AND 3+ years technical support, technical consulting experience, or information technology experience OR equivalent experience. These requirements include, but are not limited to the following specialized security screenings: 3+ years of experience developing and shipping production software services using TypeScript/Node.js, Python, or C#/.NET. Experience building AI-powered applications, including large language models (LLMs), AI agents, prompt engineering, Model Context Protocol (MCP), machine learning models, or text classification/ranking systems. Experience designing cloud and data solutions using Azure, large-scale datasets, Azure Data Explorer (Kusto), and retrieval technologies such as keyword, vector, or semantic search. Experience evaluating and improving AI systems through model evaluation frameworks, RLHF, preference-based training, LLM-as-a-judge methodologies, labeled datasets, or benchmark development. Experience implementing software quality, security, and DevOps practices, including automated testing, code reviews, CI/CD pipelines, GitHub Actions, CodeQL, dependency management, secret scanning, and branch governance. Experience developing customer-facing or enterprise applications, including solutions for privacy, compliance, PII detection, data redaction, customer support, or operational workflows.
Tips for this job
Practical JobOpportunity guidance. These tips do not replace official rules or create new eligibility requirements.
- Tailor the CV and application to the responsibilities and required skills stated on the official employer page.
- Use concrete evidence of relevant work, projects and measurable results rather than generic claims.
- Confirm location, work authorization, remote restrictions and sponsorship terms before applying.
- Apply through the original employer or official recruitment destination shown on this page.
Verification notes
Verified from public schema.org JobPosting structured data on the official source page. The complete published description, responsibilities, requirements and benefits were normalized when present; unstated facts were not inferred.
JobOpportunity is the discovery and verification layer. Confirm eligibility, dates, salary/funding and application instructions on the original source before submitting anything.
Apply through JobOpportunity →Browse current JobOpportunity listings from Microsoft Opportunities →
