Overview
Be a key cross-functional team member, and close collaborator to product teams. Make significant contributions from translating research into benchmarks to co-authoring product roadmap. Design and build evaluation systems that test LLM capabilities in the healthcare domain, and interpret and communicate results. Run experiments to determine how different prompting techniques affect results on internal and industry benchmarks. Improve the tooling that is used internally to implement, run and analyze evaluations. Design internal benchmarking and regression testing capabilities that capture model accuracy, safety and utility. Bachelor's or Higher Degree in Computer Science, or related technical discipline AND significant Python programming experience / machine learning research. Experience building with and around LLMs including prompt / context and harness engineering. Deep expertise in ev
Full job description
Full Job Description
Be a key cross-functional team member, and close collaborator to product teams. Make significant contributions from translating research into benchmarks to co-authoring product roadmap. Design and build evaluation systems that test LLM capabilities in the healthcare domain, and interpret and communicate results. Run experiments to determine how different prompting techniques affect results on internal and industry benchmarks. Improve the tooling that is used internally to implement, run and analyze evaluations. Design internal benchmarking and regression testing capabilities that capture model accuracy, safety and utility. Bachelor's or Higher Degree in Computer Science, or related technical discipline AND significant Python programming experience / machine learning research. Experience building with and around LLMs including prompt / context and harness engineering. Deep expertise in evaluating and understanding the performance of agents. Experience collaborating in cross functional teams, working through ambiguity to deliver high quality results. Have 0 to 1 experience with a bias towards shipping and learning, while balancing a high-quality bar. Proven ability to collaborate and contribute to a positive, inclusive work environment, fostering knowledge sharing and growth within the team. Experience in healthcare technology, or experience in the health domain. Experience with data engineering - handling text dataset sourcing, curation, and processing tasks at scale. Demonstrated written and verbal communication skills with the ability to work closely with cross-functional teams, including product managers, designers, and other engineers. Passion for learning new technologies and staying up to date with industry trends, best practices, and emerging technologies and patterns in AI.
Tips for this job
Practical JobOpportunity guidance. These tips do not replace official rules or create new eligibility requirements.
- Tailor the CV and application to the responsibilities and required skills stated on the official employer page.
- Use concrete evidence of relevant work, projects and measurable results rather than generic claims.
- Confirm location, work authorization, remote restrictions and sponsorship terms before applying.
- Apply through the original employer or official recruitment destination shown on this page.
Verification notes
Verified from public schema.org JobPosting structured data on the official source page. The complete published description, responsibilities, requirements and benefits were normalized when present; unstated facts were not inferred.
JobOpportunity is the discovery and verification layer. Confirm eligibility, dates, salary/funding and application instructions on the original source before submitting anything.
Apply through JobOpportunity →Browse current JobOpportunity listings from Microsoft Careers →
