Overview
About the Role Lead the development of agent intelligence that helps mechanical engineers complete multi-step workflows in desktop engineering software. You will combine hands-on engineering, technical leadership, and collaboration with user research and domain experts to improve real-world task completion and product value. What You'll Do Own agent task success metrics, establish baselines, and systematically improve completion rates across engineering workflows. Design and build reproducible evaluation infrastructure based on validated user stories, including tests for cost efficiency and failure modes. Set per-task token budgets and track the cost of completed workflows. Interview engineers with research and domain teammates, document their workflows, and translate validated stories into benchmarks. Prioritize workflow coverage based on customer value and technical feasibility. Set th
Full job description
Full Job Description
About the Role
Lead the development of agent intelligence that helps mechanical engineers complete multi-step workflows in desktop engineering software. You will combine hands-on engineering, technical leadership, and collaboration with user research and domain experts to improve real-world task completion and product value.
What You'll Do
-
Own agent task success metrics, establish baselines, and systematically improve completion rates across engineering workflows.
-
Design and build reproducible evaluation infrastructure based on validated user stories, including tests for cost efficiency and failure modes.
-
Set per-task token budgets and track the cost of completed workflows.
-
Interview engineers with research and domain teammates, document their workflows, and translate validated stories into benchmarks.
-
Prioritize workflow coverage based on customer value and technical feasibility.
-
Set the agent architecture for tool calling, state management, error recovery, model routing, and context management.
-
Lead a small AI engineering team while writing production code, reviewing designs, and unblocking teammates.
-
Work with product, integrations, and customers during proof-of-concept projects to align agent behavior with real usage.
What We're Looking For
-
At least 7 years of software engineering experience, including 2 or more years building LLM-based agents that take real-world actions.
-
Strong Python skills and deep experience with LLM application architecture, including model selection, retrieval, context management, orchestration, and tool calling.
-
Experience building rigorous agent evaluation and benchmarking systems that measure task completion, cost, and failure modes.
-
Technical leadership experience on a small engineering team, with a track record of reviewing code and continuing to contribute production software.
-
Experience shipping AI or LLM tools using proprietary engineering data or desktop engineering software.
-
Familiarity with function calling, tool APIs, tracing or observability, and evaluation tools.
-
Experience with desktop automation or application APIs, such as COM, and a background in mechanical engineering software, CAD/CAE, PLM, or an adjacent domain.
-
Understanding of deployment constraints on managed enterprise workstations.
Compensation & Benefits
Base salary: $160,000 to $250,000 USD annually. Visa sponsorship is not available.
Location
On-site in San Francisco, California, United States.
Tips for this job
Practical JobOpportunity guidance. These tips do not replace official rules or create new eligibility requirements.
- Tailor the CV and application to the responsibilities and required skills stated on the official employer page.
- Use concrete evidence of relevant work, projects and measurable results rather than generic claims.
- Confirm location, work authorization, remote restrictions and sponsorship terms before applying.
- Apply through the original employer or official recruitment destination shown on this page.
JobOpportunity.info helps you discover and organize source listings. Confirm eligibility, dates, salary/funding and application instructions on the original source before submitting anything.
Apply through JobOpportunity →