Overview
About the Role This is a hands-on infrastructure engineering role at an early-stage enterprise AI company building a context and data governance layer for AI agents deployed in highly regulated industries. You will own the inference and model-serving infrastructure end to end, making production AI agents fast, reliable, and scalable as concurrency grows. What You'll Do Design, build, and own inference and model-serving infrastructure from initial architecture through production deployment. Scale systems that enable AI agents to run reliably and efficiently under increasing concurrent load. Identify and resolve infrastructure bottlenecks in collaboration with ML and platform engineering teams. Drive performance optimization across latency, throughput, and reliability for production workloads. What We're Looking For 5+ years building and operating ML inference systems, model-serving platfo
Full job description
Full Job Description
About the Role
This is a hands-on infrastructure engineering role at an early-stage enterprise AI company building a context and data governance layer for AI agents deployed in highly regulated industries. You will own the inference and model-serving infrastructure end to end, making production AI agents fast, reliable, and scalable as concurrency grows.
What You'll Do
-
Design, build, and own inference and model-serving infrastructure from initial architecture through production deployment.
-
Scale systems that enable AI agents to run reliably and efficiently under increasing concurrent load.
-
Identify and resolve infrastructure bottlenecks in collaboration with ML and platform engineering teams.
-
Drive performance optimization across latency, throughput, and reliability for production workloads.
What We're Looking For
-
5+ years building and operating ML inference systems, model-serving platforms, or ML infrastructure in production environments.
-
Hands-on experience designing and scaling inference-serving systems using frameworks such as TensorFlow Serving, TorchServe, Triton, KServe, or equivalent custom solutions.
-
Strong distributed systems fundamentals, including experience managing concurrent requests and resource allocation under load.
-
Proficiency with containerization and orchestration technologies, particularly Docker and Kubernetes, for ML workloads.
-
Experience with cloud infrastructure platforms (AWS, GCP, or Azure) for deploying and managing ML systems.
-
Solid monitoring and observability skills using tools such as Prometheus, Grafana, ELK, or distributed tracing solutions.
-
Proficiency in at least one systems or backend language: Python, Go, Rust, C++, or Java.
-
Familiarity with knowledge graphs, semantic search, or graph databases is a plus.
-
Background in agentic or autonomous AI systems, real-time inference, or enterprise data infrastructure is a plus.
Location
On-site in San Mateo, California, United States. Visa sponsorship is not available for this role.
Tips for this job
Practical Job and Scholarship guidance. These tips do not replace official rules or create new eligibility requirements.
- Tailor the CV and application to the responsibilities and required skills stated on the official employer page.
- Use concrete evidence of relevant work, projects and measurable results rather than generic claims.
- Confirm location, work authorization, remote restrictions and sponsorship terms before applying.
- Apply through the original employer or official recruitment destination shown on this page.
Verification notes
Discovered directly from the employer’s public Ashby Job Postings API. The complete public role content and compensation metadata were normalized into safe candidate-facing sections.
Job and Scholarship is the discovery and verification layer. Confirm eligibility, dates, salary/funding and application instructions on the original source before submitting anything.
Clera (ashby) ↗Browse current Job and Scholarship listings from Clera (ashby) →
