Overview
We're a fast-moving, lean EU team looking for a high-agency QA engineer based in Kosovo, with zero middle management. You'll work alongside our founder and our Kosovo engineering team, report directly to the Head of Engineering, and own quality end-to-end. We already have a Playwright/TypeScript E2E suite covering our core flows, but it needs a real owner. Your job is to own quality across the whole product: our Next.js web app, our backend APIs, and the AI document-extraction engine underneath. You'll broaden coverage across the app, make the suite trustworthy (fast, stable, and run on every change), and extend it into an AI-driven evaluation pipeline, so neither an everyday feature change nor a prompt or model change can silently break what we ship. Note: CVs filled with LLM slop will be immediately disqualified. We want to see your actual experience in your own words. What You'll Do O
Full job description
Your Mission
We're a fast-moving, lean EU team looking for a high-agency QA engineer based in Kosovo, with zero middle management. You'll work alongside our founder and our Kosovo engineering team, report directly to the Head of Engineering, and own quality end-to-end.
We already have a Playwright/TypeScript E2E suite covering our core flows, but it needs a real owner. Your job is to own quality across the whole product: our Next.js web app, our backend APIs, and the AI document-extraction engine underneath. You'll broaden coverage across the app, make the suite trustworthy (fast, stable, and run on every change), and extend it into an AI-driven evaluation pipeline, so neither an everyday feature change nor a prompt or model change can silently break what we ship.
Note: CVs filled with LLM slop will be immediately disqualified. We want to see your actual experience in your own words.
What You'll Do
- Own quality across the product: Full E2E and regression coverage of our Next.js frontend and backend APIs, from critical user journeys to the edge cases, so every release ships with confidence.
- Make the suite trustworthy: Take ownership of our existing Playwright/TypeScript suite, drive down flakiness, and get it green and trusted as a CI gate the team relies on.
- Harden the email → extraction → downstream pipeline: We have early coverage of our email-to-document flow. You'll turn it into the rock-solid regression backbone that protects extraction accuracy and downstream processing.
- Build AI evaluation, not just pass/fail: Use Langfuse to build eval harnesses for extraction quality, golden datasets, scoring against ground truth, and drift detection that gates on quality, not just green checks.
- Use AI to test faster: Lean on AI to generate cases and synthetic documents, triage failures, and review diffs, wherever it makes the suite broader and faster than hand-written tests alone.
- Make testability a first-class concern: Partner with engineers so features ship testable by design, with clear, actionable failure reporting.
Qualifications And Requirements
- Full-stack app testing: You test real web apps end to end, UI flows, API contracts, and data integrity, and you know what makes a suite flaky and how to fix it. Strong with Playwright and comfortable in TypeScript and/or Python.
- 3+ years in QA / test engineering: We hire experienced QA only. What matters is depth of ownership: you've built and run automation strategy and frameworks yourself, not just executed someone else's test plans. A sharp engineer who's shipped real automation beats a coaster with twice the tenure.
- High Agency: You don't wait for perfectly scoped tickets. You unblock yourself, make pragmatic decisions, and take extreme ownership of quality.
- AI-first instincts: You actively use AI in your testing workflow, and you're keen to take on the harder problem of testing non-deterministic AI output, or can show real aptitude for it.
Nice to Have
- Experience testing non-deterministic systems: evaluating LLM/ML outputs, building eval sets, reasoning about quality metrics over pass/fail.
- Data/ML pipeline testing or MLOps background.
- Document processing, OCR, or data-extraction domain experience.
- Familiarity with Prisma, async/queue-based systems, or complex event patterns.
Why Us?
- High Ownership & Influence: You'll define how Holocene does QA. Your test strategy directly protects the product and the roadmap.
- Elite, Lean Team: Work with highly technical peers in an environment where experimentation is expected and red tape is nonexistent.
- Fast Decisions: We respect your time. Just 2 interviews, with a final decision within 48 hours.
Requirements & qualifications
- Full-stack app testing: You test real web apps end to end, UI flows, API contracts, and data integrity, and you know what makes a suite flaky and how to fix it. Strong with Playwright and comfortable in TypeScript and/or Python.
- 3+ years in QA / test engineering: We hire experienced QA only. What matters is depth of ownership: you've built and run automation strategy and frameworks yourself, not just executed someone else's test plans. A sharp engineer who's shipped real automation beats a coaster with twice the tenure.
- High Agency: You don't wait for perfectly scoped tickets. You unblock yourself, make pragmatic decisions, and take extreme ownership of quality.
- AI-first instincts: You actively use AI in your testing workflow, and you're keen to take on the harder problem of testing non-deterministic AI output, or can show real aptitude for it.
Nice to Have
- Experience testing non-deterministic systems: evaluating LLM/ML outputs, building eval sets, reasoning about quality metrics over pass/fail.
- Data/ML pipeline testing or MLOps background.
- Document processing, OCR, or data-extraction domain experience.
- Familiarity with Prisma, async/queue-based systems, or complex event patterns.
Tips for this job
Practical Job and Scholarship guidance. These tips do not replace official rules or create new eligibility requirements.
- Tailor the CV and application to the responsibilities and required skills stated on the official employer page.
- Use concrete evidence of relevant work, projects and measurable results rather than generic claims.
- Confirm location, work authorization, remote restrictions and sponsorship terms before applying.
- Apply through the original employer or official recruitment destination shown on this page.
Verification notes
Discovered directly from the employer’s published Personio open-positions XML feed. All public jobDescriptions sections were normalized into complete candidate-facing content.
Job and Scholarship is the discovery and verification layer. Confirm eligibility, dates, salary/funding and application instructions on the original source before submitting anything.
Holocene Gmbh Careers ↗Browse current Job and Scholarship listings from Holocene Gmbh Careers →
