You build environments and graders that models can't cheat. Turn real agent failures into benchmarks that matter.
A profitable seed-stage startup in San Francisco builds AI agents that handle complete business tasks, such as chasing invoices over Slack or scheduling over email. It produces a couple of hundred thousand agent traces every week, and all its customers so far came through referrals. As one of the first two engineers, you'll take real failures from those traces and turn them into simulated companies (inboxes, CRMs, records, files) with solid graders and benchmarks that predict how agents perform in production. Your work feeds product evals and data and RL partnerships with AI labs.
They're hiring 2 founding engineers.
What you'll own
- Simulated business software that holds up under agent tool calls
- Graders that resist reward hacking
- Benchmarks whose rankings match real-world performance
- Datasets and environments for evals and for training with AI labs
Must-haves
- Proof that you built (or helped build) an RL environment, benchmark, dataset or eval system that was actually used to train or test models
- Data-company experience only counts if it involved synthetic data, post-training or evaluating models, not just labelling
- You can mimic business software like an email inbox or CRM so it survives an agent's tool calls, and you write graders that are hard to game
- 1+ year of experience
- Working from the San Francisco office Monday to Friday, 9-9-5
Bonus points
- An environment or benchmark used by a frontier AI lab
- A grader you caught being reward-hacked and fixed, or a real flaw you found in a well-known benchmark
- Founder experience, a founder fellowship, or strong results in maths, programming, debate or game competitions
What's in it for you
- $150k–$300k base + 0.1–2% equity
- A founding role at a profitable, growing company
- No hour tracking, unlimited AI budget and lunch every day
- Visa sponsorship available
Good to know
- Full-time, on-site in San Francisco, 9-9-5
- This is an environments and data role, not product engineering or pure research
- Process: founder conversation → technical deep dive on a project from your CV → a 2–4 day work trial or a 1–2 day take-home → references → offer
When you apply, please include
- Your CV (PDF), your LinkedIn URL, and GitHub if you have one
- An environment, benchmark, dataset or eval you built, and how it was used
- Your current location, and confirmation that five days a week in San Francisco works for you
- Your visa or citizenship status
- Your base salary expectation, your notice period, and any other interview processes you're in
Does this sound like you? Apply now. If you're a match, we'll be in touch within 3 working days.
RnB Consultancy | Netherlands | 3604 AB Maarssen | KvK 71489703