Responsibilities of the Candidate:

The core agent loop and its execution model, covering orchestration, concurrency, sub-agent coordination, cancellation, timeouts, checkpointing and resume.
Context and state management for long-running tasks, including retention, compaction, and how an agent reasons over what it has already done.
The shared tool protocol, its schemas, versioning, and the internal libraries every agent team builds against.
The execution substrate. Sandboxed environments that are reproducible, snapshot-able and resource-bounded, and that behave identically for a repository task, a training run and a data analysis.
Failure semantics. Retry policy, partial failure, idempotency, and the distinction between a recoverable error and a task that should stop and hand back.
The trace schema every agent emits, which serves at once as our debugging surface, our training signal, and the audit record our customers keep.

Evals :

Task suites for each agent type, built from real work rather than synthetic benchmarks, and the infrastructure to run them at scale in parallel.
Verifiers and scoring. Deterministic checks where the task allows it, model-graded rubrics where it does not, and calibration of the graders themselves.
Regression gates that run on every harness change, with cost and latency accounted alongside quality.
The statistics to say a result is real. Seeds, variance, pass@k, confidence, and knowing how many runs are needed before anyone claims an improvement.
Turning observed production failures into permanent test cases.
Tooling for how we work. Trace inspection, replay, and diffing one run against another.

Strong software engineering fundamentals and advanced Python. You have shipped and operated production services, not only written them.
Concurrency and distributed systems. Async execution, worker pools and queues, retries and idempotency, backpressure, cancellation, and reasoning clearly about what happens when a step fails halfway through.
Containers and sandboxing. Docker and OCI internals, resource isolation, reproducible environments, and an understanding of why a job that passes locally fails in a sandbox.
Test and CI infrastructure. You have built test harnesses, run large suites in parallel, and dealt with flakiness as an engineering problem rather than an annoyance.
Measurement literacy. Comfort with variance, sampling and significance, and healthy skepticism toward benchmark results including your own.
Observability instincts. Distributed tracing, structured logging, and building the inspection tooling that makes non-deterministic systems debuggable.
You debug systematically. You find out what actually happened rather than adjusting things until the symptom disappears.
You are comfortable when problems are loosely specified and ownership is assumed rather than assigned.
Experience with evaluation or benchmarking work for AI systems is a strong plus.
Experience with agentic systems, LLM inference and serving, or developer tooling is a plus.
Open source contributions we can read are a plus.