The Agent Is Already Warm When You Arrive

A terminal-based coding question runs on a real environment (a machine, a shell, and a coding agent running inside it) that takes a moment to start. That moment used to happen in front of the candidate, right as the clock on their question began, so the first minute of a timed question could be spent watching a terminal boot rather than reading the problem.

What changed

The environment now starts warming up in the background from the moment a session begins, while a candidate is still reading the pre-assessment instructions, before any timer has started on any question. Once the first question's environment is ready, every other question in the assessment quietly starts warming up too, in the background, so moving from one question to the next finds an environment already running instead of starting from zero each time.

Getting this right took a few passes. An early version warmed each question's environment independently, which worked but meant several environments booting at once could compete with each other and slow each other down. That was fixed so environments come up in a steadier sequence instead of all fighting for the same startup resources at once. A related fix taught pre-warming about questions that grant more than one agent, so it now works for any combination of agents.

Why it matters

None of that startup time is signal. It does not measure anything about how someone codes, how they think, or how they communicate, and a candidate should not lose working minutes of a timed assessment to it. Moving the wait to before the clock starts, into a moment that was already happening anyway while instructions are read, is a straightforward trade: the same real environment, with less dead time standing between a candidate and the actual question.

It also removes a small but real source of unfairness between candidates. Two people taking the same assessment at different times should face the same conditions, and a cold start whose length can vary run to run is one more variable that has nothing to do with either person's skill. Warming the environment ahead of time takes that variable out of the comparison entirely.

The tab signal that marks an agent still working off screen is used everywhere an agent can run in the background. Pre-warming and that signal are the same idea at two moments: start the work before it is needed, and never let a candidate wonder whether something is happening where they cannot see it.

A watchdog runs alongside that signal so that no background process can leave a tab looking busy forever after the work has finished or gone idle. A signal that can get stuck on is worse than no signal, since a candidate learns to distrust it, so the same infrastructure that starts things early also has to be strict about clearing itself when there is genuinely nothing left running.

Where it points

The same principle keeps showing up across the product: time a candidate cannot control should not come out of time meant to measure them. Any place where a loading spinner still stands in for something that could have started earlier is worth finding and fixing the same way.