The Agent Is Already Warm When You Arrive

A terminal-based coding question needs a real environment underneath it, a real machine, a real shell, a real coding agent running inside it, and a real environment takes a moment to start. That moment used to happen in front of the candidate, right as the clock on their question began, so the first minute of a timed question could be spent watching a terminal boot rather than reading the problem.

What changed

The environment now starts warming up in the background from the moment a session begins, while a candidate is still reading the pre-assessment instructions, before any timer has started on any question. Once the first question's environment is ready, every other question in the assessment quietly starts warming up too, in the background, so moving from one question to the next finds an environment already running instead of starting from zero each time.

Getting this right took a few passes. An early version warmed each question's environment independently, which worked but meant several environments booting at once could compete with each other and slow each other down. That was fixed so environments come up in a steadier sequence instead of all fighting for the same startup resources at once. A related fix made sure this pre-warming works correctly no matter which combination of agents a question grants, since the pre-warming logic had to be taught about the same expanded, independent agent choices described in a separate update.

Why it matters

None of that startup time is signal. It does not measure anything about how someone codes, how they think, or how they communicate, and a candidate should not lose working minutes of a timed assessment to it. Moving the wait to before the clock starts, into a moment that was already happening anyway while instructions are read, is a straightforward trade: the same real environment, with less dead time standing between a candidate and the actual question.

It also removes a small but real source of unfairness between candidates. Two people taking the same assessment at different times should face the same conditions, and a cold start whose length can vary run to run is one more variable that has nothing to do with either person's skill. Warming the environment ahead of time takes that variable out of the comparison entirely.

A background environment doing this work also has to be legible when a candidate switches away from it and back. A tab whose agent is still working lights up while it runs and marks itself once the work finishes somewhere off screen, the same signal used everywhere a coding agent can keep going while the candidate's attention is elsewhere. Pre-warming and that visibility are really the same idea applied at two different moments: start the work before it is needed, and never let a candidate wonder whether something is happening where they cannot currently see it.

A watchdog runs alongside that signal for a reason worth mentioning: nothing about a background process is allowed to leave a tab looking busy forever if the work has actually finished or gone idle. A signal that can get stuck on is worse than no signal, since a candidate learns to distrust it, so the same infrastructure that starts things early also has to be strict about clearing itself when there is genuinely nothing left running.

Where it points

The same principle keeps showing up across the product: time a candidate cannot control should not come out of time meant to measure them. Anywhere that is still true, a loading spinner standing in for something that could have started earlier, is worth finding and fixing the same way.