One Question, More Than One Agent

An agentic question used to force a choice before the candidate ever saw it: our own agent, or one outside CLI, picked once by the person building the assessment. That was a simplification we did not need to keep.

What changed

A question can now grant a candidate more than one coding agent at the same time, our own in-browser agent alongside outside CLIs. When more than one is granted, the candidate sees a tab per agent and can move between them mid-question. Each tab keeps working while another is open, so switching tabs to check on something does not pause the one you left.

The same choice now works the same way in three places: a hiring assessment, a classroom coding assignment, and the self-serve building product. Set it up once, understand it everywhere, rather than relearning a different picker for each.

Two bugs surfaced and got fixed along the way, which is worth being honest about rather than only describing the feature as it looks now. A question configured to grant more than one agent could silently lose part of that grant, saving fewer agents than the assessment actually specified. And a batch of library questions had quietly reverted to an older, more limited setup because a gap in how the runtime read saved configurations made the newer option unreachable. Multi-agent support only means something if the agents you configured are the agents that actually show up, so both were treated as real bugs, not edge cases.

Why it matters

Real engineers do not use one tool. They reach for whichever one fits the moment, and switch without thinking about it, sometimes mid-task. A question that only allows one agent measures how well someone performs inside an unfamiliar constraint, not how they actually work day to day.

Letting a candidate move between agents, and showing a reviewer which one they reached for and how much, is a better proxy for the skill everyone actually cares about: judgment about which tool suits which task, not memorized loyalty to whichever one the test happened to allow. A candidate who tries one agent, decides it is not converging, and switches to another mid-question is demonstrating exactly the kind of tool judgment a single-agent test cannot see at all.

The condition still comes first. A recruiter sets how much AI help a question allows before deciding which agents are on offer, and a "No AI" question shows plainly that no agent was ever available, rather than leaving the reviewer to infer it. Multi-agent choice widens what a candidate can do inside the rules a recruiter already set, it does not loosen the rules themselves.

The classroom picture is a little different from the hiring one, but the underlying capability is the same three-way choice. A teacher setting up a coding assignment can hand students one specific agent, or let them choose their own between our agent, one outside CLI, or the other, the same decision a recruiter makes when building an assessment, just answering a teaching question instead of a hiring one.

Where it points

The tab a candidate is not looking at still needs to be legible when they come back to it. A still-working tab now lights up while it runs and marks itself once it finishes somewhere off screen, so nothing about a background agent's progress goes unnoticed just because attention was elsewhere. Making multi-agent work feel calm instead of busy, rather than turning one question into three things to babysit at once, is the ongoing project.