How to Assess Developers Who Code With AI

The assessment most companies still run was built to answer one question: can this person produce correct code on their own, from a blank file, with nothing in the room but what is in their head. For a long time that was a fair proxy for whether someone could do the job. It has quietly stopped being one. Once a model can generate a plausible, test-passing solution to most interview-shaped problems on demand, a green check no longer tells you who wrote the code, whether the person understood it, or whether they would have caught it on the day it broke. The skill worth measuring moved. This piece is about where it went, and how you capture it instead of pretending the move never happened.

The signal that quietly stopped working

The old model rewarded unaided production, and it worked because production was hard. Writing a correct solution from scratch took knowledge, care, and a working model of the problem in your head, so the output was a reasonable stand-in for all three. That link is what broke. When anyone can prompt a plausible answer into existence, the finished artifact stops carrying evidence about the person who submitted it. Two candidates hand in the same passing code. One reasoned through it and one pasted the first thing that compiled. The grader that only reads the final files ranks them as equals and moves on, which means it is scoring luck and calling it skill.

We have made a version of this argument on the marketing side too. Anyone can write "we measure real skill" on a landing page. The claim only becomes trustworthy when a skeptical reader can check it against the actual work. The same shift is happening one level down, inside the assessment itself. A passing test used to be scarce evidence. Now the answer is the cheap part of the transaction, and the expensive part, the judgment that produced it, is exactly what the old format throws away.

The question moved: direct, inspect, verify, correct

If producing code is no longer the scarce thing, then directing the production of it is. The question worth asking a candidate in 2026 is not whether they can write a function unaided. It is whether they can point a capable tool at a real problem, read what comes back, confirm it does what they meant, and steer it when it does not. Those four moves are the job now, on most days, for most working developers.

Each move is a place where skill either shows up or fails to. Direction is precision: turning a vague intention into a request specific enough that the tool cannot wander off. Inspection means reading a generated diff before you keep it, instead of trusting that green means good. Verification is the spine of the whole thing. A model will always generate faster than a person can read, so the skill that separates people is deciding what has to be true and then actually checking it. Correction is the recovery move: when the check fails, the candidate has to recognize a subtly wrong answer and pull it back on course. None of that is visible in the final file. All of it is visible in how the work was done, which is the part a modern assessment has to be built to keep.

Put the tool where you can see it

The first instinct when you add AI to an assessment is to give it its own room. A separate agent lane, a private set of files, a special button to promote its work into the candidate's. It looks impressive and it measures almost nothing, because the reviewer ends up staring at a polished final file with no way to tell whether the candidate wrote it, directed it, or merely inherited it from a run they never read.

The better shape is duller and far more useful. Agents are chats, and chats work on the same set of files the candidate does. When the tool proposes a change, that change lands through the same review path as any other edit, and the candidate keeps it or rejects it out in the open. This does not make AI assistance disappear. It makes it inspectable, which is the entire point. There is one artifact, so the diff, the attribution, and the validation proof can all refer to the same thing.

Sharing the workspace carries a cost we had to design against. One set of files means one blast radius, and a tool that rewrites a module the candidate was mid-thought on can erase the very context a reviewer needs. Knowing which line the tool wrote does not tell you whether it clobbered a better idea underneath. The fix is not to hand the agent its own sandbox again, which drops you right back into the ceremony you were trying to avoid. It is to make agent edits arrive as proposals the candidate resolves, so any change to shared state comes with a human decision attached to it.

Attribution: telling direction from blind acceptance

Once the tool works in the open, attribution stops being a cosmetic label and becomes part of what you are scoring. A line can be written by the candidate, by a guided assistant, or by an agent. A candidate can read a diff before keeping it, keep it without reading, reject it after inspection, or ask a sharper follow-up. Those are different behaviors, and flattening them into a single "used AI" flag throws away the only thing you were trying to learn.

Here is why that distinction is load-bearing. Picture the same small take-home task, one with a known edge case around empty input, handed to two candidates. Both reach for an agent. Both submit code that passes the visible tests. Read only the final files and you would rank them identically.

The trail pulls them apart. The first candidate asks for a plan before any edit, reads the proposed diff, and rejects the first version because it mishandles the empty case. She reruns the check, watches it fail, tightens her request, and keeps the second version only once the behavior matches what she described. Her session has friction in it, and the friction is the signal: a question with intent, a rejected diff, a verification step that reacted to a real failure. The second candidate accepts every suggestion on the first pass. The tool happens to produce working code, so the tests go green, but there is no rejected diff, no follow-up, no moment where the candidate caught something the tool missed. The empty case is correct by luck. When a reviewer opens that trail, the absence is loud, a straight line from prompt to submit with no reading in between.

Neither candidate is disqualified for using the tool. That was never the question. What attribution buys is the ability to tell a directed session from a delegated one without interrogating anyone afterward, and without pretending the tool was not in the room. Accepting a good suggestion blindly and steering a tool toward a good suggestion look identical in the artifact and nothing alike in the record of how it was made.

Validation you can see, not a gate you cannot

For project and interface work, the final code is not the whole artifact either. What the candidate checked, and how they reacted when a check failed, is often the more revealing half. So validation has to show up as evidence a reviewer can read, not as a hidden pass or fail buried in the pipeline. Browser validation, rendered snapshots, viewport checks, console findings, replay of what actually ran: these only help if a reviewer can see what was tested and whether the candidate reacted to what it found. A green result with no context is just another badge to take on faith. A useful proof says what the goal was, what steps ran, where the rendered result was inspected, and what happened on failure. That is the difference between "the candidate used AI" and "the candidate used AI, opened the preview, caught the layout collapsing on a narrow screen, and asked for a targeted fix." The second sentence is evidence. The first is theater. This is why validation belongs on the surface rather than in a locked gate. The candidate should be able to run checks, the tool should be able to request them when the work calls for it, and the reviewer should see enough of the path to tell a tight verify loop from a last-second batch check bolted on at the end.

One answer stopped being enough

A single algorithm answer rarely reveals judgment, because the path to it is short and most of the decisions happen in someone's head. A larger, multi-step task has more surface area for judgment, or its absence, to show. Where did the candidate scope the work, what did they verify before moving on, how did they recover when something broke. Project-style tasks make those moments legible in a way a one-shot answer cannot, which is why we pushed assessments toward them.

That richer work needs a report that can hold more than a number. Ours splits the judgment into a small set of named competencies: problem solving and deliverable quality, planning and decomposition, direction and communication with the tool, verification and iteration, and how independently the candidate ran the agentic workflow.

A handful of axes is a deliberate choice. One overall score is easy to publish and impossible to argue with, which is the problem, because it hides the disagreement instead of showing it. Splitting the call into named dimensions forces the report to say where a candidate was strong and where they were not, and it gives a reviewer somewhere to push back. Different roles weight those axes differently. A front-end build leans harder on rendered evidence, a debugging task on iteration and verification, a deliberate no-AI baseline sets direction to zero. The weights move underneath, but the language stays fixed, so a reviewer who reads dozens of these a week learns one report shape and never has to relearn it.

A trail a reviewer can actually read

Visibility has a failure mode, and it is the opposite of the one people expect. The risk is rarely too little evidence. It is too much. A reviewer works against a budget of minutes per candidate, and a record that captures every keystroke, every tool turn, and every intermediate diff can bury the two moments that actually mattered under twenty minutes of scrubbing. Evidence that cannot be read on a reviewer's clock is closer to noise than proof.

So capturing the trail is only half the work. The other half is ranking it, surfacing the rejected diff and the reacted-to failure ahead of the many edits that changed nothing. Volume is a tempting proxy and a bad one. A candidate who typed a lot did not necessarily think a lot, and a short session can hold the sharpest decision in the room. The discipline is to capture what the candidate acted on, not every twitch of the cursor, because a raw stream of every keypress is noise wearing a lab coat, and a candidate who feels watched rather than assessed starts performing for the log instead of solving the problem.

Make the reviewer's side concrete. Two candidates leave rich sessions behind. One trail has been ranked, so the reviewer opens it and lands directly on the moment the candidate pushed back on the tool and the check that caught the failure, and reaches a read in about a minute. The other is a flat, unranked haystack of every action in order, and the same reviewer scrubs for ten minutes, gives up, and falls back to the green check, which tells them nothing the two sessions did not share. Same underlying evidence. Only one of them survived contact with a real reviewer's day.

The trail also has to outlast the session. Most review happens after the candidate has closed the tab, with nobody there to narrate a choice or explain why a diff got rejected. A proof that only makes sense while the work is live is a memory that fades, not evidence. A reviewer picking up a session cold should be able to reconstruct its shape on their own: what the goal was, where the candidate pushed back, which check caught which failure, what the final decision rested on. That is also what lets a result travel. A recruiter, a hiring manager, and a skeptical engineer reading three weeks apart should reach the same read of the same session, because the artifact carries its own explanation. And there is a second reader the whole industry tends to design away, which is the candidate. A report legible only to the people making the decision is a verdict, not an assessment. If a candidate can open their own result and see why a clean submission still scored soft on verification, the scoring stays honest, because a reading you can explain to the person it judged is one you have actually reasoned through.

Why not just switch the tools off

The clean-looking answer to all of this is to ban the tools and grade the code the old way. It is tempting because it deletes the hard measurement problem in one move. It also measures a workflow the job no longer uses. A no-AI assessment tells a hiring team how a candidate performs in an environment they will never work in again, and sends the careful engineer and the reckless one home with the same score, because with the tools removed there is nothing left to tell them apart.

There are real reasons to run an AI-off baseline, and we keep one in the presets for exactly those cases. It should be a deliberate choice for a specific reason, though, not the default hiding place for a product that found the messy version too hard to score. Banning the tools does not protect rigor. It hides who has it. If you want to know whether someone is a strong engineer now, you do not take the tool away and watch them solve a puzzle by hand. You hand them the tool and watch how they use it.

What this means for hiring teams and candidates

For a hiring team, the practical shift is to stop treating the passing test as the answer and start treating it as the least interesting line on the page. The questions that separate candidates are the ones the old format could not ask. Did they direct the tool or defer to it. Did they read the diff. Did they verify, and did they catch and correct the thing the tool got wrong. A report built around those questions gives you something a number never could: a result you can argue with. A verdict closes the conversation. Evidence opens one, and the argument is where the real hiring judgment happens.

For a candidate, the change is freeing once you see it. You are no longer being tricked into proving you can work in an environment nobody works in anymore. You are being asked to show the judgment you already use every day, out in the open, where it counts in your favor. The way to do well is not to hide the tool or to lean on it blindly. It is to direct it clearly, read what it gives you, check the thing that has to be true, and fix what it got wrong. That was always the skill that separated good engineers from careless ones. The model changed. The discipline did not. What changed is that we can finally see it.