How to Assess Developers Who Code With AI

The assessment most companies still run was built to answer one question: can this person produce correct code on their own, from a blank file, with nothing in the room but what is in their head? For a long time that was a fair proxy for whether someone could do the job. It has quietly stopped being one. Once a model can generate a plausible, test-passing solution to most interview-shaped problems on demand, a green check no longer tells you who wrote the code, whether the person understood it, or whether they would have caught it on the day it broke. This piece is about where the skill worth measuring has moved and how to capture it there.

The signal that quietly stopped working

The old model rewarded unaided production, and it worked because production was hard. Writing a correct solution from scratch took knowledge, care, and a working model of the problem in your head, so the output was a reasonable stand-in for all three. When anyone can prompt a plausible answer into existence, that link breaks, and the finished artifact stops carrying evidence about the person who submitted it. A grader that reads only the final files cannot tell a solution someone reasoned through from one they pasted, so it ends up scoring luck and calling it skill.

A passing test used to be scarce evidence. Now the answer is the cheap part of the transaction, and the expensive part, the judgment that produced it, is exactly what the old format throws away.

The question moved: direct, inspect, verify, correct

If producing code is no longer the scarce thing, then directing the production of it is. The question worth asking a candidate in 2026 is whether they can point a capable tool at a real problem, read what comes back, confirm it does what they meant, and steer it when it does not. Those four moves are the job now, on most days, for most working developers.

Each move is a place where skill either shows up or fails to. Direction is precision, asking for exactly what you mean so the tool has nothing left to guess, as the login page example in A Good Idea Shouldn't Have to Be in English shows. Inspection means reading a generated diff before you keep it, instead of trusting that green means good. Verification is the spine of the whole thing. A model will always generate faster than a person can read, so the skill that separates people is deciding what has to be true and then actually checking it. Correction is the recovery move: when the check fails, the candidate has to recognize a subtly wrong answer and pull it back on course. None of that is visible in the final file, only in how the work was done, which is the part a modern assessment has to be built to keep.

Put the tool where you can see it

The first instinct when you add AI to an assessment is to give it its own room, a separate agent lane with private files and a button to promote its work. That leaves the reviewer with a polished final file and no way to tell whether the candidate wrote it, directed it, or inherited it from a run they never read. We moved the tool onto the same files the candidate works on, so every change it proposes gets kept or rejected in the open. Shared Agent Workspaces Need Evidence, Not Ceremony explains why the separate lane fails, and why tool edits arrive as proposals so a shared file cannot be rewritten out from under the candidate.

Attribution: telling direction from blind acceptance

Once the tool works in the open, who wrote each line and what the candidate did with each diff become part of the score, as the shared workspaces post spells out.

Picture the same small take-home task, one with a known edge case around empty input, handed to two candidates. Both reach for an agent. Both submit code that passes the visible tests. Read only the final files and you would rank them identically.

The trail pulls them apart. The first candidate asks for a plan before any edit, reads the proposed diff, and rejects the first version because it mishandles the empty case. She reruns the check, watches it fail, tightens her request, and keeps the second version only once the behavior matches what she described. Her session has friction in it, and the friction is the signal: a question with intent, a rejected diff, a verification step that reacted to a real failure. The second candidate accepts every suggestion on the first pass. The tool happens to produce working code, so the tests go green and the empty case comes out correct by luck, but there is no rejected diff, no follow-up, no moment where the candidate caught something the tool missed. When a reviewer opens that trail, the absence is loud, a straight line from prompt to submit with no reading in between.

Neither candidate is disqualified for using the tool. What attribution buys is the ability to tell a directed session from a delegated one without interrogating anyone afterward, and without pretending the tool was not in the room. Accepting a good suggestion blindly and steering a tool toward a good suggestion look identical in the artifact and nothing alike in the record of how it was made.

Validation you can see, not a gate you cannot

The verify move only counts if a reviewer can see it. For project and interface work, what the candidate checked and how they reacted when a check failed is often the more revealing half of the artifact, so validation has to show up in the report as evidence instead of a hidden pass or fail in the pipeline. AI-Native Assessments Need Evidence, Not Theater covers what a useful proof has to say.

One answer stopped being enough

A single algorithm answer rarely reveals judgment, so we pushed assessments toward multi-step project work, where far more of it shows, as Measuring Project Work, Not Just Answers explains.

That richer work needs a report that can hold more than a number. Ours reads out a small set of named competencies, each tied to the evidence behind it.

The AI-native assessments post explains why a few named axes beat one overall score.

A trail a reviewer can actually read

Visibility usually fails through too much evidence, because a reviewer has only a few minutes per candidate.

So capturing the trail is only half the work. The other half is ranking it, surfacing the rejected diff and the reacted-to failure ahead of the many edits that changed nothing. Volume is a tempting proxy and a bad one, since a candidate who typed a lot did not necessarily think a lot, and a short session can hold the sharpest decision in the room.

A reviewer opens two rich trails. The first has been ranked, so she lands directly on the moment the candidate pushed back on the tool and the check that caught the failure, and reaches a read in about a minute. The second is a flat list of every action in order, so the reviewer scrubs for ten minutes, gives up, and falls back to the green check. Both hold the same kind of evidence, but only the ranked one survived contact with a real reviewer's day.

A ranked trail also has to make sense to a reviewer reading it cold weeks later, as the shared workspaces post explains. The candidate should be able to read it too, the second reader the AI-native assessments post argues for.

Why not just switch the tools off

Banning the tools and grading the code the old way deletes the hard measurement problem in one move, but it hides who is rigorous instead of protecting rigor, as The Problem Was Never Vibecoding argues in full. An AI-off baseline still has real uses, and we keep one in the presets for teams that choose it on purpose.

What this means for hiring teams and candidates

For a hiring team, the practical shift is to stop treating the passing test as the answer and start treating it as the least interesting line on the page. The questions that separate candidates are the ones the old format could not ask. Did they direct the tool or defer to it? Did they read the diff? Did they verify, and did they catch and correct the thing the tool got wrong? A report built around those questions gives you a result you can argue with, and the project work post explains why that beats a cleaner score nobody can inspect.

For a candidate, the change is freeing once you see it. You are no longer being tricked into proving you can work in an environment nobody works in anymore. You are being asked to make the four moves you already make every day, out in the open, where they count in your favor. That was always the skill that separated good engineers from careless ones. What changed is that we can finally see it.