A Replay Should Show What the Candidate Actually Saw

A reviewer watching an assessment replay is reconstructing a working session they were not present for. Every place the replay differs from what the candidate actually saw is a place the reconstruction silently goes wrong. Two of those places bothered us enough to fix.

One timeline, not five

Our replays used to be organized by question. Pick a question from a dropdown, watch that question's timeline, pick the next one. Candidates do not work in separate compartments for each question. They open question five at minute five, bounce back to question two, and return later with a new idea. The dropdown model erased that structure and made the reviewer do the reassembly by hand.

The replay is now one continuous session timeline. Which question is on screen is derived from the recorded switching events, so scrubbing the one timeline walks the code and preview through every question in the order the candidate actually visited them. A colored band above the scrubber shows the dwell structure at a glance, and clicking a segment jumps into it.

One honesty rule made this work: for older sessions that never recorded switching events, we refuse to invent the band. The replay falls back to manual selection rather than displaying a reconstruction that looks authoritative and is fiction.

The frozen code problem

Vibecoding sessions have two surfaces: the editor, and an agents window where candidates orchestrate parallel AI conversations. Our code frames only ever captured the editor workspace. So a candidate who spent forty minutes directing agents replayed as someone staring at unchanged code for forty minutes. The pixels matched those of a candidate who walked away from the keyboard, even though the work was completely different.

The session recorder had been logging surface switches all along. The replay just never rendered them. It now reconstructs a band that separates editor time from agent time, using the same tiling logic as the question track, and shows the total time spent with the agents. While the playhead sits inside an agents dwell, the code pane says so explicitly: the frames below are what the agents were writing into, not what was on screen.

Evidence means resisting inference

Both fixes treat a replay as evidence, and evidence is allowed to say "I do not know" but never allowed to guess. A replay that admits what it does not know is sometimes less impressive to look at, but it is the only kind a hiring decision should rest on.

A later change keeps the recording clean even when a reviewer wants to look around. While the replay is paused, the reviewer can switch between the editor and the agents window or open files, the way the candidate could, without touching the recording. Pressing play or moving the scrubber throws that exploring away, so the next frame is again exactly what the candidate saw. A note beside the controls tells the reviewer that play returns to the candidate's view.

The replay also began drawing the candidate's pointer and clicks over the reconstructed screen. The pointer is shown to reviewers and never used in any automated score, which the candidate notice spells out.