We have added "Editor Mode" to Playback for the AI Collaboration format. You can replay where and how the candidate changed code in the editor during the exam, at the same level of detail as Playback for the Coding Format.

Until now, Playback let you review the exchanges between the candidate and the AI agent, and the submitted deliverables. From these two alone, you could not tell "whether the candidate submitted the agent's output as is, or rewrote the key parts themselves." Editor Mode replays the editing itself that happens in between.

What's New

Switching between Agent Mode and Editor Mode

A switch is added to the top right of Playback. The existing screen for reading the exchanges is "Agent Mode," and the screen for replaying editor activity is "Editor Mode." The switch is shown only for exams in which an edit history was recorded.

Edits are color-coded by candidate and AI agent

During replay, the code is color-coded by who last wrote each line. Lines written by the AI agent are shown in light blue, lines written by the candidate are shown in light yellow, and edits whose author could not be determined are shown as "Unknown." You can find the sections the candidate wrote themselves directly on the screen.

Edited files and view counts

On the left, a list of files with recorded edits and a list of files the candidate opened are displayed. Each file shows how many times the candidate opened it, and files that were never opened are marked as such. You can check whether the candidate didn't even look at the code the agent wrote, or read it and then fixed it.

Commands run in the terminal

At the bottom of the replay screen, the commands the candidate ran in the integrated terminal are displayed, along with the directory they were run in and their exit codes. Commands that served as evidence for the evaluation are marked as such.

What Stays the Same

There are no changes to the evaluation criteria, scores, or how candidates take the exam. The candidate's screen and operations remain the same as before.