October 2026 | Upgrading the Test Formats and Retiring the Practical, System Design, Coding, and Quiz Formats
On this page
With the spread of generative AI, anyone can now produce working code and design documents in a short time, which makes it harder to judge a candidate's own ability from the submitted deliverables alone. In development teams, too, working together with AI is becoming the norm.
HireRoo is therefore upgrading its test formats for this new era, so that you can see not only the deliverables but also the candidate's own understanding, judgment, and how they collaborate with AI. With this upgrade, the existing Practical, System Design, Coding, and Quiz formats will be retired in stages between the end of December 2026 and the end of June 2027. Please switch the questions you use, review your evaluation criteria, and save any past data you need.
For internal review and sharing, you can also use our overview deck on the format upgrade (Japanese). If you have questions about this page, please contact your HireRoo representative or use the chat in the lower right corner.
What the upgrade brings
| Existing format | Upgraded format |
|---|---|
| Practical Format | AI Collaboration format questions |
| System Design Format | AI Collaboration format system design questions |
| Coding Format | AI interviewer format |
| Quiz Format | AI interviewer format |
The AI interviewer format is expected to be available in early November 2026. We will let you know the exact release date and how your current questions will be handled once they are confirmed.
How the test experience changes
| Format | Taking the test in the new format | What you can still review |
|---|---|---|
| Practical Format | Instead of selecting a GitHub PR, candidates work in a development environment in the browser and submit their deliverables at the end of the test | Submitted code, test case results, and the interview guide |
| System Design Format | Instead of the dedicated diagram tool, candidates create design documents and diagrams in a VS Code-style editor. You can also review their interactions with the AI agent | Diagrams created by the candidate and test case results |
| Coding Format | Candidates write the code themselves, the AI interviewer gives hints, and after implementation it asks about their approach and complexity. The work at the end of the test is submitted automatically | Submitted code, results for each test case, and a playback of the work process. Candidates can also run their code during the test |
| Quiz Format | The AI interviewer asks one question at a time, and candidates answer by choosing an option or in their own words. It follows up on answers as needed | Whether each question was answered correctly, and the number of correct answers |
In the Coding format, the previous ChatGPT assist and the suspiciousness score display are removed, and understanding is confirmed through a dialogue with the AI interviewer. If you want to evaluate how well candidates use AI, algorithm questions in the AI Collaboration format, where candidates write code together with an AI agent, remain available.
How evaluation and reports change
| Item | Existing formats | New formats |
|---|---|---|
| Approach to evaluation | Results are evaluated by automatic scoring for each format (test case pass rate, number of correct answers, and so on) | Deliverables are evaluated by the requirement fulfillment rate (test case pass rate), and the judgments and actions observed during the work are evaluated by SEI |
| Score and rank | Calculated based on Requirement Fulfillment | An evaluation of the work process using SEI, calculated per test rather than per question |
| Evaluation perspectives | Scoring items for each format | The five SEI skills: AI Collaboration Skill, Problem-solving Skill, Communication Skill, Computer Science Fundamentals, and System Design Skill |
| Report | The previous report | The new SEI-based report (released October 30, 2026). You can see which evaluation items were judged and on what observations |
SEI is the name of the Skill Index that HireRoo provides as its standard. If your company has created and published its own Skill Index, evaluation is based on that Skill Index.
Please do not use scores from the existing formats directly as new passing lines; instead, confirm the level your company needs, for example through internal tests. Results from the existing formats are not automatically re-evaluated in the new formats.
Retirement of the existing formats
With this upgrade, the existing formats will be retired on the following schedule.
| Existing format | Retirement |
|---|---|
| Practical Format | End of December 2026 |
| System Design Format | End of March 2027 |
| Coding Format | End of June 2027 |
| Quiz Format | End of June 2027 |
Formats you are using remain available as before until their retirement dates. From the end of November 2026, a format that has had no test started in the last 6 months can no longer be selected when creating a new test.
After retirement, tests in that format can no longer be issued or taken. Tests that include an existing format are not automatically replaced with new questions.
How to switch
- Check tests in use or in preparation: In addition to tests that have been taken, please check whether any tests that have not yet been taken include a format that is being retired.
- Choose the new questions: For the System Design and Coding formats, please check the corresponding questions we provide. For the Quiz format, please replace it with an AI interviewer format test.
- Review new test settings and evaluation criteria: Try the new formats internally and confirm the questions and evaluation criteria you will use for screening.
- Save the past results you need: Before each format is retired, decide which results and data you want to keep.
The Practical format will be retired at the end of December 2026, so please review it first. Let us know the skills you want to measure and the frameworks you need. If the existing questions are not enough, we will discuss which questions you would like to move to the new format. Not every requested question will be moved automatically.
The Swift and Kotlin mobile questions will not be moved to the new formats and will be retired together with the Practical format. Depending on the skills you want to measure, please consult us about alternatives such as system design questions on mobile architecture.
Viewing and saving past results
After each format is retired, you will no longer be able to view the code, diagrams, test case details, playback, and other details of results in that format. Please save any detailed data you need before each format is retired.
After each format is retired, only scores will remain available. The old report screen itself will also be retired at the end of June 2027. The deadline for viewing the details of each format differs from the retirement of the old report screen.
For the scope and steps for saving data, please consult your HireRoo representative.
About your plan
If your current plan does not include the new formats, we will contact you individually with the details and price of the plan change. Please check the price for your current contract period, the new plan, the change date, and the price after renewal.
For eligible customers, we will propose keeping the current price for the current contract period and moving to the price we informed you of in advance if you renew. For this mid-term change, we will ask you to confirm the details by email or Slack, and no signature on an application form is required. Whether to renew, and the renewal procedure, will be handled at your usual contract renewal.
FAQ
Do we need to do anything if we have not used the formats being retired recently?
Please check whether you plan to use them in the future, whether you have tests in preparation that use those formats, and whether there are past results you want to keep. If none of these apply, please let us know.
What should we do if we cannot find the same question in the new formats?
Please tell us the name of the current question and the skills you want to measure with it. We will discuss Practical format questions, mobile questions, company-authored questions, and some System Design format questions individually. If there is no corresponding question, we will create a new question in the new format as needed.
Can we save all past detailed results by exporting a CSV?
Exporting a CSV may not save all detailed data. Please let us know the scope of the data you want to keep.