July 2026 | New Ability to Configure Available AI Models and Usage Limits in AI-Collaborative Exams

Account

Overview

In AI-collaborative coding tests, you can now set the range of available AI models and the AI usage budget limit for each problem. This allows you to flexibly control the AI environment used by candidates based on the skills you wish to evaluate and the expected level of AI utilization.

Changes and Impact for Employers

1. Classification Settings for Available AI Models

  • Setting: For each problem, you can select the AI model classification available to the candidate as either "Standard" or "High Performance."
  • Default: The default for newly created problems is "Standard."
  • The models available in each classification are as follows (as of July 9, 2026. These are subject to future updates).
Classification Description Available Models
Standard A set of standard AI models used for general practical tasks, implementation, verification, and bug fixing. claude-sonnet-5, gpt-5.4, claude-haiku-4-5, gpt-5.4-mini
High Performance A set of AI models including high-performance models suitable for more advanced implementation, research, design, and complex bug fixing. claude-opus-4-8, gpt-5.5, claude-sonnet-5, gpt-5.4, claude-haiku-4-5, gpt-5.4-mini

2. Setting the AI Usage Budget Limit

  • Setting: For each problem, you can set a maximum budget for the AI usage available to the candidate.
  • Range: Can be set between 1 USD and 30 USD. The default for newly created problems is 20 USD.

3. Configuration Method and Scope

  • These settings can be configured from the Create/Edit screen of AI-collaborative problems.
  • Settings will be applied to newly issued tests.
  • Previously created coding tests will be treated as "High Performance" / AI usage limit of 30 USD.
  • These settings will not be applied to exams that have already been completed.
  • In Demo Exams, the available models are limited to claude-haiku-4-5 and gpt-5.4-mini, and the AI usage limit is fixed at 1 USD.

Changes for Candidates

1. Display of Settings

  • The available AI model classification and AI usage limit will be displayed under Exam Rules on the testing screen.
  • Exam rules can be checked at any time via the modal at the start of the exam or the "Exam Rules" tab during the exam.

2. Scope of Application

  • The settings apply to all AI tools used within the testing environment (GitHub Copilot Chat, Claude Code / Codex, etc.).

3. Behavior when AI Usage Limit is Reached

  • If a candidate reaches the AI usage budget limit, they will no longer be able to use AI for the remainder of the session.