What is the AI Interviewer Format?
On this page
Overview
The AI Interviewer format is a question format in which an AI interviewer runs the test through a conversation with the candidate. The themes currently available are as follows.
- Knowledge: The interviewer presents the items one at a time and, for some items, asks follow-up questions about the reasons or mechanisms behind the answer.
Finding the questions
In the question list, select "AI Interviewer" in the "Question Format" filter to show only AI Interviewer questions.
Adding the questions to a test
When creating a test, you can select these questions just like questions in other formats.
- AI Interviewer questions have no AI model tier or AI usage limit settings.
- They do not count toward the limit on AI Collaboration questions (2 questions) or the total AI usage limit.
Report
You can check the results of the AI Interviewer format in the SEI-based report. For a test with only AI Collaboration and AI Interviewer questions, this report opens first. For a test that includes other formats, the legacy report opens, and you can switch from "View the new report".
- Requirement Judgments: For how to read them for each theme, see the section on that theme.
- Playback: Shows the exchanges with the interviewer in the order they took place.
Knowledge
A multiple-choice item can be answered correctly by guessing. In a Knowledge question, in addition to whether the choice was correct, you can see from the answers to the follow-ups how far the candidate understands.
How the test proceeds
The candidate talks with the interviewer on a chat-only exam screen. There is no code editor or AI agent.
- Start: When the candidate opens the question, the interviewer's greeting and the first item are already shown.
- Multiple-choice items: Each item is shown as a card. The candidate selects an option and presses "Answer". Once pressed, the answer is final and cannot be changed.
- Follow-ups: For items with follow-ups, the interviewer asks questions after the answer. The candidate answers in writing. When an answer is vague, the interviewer asks one step further based on the candidate's answer.
- Free-text items: For items without options, the candidate answers in writing.
- End: After the last item, the interviewer closes the interview and the input field closes.
The items proceed in order, and the candidate cannot go back to a previous item. The interviewer does not tell the candidate whether an answer is correct and gives no hints. Follow-ups do not depend on whether the chosen answer was correct: for the same item, every candidate is asked about the same points.
When the exam screen is in English, the interviewer conducts the interview in English.
For the candidate's guide, see Working with the AI Interviewer Format.
Questions
| Question | Content | Items | Suggested time limit |
|---|---|---|---|
| [Demo] IT Fundamentals | Basic knowledge of web development and programming | 11 (8 multiple-choice, 3 free-text) | 30 minutes |
| AI Fundamentals | How generative AI works, its limits, and how to use it safely at work | 10 (multiple-choice) | 30 minutes |
On the question details, you can check the items, the options and correct answers, the explanations, and the points the follow-ups ask about.
Evaluation
Requirement fulfillment rate
Each item and each follow-up point is judged as one evaluation item. The requirement fulfillment rate is the percentage of items passed. An item with follow-ups counts for more, by the number of its follow-up points.
- Multiple-choice items: The answer the candidate finalized is compared with the correct answer.
- Free-text items: Judged by whether the candidate's answer to the item meets the evaluation criteria.
- Follow-up points: Judged by whether the candidate showed the content of the point in their own words during the exchanges for that item.
A candidate who only guessed the options falls behind on the follow-ups.
SEI
SEI evaluates Specialized Knowledge and Communication Skill.
- Specialized Knowledge: Evaluates which knowledge the candidate has and how deeply they understand it, based on whether the items were answered correctly and on the explanations in the follow-ups. Whether a multiple-choice item was answered correctly is used only as evidence of knowing the fact; the levels above that are judged from the candidate's own explanations.
- Communication Skill: Evaluates, from the conversation with the interviewer, how clearly the candidate answers the follow-ups and similar aspects.
AI Collaboration Skill, Problem-solving Skill, Computer Science Fundamentals, and System Design Skill are not evaluated. For details on SEI, see What is SEI (Software Engineering Index)?.
Requirement Judgments
"Requirement Judgments" in the report shows, for each item, whether the answer was correct and how many of its follow-up points were passed. Expanding an item shows the chosen answer and the correct answer, the explanation, the result and reason for each follow-up point, and the exchanges used as evidence.