Report in coding format

How to View Each Key Indicator

After a candidate submits a coding-style exam, the submitted code is automatically and quantitatively evaluated.
In the red frame of the image, from left to right: "Score," "Percent Correct," "Performance," and "Readability.

Score

• An indicator of  "total score".
• It is the average result weighted by 0.7 for percentage correct, 0.2 for performance, and 0.1 for readability.

Correctness Rate

• An indicator of "code correctness".
• This is a measure of how well the submitted code operates as expected out of several pre-prepared test cases, expressed as a percentage.
• The test cases range from normal input to input of abnormal values that do not exceed the constraints indicated in the question, to confirm that the candidate's code is operating correctly.
• The test cases can also be executed in the candidate's examination environment to check if the answers are correct. However, there may be test cases that are used only for scoring.

Performance

• An indicator of "code execution speed.
• This compares the execution speed of the candidate's code (solid light blue line) with that of the optimal solution (dashed green line) for the large size input provided in advance. The percentage of test cases whose execution speed is within 3σ of the optimal solution's code is calculated. σ is the standard deviation of multiple runs of the optimal solution in the same environmentalization.
• For some questions, a foolish solution and a suboptimal solution are provided, and in such cases, a comparison with each solution can be made graphically.
• If the percentage of correct solutions is zero, the performance value is zero. This is because it is not essential to score performance points for solving incorrect code in a performant manner.

Readability

• An indicator of the "readability of the code.
• The cyclomatic complexity of the candidate's submitted code is calculated and compared to the minimum and maximum of the multiple exemplars x 1.5, with the result expressed as 0-100%.
• The more complex the candidate's code, the higher the cyclomatic complexity and the lower the score.

Relative Evaluation

This is a comparison of the candidate's relative position to other candidates who solved the same problem. For more information on relative evaluation, please see this page.

Solution Details

The details of the candidate's answer include the time taken to submit the solution, the number of clues used, and the predicted computational complexity. The following items are currently displayed in the details.

• Selected language: The language selected at the time of submission.
• Elapsed time: The time the candidate had the question open
• Number of executions: Number of times the code was executed for testing before submission.
• Hints used: Number of hints used before submission
• Predicted computation: The amount of computation predicted in the system
• Performance: Average execution speed calculated during the performance calculation
• Memory: Average memory usage as calculated during performance calculations

Behavior Detection

• Leaves: The number of times a candidate leaves the answer screen by opening another tab or operating another application while solving a question.
• Paste Count: This is the number of times the code has been pasted in the editor.
• Paste Detection: The pasted area is highlighted in red.
• IP Address Detection: Shows the IP address of the respondent.

Submission Code

Allows you to see the final deliverables. The "Description" will only be displayed if it is entered by the candidate.

• Submission Code
  
  • This is the code when the candidate presses "Submit".
  • If the submission has been submitted more than once, this will be the code of the selected submission.
• Description
  
  • This is a natural sentence that the candidate can enter during the examination.
  • It is possible to enter the details of the approach to the problem and the process leading up to the deliverable.