Overview

This article explains what to check when Skill Assessment setup or measurement does not progress. For the setup steps themselves, see Setting Up Measurement (Installing the hireroo CLI).

About the beta Skill Assessment is currently offered as a beta.

Try This First: Run Setup Again

It is safe to run hireroo spider setup as many times as you need. If something does not work, run it again first. It redoes the browser authorization and rewrites the AI tool settings.

hireroo spider setup

My Page Stays at "Install the CLI and run setup"

  • After running hireroo spider setup, check that you approved the authorization shown in the browser.
  • If the browser does not open automatically, open the URL shown in the terminal.
  • Check that the account you are logged in to HireRoo with is the same account that is being measured.

Setup Shows [failed]

A configuration file marked [failed] is left as it was. This happens, for example, when settings.json is not valid JSON. Fix the file using the message shown, and then run hireroo spider setup again.

AI tools marked [ok] are already configured. You can leave a [failed] entry as it is for an AI tool you do not use.

You Are Asked "Replace them? [Y/n]"

This appears if you have configured your own usage log (OpenTelemetry) destination. The current values are shown beside the values after the change.

  • If the change is fine, enter Y. The original values are saved and restored by hireroo spider uninstall.
  • If you answer anything other than yes, setup ends without changing anything.

When running setup where you cannot answer in a terminal, such as from a script, you can skip the question with hireroo spider setup --yes.

My Page Stays at "Verify the connection"

Run the Claude Code or Codex command shown on My Page to send a message.

AI tool Command
Claude Code claude 'hello world'
Codex codex 'hello world'

If it still does not progress, run setup again and then run the command again.

GitHub Copilot Usage Is Not Measured

  • Copilot CLI: Open a new terminal after setup (or run source ~/.zshrc). Terminals that were already open before setup are not affected. Windows is not yet supported.
  • GitHub Copilot in VS Code: Reload the window (Developer: Reload Window in the Command Palette).

What My Page Shows and What It Means

Display Meaning and what to do
Waiting to start measurement Setup is not finished, or the first data is pending. Follow the steps on My Page
Measuring Measurement has started
Measurement failed Measurement is not working. Run setup again, and if that does not help, contact your company administrator
Measurement ended An administrator has ended the measurement. AI tool usage is no longer recorded
Generating score Not enough data has been collected for a score yet. It appears automatically once enough data is collected
You don't have an AI monitoring assessment yet You are not being measured. If you have received an invitation email, start again from its link

If you have been assigned an internal test, take it from Skill Development › My Page, not from the Skill Assessment My Page. For details, see Taking an Internal Test.

Updating the hireroo CLI

The following command updates the hireroo CLI to the latest version. Running the install command again also updates it.

hireroo upgrade

Uninstalling

The following command restores the changed settings to their values before setup and removes the stored authentication information.

hireroo spider uninstall

If some configuration files are marked [failed], the authentication information is removed but those files have not been restored. Fix the files and run hireroo spider uninstall again.

If the Problem Persists

Contact your company administrator (the person in charge of Skill Assessment measurement).