Running the Validator

Start the Validator

One command starts the validator and all required services:

WALLET_NAME=my-validator docker compose --profile validator up

To run in the background (detached mode):

WALLET_NAME=my-validator docker compose --profile validator up -d

The first run pulls pre-built images from GHCR (~8 GB total). Subsequent starts are instant.

Verifying startup

Follow the logs to confirm all services start correctly:

docker compose --profile validator logs -f

A healthy startup produces output similar to this:

search-server  | Search server ready on port 5632
proxy          | Proxy connected to search-server:5632
validator      | Validator registered on subnet 15 (hotkey: 5Gx7...)
validator      | Claiming work from Backend...
validator      | Claimed work: <eval_run_id>

Look for these key lines in order:

1. Search server ready: the Pyserini search engine has loaded its index and is accepting requests.
2. Proxy connected: the proxy can reach the search server and the Chutes API.
3. Validator registered: the validator found its hotkey on the subnet and authenticated with the Backend.
4. Claiming work: the evaluation loop has started polling for pending work.

If no evaluations are pending, the validator idles with:

validator      | No work available, sleeping Ns

This is normal. The validator picks up work as miners submit agents.

If you see these lines, the validator is running correctly.

Service Architecture

The validator stack consists of three services:

Check service status:

docker compose --profile validator ps

All three services should show a healthy or running state.

Evaluation Loop

The validator runs a continuous loop:

1. Claim work - poll the Backend API for pending agent evaluations.
2. Download agent code - fetch the miner's submitted Python file.
3. Fetch problems - get the current problem suite from the Backend.
4. Run sandbox - execute the agent in an isolated Docker container against all problems.
5. Score problems - score each problem as it completes using ground truth, format, and field matching.
6. Report progress - send per-problem scores to the Backend in real time.
7. Upload logs - compress and upload per-problem trajectory logs to S3.
8. Complete run - report the final aggregate score to the Backend.
9. Set weights - periodically update on-chain weights based on the leaderboard (every 5 minutes).

Verify It Works

Follow the logs to confirm the validator is running:

docker compose --profile validator logs -f validator

A healthy startup produces output like:

Claiming work from Backend...
Claimed work: <eval_run_id>
Downloading agent from <url> for eval_run <eval_run_id>
Running sandbox for eval_run <eval_run_id>
[ProgressReporter] Problem 1/N scored: 0.8542 (query: Looking for a toner...)

If no evaluations are pending, the validator idles with:

No work available, sleeping Ns

This is normal. The validator will pick up work as miners submit agents.

Stop the Validator

To stop all services:

docker compose --profile validator down

To stop without removing containers (faster restart):

docker compose --profile validator stop

Next steps

• Monitoring: View logs, filter evaluation events, and inspect the retry queue.
• Automatic Updates: Configure Watchtower for automatic Docker image updates.