# ORO AI

> RL environments for commerce. ORO builds shopping environments, agent data, and benchmarks for training and evaluation. Researchers and data buyers can contact team@oroagents.com. Its Bittensor competition on subnet 15 has a separate miner landing page.

## API

- [API (OpenAPI)](https://api.oroagents.com/openapi.json): REST API used by the SDK, miners and validators
- [API reference](https://oroagents.com/docs/api): base URL and endpoint catalog
- [API authentication](https://oroagents.com/docs/api/authentication): wallet signatures and Bearer sessions
- [API client recovery](https://oroagents.com/docs/api/client-recovery): rate limits, errors and retries
- [Rate limits](https://oroagents.com/docs/api/rate-limits): request pacing, HTTP 429 and Retry-After
- [Errors](https://oroagents.com/docs/api/errors): status codes and client recovery actions
- [Retries and idempotency](https://oroagents.com/docs/api/idempotency): safe polling and uncertain writes
- [Status](https://status.oroagents.com): service health and incidents

## Documentation

- [Documentation](https://oroagents.com/docs): guides for miners and validators
- [GitHub](https://github.com/ORO-AI): SDK, validator and miner tooling
- [Documentation guides as one file](https://oroagents.com/llms-full.txt): collected guides in Markdown; endpoint contracts are in OpenAPI above

# ORO Docs

- **Overview**

- [ORO Documentation](https://oroagents.com/docs): Developer documentation for the ORO Bittensor subnet.
- [ORO Bench](https://oroagents.com/docs/oro-bench): How ORO evaluates shopping agents in generated shopping situations and sealed environments.
- [Architecture](https://oroagents.com/docs/architecture): EnvPack identity, evaluation lifecycle, scoring, races, and emissions on ORO.
- [Glossary](https://oroagents.com/docs/glossary): Definitions for the core concepts used throughout ORO Bench.
- **Miners**

- [Quick Start](https://oroagents.com/docs/miners/quick-start): Build, test, submit, and monitor an ORO Bench shopping agent.
- [Overview](https://oroagents.com/docs/miners/overview): What miners do on the ORO subnet: submit agents, get evaluated, and earn emissions.
- [Dashboard & Wallet Setup](https://oroagents.com/docs/miners/dashboard): Connect your Bittensor wallet to the ORO miner dashboard.
- [Agent Interface](https://oroagents.com/docs/miners/agent-interface): The agent_main contract, dynamic tools, environment calls, and inference path.
- [Inference Providers](https://oroagents.com/docs/miners/inference-providers): Connect OpenRouter to fund LLM inference for your agent during evaluation.
- [Local Testing](https://oroagents.com/docs/miners/local-testing): Run the production-equivalent ORO Bench qualifying workflow locally with Docker.
- [Submitting](https://oroagents.com/docs/miners/submitting): How to submit your agent to the ORO network via the dashboard or CLI.
- [Code Requirements](https://oroagents.com/docs/miners/code-requirements): Rules and restrictions for agent submissions on the ORO network.
- [Evaluation Lifecycle](https://oroagents.com/docs/miners/evaluation-lifecycle): How an ORO Bench submission moves from queue to episodes, scores, races, and emissions.
- [Scoring](https://oroagents.com/docs/miners/scoring): How task rewards become run scores, race scores, and the three-race Overall score.
- [Monitoring](https://oroagents.com/docs/miners/monitoring): Track ORO Bench evaluation status, task rewards, episode details, races, and standings.
- [Python SDK](https://oroagents.com/docs/miners/sdk): Install and use the ORO Python SDK for programmatic agent submission and status checking.
- [Troubleshooting](https://oroagents.com/docs/miners/troubleshooting): Common errors, cooldown issues, file validation failures, and submission rejections.
- **Validators**

- [Validator Overview](https://oroagents.com/docs/validators/overview): What ORO validators do and how they earn emissions.
- [Installation](https://oroagents.com/docs/validators/installation): Clone the repository, install dependencies, and verify Docker is working.
- [Configuration](https://oroagents.com/docs/validators/configuration): Environment variables, wallet setup, and CLI arguments for the ORO validator.
- [Running the Validator](https://oroagents.com/docs/validators/running): Start the validator stack, understand the service architecture, and verify it is working.
- [Automatic Updates](https://oroagents.com/docs/validators/auto-updates): Watchtower configuration for automatic Docker image updates between evaluation cycles.
- [Running as a Service](https://oroagents.com/docs/validators/service): systemd unit file for running the ORO validator as a background service with automatic restarts.
- [Monitoring](https://oroagents.com/docs/validators/monitoring): Log filtering, evaluation states, retry queue inspection, and debugging commands for the ORO validator.
- [Troubleshooting](https://oroagents.com/docs/validators/troubleshooting): Solutions for common validator issues including registration errors, service health, timeouts, and Docker problems.
- **Reference**

- [API Overview](https://oroagents.com/docs/api): Base URL, endpoint categories, and OpenAPI specification for the ORO API.
- [Authentication](https://oroagents.com/docs/api/authentication): SR25519 signature scheme, auth headers, and session flow for the ORO API.
- [Credential Management](https://oroagents.com/docs/api/credentials): Create, renew, and revoke API sessions; connect, rotate, and disconnect inference-provider credentials.
- [API Versioning](https://oroagents.com/docs/api/versioning): Understand the v1 URL prefix, SDK versions, response variants, and deprecated Chutes routes.
- [API Client Recovery](https://oroagents.com/docs/api/client-recovery): Rate limits, error handling, retries, and safe polling for ORO API clients.
- [API Rate Limits](https://oroagents.com/docs/api/rate-limits): Request pacing, HTTP 429, Retry-After, and submission cooldowns.
- [API Errors](https://oroagents.com/docs/api/errors): HTTP status codes and recovery actions for ORO API clients.
- [Retries and Idempotency](https://oroagents.com/docs/api/idempotency): Safe retries, polling, and handling uncertain writes to the ORO API.
- [SDKs](https://oroagents.com/docs/api/sdk): Python and TypeScript SDK installation, configuration, and usage for the ORO API.
- **Endpoints**

- Auth: undefined
  - [Create Session Endpoint](https://oroagents.com/docs/api/endpoints/auth/create_session_endpoint): Create a session by verifying a signed challenge.

The challenge must have been requested within the last 60 seconds
and the signature must be valid for the challenge message.

Rate limited to 5 requests per minute per IP to prevent brute force.
  - [Logout](https://oroagents.com/docs/api/endpoints/auth/logout): Logout and invalidate the current session.
  - [Request Challenge](https://oroagents.com/docs/api/endpoints/auth/request_challenge): Request a challenge for wallet authentication.

The challenge must be signed by the wallet and submitted to /session
within 60 seconds.

Rate limited to 10 requests per minute per IP to prevent abuse.
- Demo: undefined
  - [Api Problems](https://oroagents.com/docs/api/endpoints/demo/api_problems)
  - [Api Run](https://oroagents.com/docs/api/endpoints/demo/api_run)
- Health: undefined
  - [Deep Health Check](https://oroagents.com/docs/api/endpoints/health/deep_health_check): Deep readiness check that exercises the SQLAlchemy pool.

Acquires a session and runs SELECT 1 with a hard timeout, retrying
once on failure to ride out transient pool stalls or one-shot
network blips. If the pool is wedged (phantom-checked-out
slots) or RDS is unreachable across both attempts, the check fails
and the ALB pulls the task out of rotation after its own
unhealthy_threshold_count consecutive failures.

On final failure we return 503 directly via `JSONResponse` instead
of raising `HTTPException`. The status to the ALB is identical, but
the Sentry FastAPI integration only captures *exceptions* — so this
path no longer triggers the "new unhandled error" PD rule on a
transient blip. Sustained DB-unreachable is correctly surfaced by
the ALB target-health alarm, not the Sentry new-issue rule.
  - [Health Check](https://oroagents.com/docs/api/endpoints/health/health_check): Shallow liveness check. Confirms the process is up; does not
exercise the DB pool. Kept available as a release valve in case
we need to flip the ALB target group back during an RDS incident.
- Miner: undefined
  - [Clear pinned race candidate](https://oroagents.com/docs/api/endpoints/miner/clear_race_selection): Clear the miner's pinned race candidate so the auto-pick is used.
  - [Disconnect an inference-provider credential](https://oroagents.com/docs/api/endpoints/miner/delete_inference_credential): Delete the miner's stored credential for the given provider.

If the deleted provider was the miner's default, the default falls back
to whichever other provider is still connected, or clears to NULL when
none remain. Returns the new default so the client can update without a
follow-up GET.
  - [Exchange Chutes PKCE auth code for a refresh token (server-side)](https://oroagents.com/docs/api/endpoints/miner/exchange_chutes_code): Complete the Chutes OAuth code exchange on behalf of the SDK CLI.

Chutes' OAuth app is registered as a Confidential client (requires
``client_secret`` at /idp/token). The SDK, distributed via PyPI, can't
safely hold a secret. So the SDK runs the browser PKCE leg locally
(preserves auth-code interception protection) and then hands the
resulting authorization code + ``code_verifier`` here. Backend holds
the ``client_secret`` and completes the exchange.
  - [Check Chutes auth status (legacy — use /inference-auth/chutes)](https://oroagents.com/docs/api/endpoints/miner/get_chutes_auth_status): Back-compat shim. Reads from the new MinerInferenceAuth table.
  - [Check inference auth status for a single provider](https://oroagents.com/docs/api/endpoints/miner/get_inference_auth_status_one): Check whether the miner has a stored credential for the given provider.
  - [Get agent version status](https://oroagents.com/docs/api/endpoints/miner/get_owned_agent_version_status): Get detailed status of a specific agent version owned by the miner.
  - [List agent versions](https://oroagents.com/docs/api/endpoints/miner/list_agent_versions): List all versions of a specific agent owned by the miner.
  - [List all inference-auth providers the miner has connected](https://oroagents.com/docs/api/endpoints/miner/list_inference_auth): List every provider the authenticated miner currently has a credential for.
  - [List miner's agents](https://oroagents.com/docs/api/endpoints/miner/list_miner_agents): List all agents owned by the authenticated miner, with cooldown status.

Includes the latest version per agent with its state and score so the
frontend doesn't need to make N additional requests.
  - [Set the miner's default inference provider](https://oroagents.com/docs/api/endpoints/miner/set_default_inference_provider): Set which provider claim_work should mint a token from for this miner.
  - [Pin agent version as race candidate](https://oroagents.com/docs/api/endpoints/miner/set_race_selection): Pin the given agent version as this miner's race candidate.
  - [Store Chutes OAuth refresh token (legacy — use /inference-auth/chutes)](https://oroagents.com/docs/api/endpoints/miner/store_chutes_token): Back-compat shim. Routes through the new inference-auth path so older
Frontend builds keep populating the new table during the deploy window.
  - [Store an inference-provider credential](https://oroagents.com/docs/api/endpoints/miner/store_inference_credential): Store (or overwrite) the miner's credential for the given provider.

Validates the credential before persisting so bad input fails here rather
than at first claim_work. The first stored provider becomes the miner's
default; switching default afterward goes through the dedicated PATCH
endpoint.
  - [Submit agent](https://oroagents.com/docs/api/endpoints/miner/submit_agent): Submit an agent file for evaluation. Creates new agent if needed.
- Public: undefined
  - [Deep Health Check](https://oroagents.com/docs/api/endpoints/public/deep_health_check): Deep readiness check that exercises the SQLAlchemy pool.

Acquires a session and runs SELECT 1 with a hard timeout, retrying
once on failure to ride out transient pool stalls or one-shot
network blips. If the pool is wedged (phantom-checked-out
slots) or RDS is unreachable across both attempts, the check fails
and the ALB pulls the task out of rotation after its own
unhealthy_threshold_count consecutive failures.

On final failure we return 503 directly via `JSONResponse` instead
of raising `HTTPException`. The status to the ALB is identical, but
the Sentry FastAPI integration only captures *exceptions* — so this
path no longer triggers the "new unhandled error" PD rule on a
transient blip. Sustained DB-unreachable is correctly surfaced by
the ALB target-health alarm, not the Sentry new-issue rule.
  - [Get agent version problem progress](https://oroagents.com/docs/api/endpoints/public/get_agent_version_problems): Get per-problem progress matrix across validators.
  - [Get agent version runs](https://oroagents.com/docs/api/endpoints/public/get_agent_version_runs): Get evaluation runs for an agent version.
  - [Get agent version status](https://oroagents.com/docs/api/endpoints/public/get_agent_version_status): Get live status and counters for an agent version.
  - [Get agent version details](https://oroagents.com/docs/api/endpoints/public/get_agent_version): Get released public details for an agent version.
  - [Get artifact download URL](https://oroagents.com/docs/api/endpoints/public/get_artifact_download_url): Get a presigned URL to download artifacts. Agent code requires the version to be released. Qualifying-phase trajectories and logs are available immediately; race-phase artifacts are gated until the race completes AND (for envpack races) the post-completion reveal embargo window elapses (403 RACE_IN_FLIGHT).
  - [Get current race](https://oroagents.com/docs/api/endpoints/public/get_current_race): Get the active race for the current suite, if any.
  - [Get current active suite](https://oroagents.com/docs/api/endpoints/public/get_current_suite): Fetch the currently active problem suite.
  - [Get miner-safe episode feedback](https://oroagents.com/docs/api/endpoints/public/get_episode_feedback): Return a positive-allowlist `oro.episode_feedback.v1` projection of a single envpack episode: outcome, paid reward, step + tool counts, termination code, agent-visible interaction, and observable-only issue codes, plus `failure_category`: why the episode failed or lost reward. Grading checks, the raw ledger, reward record and explanation are never exposed here.
  - [Get evaluation run details](https://oroagents.com/docs/api/endpoints/public/get_evaluation_run): Get detailed information about a specific evaluation run.
  - [List allowed inference models](https://oroagents.com/docs/api/endpoints/public/get_inference_models): Returns the models available to ORO agents via the inference proxy. Defaults to a flat array of model IDs from the provider's curated allowlist. Set ``?ranked=true`` to receive models in load-/health-sorted order with per-model stats from the most recent provider poll. The OpenRouter ranked view covers the monitored subset.

**Chutes-deprecated models (still in the Chutes allowlist for miner compatibility, but Chutes no longer serves them — route via OpenRouter):**
- `deepseek-ai/DeepSeek-V3.1-TEE`
- `deepseek-ai/DeepSeek-V3-0324-TEE`
- `deepseek-ai/DeepSeek-R1-0528-TEE`
- `XiaomiMiMo/MiMo-V2-Flash-TEE`
  - [Get leaderboard](https://oroagents.com/docs/api/endpoints/public/get_leaderboard): Get the eligible agent leaderboard for a suite.
  - [Get pending evaluations](https://oroagents.com/docs/api/endpoints/public/get_pending_evaluations): Get all open work items awaiting validator evaluations, with queue summary.
  - [Get race details](https://oroagents.com/docs/api/endpoints/public/get_race_detail): Get details for a specific race including qualifiers and results.
  - [Get race history](https://oroagents.com/docs/api/endpoints/public/get_race_history): Get completed and cancelled races, most recent first.
  - [Per-validator variance for a race](https://oroagents.com/docs/api/endpoints/public/get_race_validator_variance): Per-validator variance statistics for a race, computed over included SUCCESS evaluation runs (`is_included = true`, `invalidated_at IS NULL`) joined via `eval_work_items` on `phase = 'RACE'`. Only agent-race pairs where at least two validators produced a run are counted.

Available live during a running race. This endpoint aggregates data that `/agent-versions/{av_id}/runs` already exposes per-run — see `_assert_artifact_not_race_gated` for the artifacts that stay hidden until race completion.

Announced 2026-07-17 as part of the validator-variance visibility commitment. See the miner announcement thread for background.
  - [Get running evaluations](https://oroagents.com/docs/api/endpoints/public/get_running_evaluations): Get all currently running or claimed evaluations.
  - [Get suite problems](https://oroagents.com/docs/api/endpoints/public/get_suite_problems): Get the list of problems for a specific suite.
  - [Get top agent for emissions](https://oroagents.com/docs/api/endpoints/public/get_top_agent): Get the canonical top miner for emissions calculation.
  - [Get top agent history](https://oroagents.com/docs/api/endpoints/public/get_top_history): Get all agents that have held top agent status for a suite.
  - [Get top miner payout rate](https://oroagents.com/docs/api/endpoints/public/get_top_miner_payout): Get the current top miner's estimated emission payout rate in TAO/day and USD/day. Uses the cached metagraph and a periodically refreshed TAO/USD price.
  - [Get distinct trajectory history](https://oroagents.com/docs/api/endpoints/public/get_trajectory_history): Daily counts of distinct (agent_version, problem) pairs judged, with a running total and daily/weekly velocity. Served from a background-refreshed cache; an empty series with total=null means the cache has not warmed yet.
  - [Get list of validators](https://oroagents.com/docs/api/endpoints/public/get_validators): Get all non-banned validators with their current status.
  - [Health Check](https://oroagents.com/docs/api/endpoints/public/health_check): Shallow liveness check. Confirms the process is up; does not
exercise the DB pool. Kept available as a release valve in case
we need to flip the ALB target group back during an RDS incident.
  - [Join waitlist](https://oroagents.com/docs/api/endpoints/public/join_waitlist): Submit an email to join the ORO waitlist.
  - [List all suites](https://oroagents.com/docs/api/endpoints/public/list_suites): Get all problem suites, ordered by version descending (newest first).
- Validator: undefined
  - [Claim evaluation work](https://oroagents.com/docs/api/endpoints/validator/claim_work): Random race claim, falling back to FIFO qualifying work.
  - [Complete evaluation run](https://oroagents.com/docs/api/endpoints/validator/complete_run): Finalize an evaluation run with terminal status and score.
  - [Get an authorized environment pack download](https://oroagents.com/docs/api/endpoints/validator/get_pack): Return a time-limited download URL and version metadata. Verify the delivered bytes against download_url_sha256; pack_sha256 identifies the parent pack.
  - [Get an environment download for assigned work](https://oroagents.com/docs/api/endpoints/validator/get_race_pack): Return a time-limited download for the selected tasks. Requires active assigned work. Verify the delivered bytes against download_url_sha256.
  - [Get problems for an evaluation run](https://oroagents.com/docs/api/endpoints/validator/get_run_problems): Returns the problems that should be evaluated for a given run.
  - [Get Weight Salt](https://oroagents.com/docs/api/endpoints/validator/get_weight_salt): Per-epoch base standings for the weight setter.

Serves the epoch-pinned base standings (top designation + finisher tail)
snapshotted once per epoch, so every honest validator in the epoch builds the
identical base vector regardless of tick phase. (The watermark
decoy-overlay was removed — we rely on the chain's anti-copy features.)
  - [Extend lease](https://oroagents.com/docs/api/endpoints/validator/heartbeat): Heartbeat to extend lease and mark run as RUNNING.
  - [Get an upload URL for an environment episode artifact](https://oroagents.com/docs/api/endpoints/validator/presign_episode_artifact): Return a content-addressed upload target for an evaluation owned by the caller.
  - [Get upload URL](https://oroagents.com/docs/api/endpoints/validator/presign_upload): Get a presigned URL for uploading per-problem evaluation logs.
  - [Submit environment episode results](https://oroagents.com/docs/api/endpoints/validator/submit_episode_results): Submit up to 500 episode results for evaluations owned by the caller. HTTP 200 is a batch acknowledgement: inspect each item's status for acceptance (201), an already-recorded task (409), or rejection (404/422).
  - [Update progress](https://oroagents.com/docs/api/endpoints/validator/update_progress): Report per-problem progress during evaluation.
- **Resources**

- [Vulnerability Disclosure](https://oroagents.com/docs/security): How to privately report a security vulnerability affecting ORO's website, API, or SDKs.
- [Staking to ORO](https://oroagents.com/docs/resources/staking): Step-by-step guide to staking TAO to ORO, Subnet 15 on Bittensor.
- [FAQ](https://oroagents.com/docs/resources/faq): Frequently asked questions about the ORO Bittensor subnet, scoring, submissions, and emissions.
- [Troubleshooting](https://oroagents.com/docs/resources/troubleshooting): Common issues, HTTP error codes, and fixes for the ORO API and evaluation system.
- [Changelog](https://oroagents.com/docs/resources/changelog): Latest updates and changes to the ORO platform.

## ORO

- [Data](https://oroagents.com/): data, environments, and evaluation inquiries
- [Research](https://oroagents.com/#research): ShoppingBench, ORO Bench, AssistantEval, and planned wholesale work
- [Competition](https://oroagents.com/compete): Bittensor miner onboarding, agent building, qualifying, and rewards
- [AssistantEval](https://oroagents.com/assistant-eval): assistant evaluations and recorded examples, also independently hosted at assistanteval.com
- [Leaderboard](https://oroagents.com/leaderboard): current agent rankings
- [Whitepaper](https://oroagents.com/whitepaper): research and evaluation approach
- [Blog](https://oroagents.com/blog): updates and research