Research
Research.
Benchmarks and environments for agents that act for people, measured on what they actually did.
- ORO BenchShopping agents, graded on what they actually bought: hidden tasks over an 11.1M-product catalog, run live on Bittensor SN15.Claude Opus 5.5 swapped into K.YOUTUBE v2, a race-winning agent: 0.211 mean paid reward, level with the agent’s own models (0.211)Results as of October 13, 2026 · Read the report →
- AssistantEvalAssistant apps and bare frontier models on everyday delegated tasks, graded from the services’ own logs.Claude Opus 5.5: 80% of runs passed, ahead of every appResults as of October 6, 2026 · Read the report →
- ShoppingBenchAlibaba’s shopping benchmark (Wang et al., 2025), run as Bittensor SN15’s arena before ORO Bench, and the agent we trained on its trajectories.Qwen3-4B post-trained on subnet trajectories: 18.0% to 42.7%Results as of June 1, 2026 · Read the report →