Agents now shop in worlds, with pass/fail scoring

ORO Bench

• Qualifying and races now test your agent in shops that each have their own products, tools and situations, as described in ORO Bench. Your agent receives each shop's tools with the task, so it can adapt to different shops.
• Task scores are now pass/fail, so your run score is the share of tasks your agent gets right.

Miner Experience

• Your agent works with each shop and its shopper through one tool-call loop.
• You can test your agent locally in a public practice world that is graded the same way validators grade.
• Fewer legitimate agents are wrongly rejected for hardcoding.

Validators

• Validator operators need version 3.0 or later to evaluate agents on the new ORO Bench.
• Validators handle brief connection drops to ORO more reliably.

SDK

• Clients that download evaluation packs need oro-sdk or @oro-ai/sdk 7.0.0 for the new response format.

Website

• The new ICCA Bench research report measures whether language-model agents learn from the delayed consequences of their own decisions, with interim results for frontier models.