Inverse Rubric Optimization: A testbed for agent science

We propose inverse rubric optimization (IRO): tasks where an agent must learn the preferences of a black-box judge under a label budget. IRO tasks induce rich agent behavior and smooth scaling, making them a useful testbed for agent science.

Fulcrum
Google Open Sources Experimental Multi-Agent Orchestration Testbed Scion

Designed to manage concurrent agents running in containers across local and remote compute, Scion is an experimental orchestration testbed that enables developers to run groups of specialized agents w

InfoQ
Example of what an imagery domain ontology could look like #GIMI #codesprint27 #testbed