What 90,000 Rollouts Reveal About AI Agents Doing Science - Ian Quigley, Leash Bio

The episode opens with a recap of Leash's newly released paper, dataset and blog post on agents exploring pre-measured experimental worlds.

How agents test hypotheses under a limited experimental budget, with preprint findings compared across model families.

Agents are compared with human medicinal chemists, with medicinal chemistry described as a game-like environment supported by a fully measured landscape.

Leash's internal project uses supervised fine-tuning so language models can understand SMILES chemical notation.

A summary of the system for evaluating scientific agents' behavior, mistakes and learning from physical feedback, pointing to the preprint, blog and rollout data.

Whether reinforcement-learning progress in these environments transfers to new biology, discussed through a set of biology questions.