Main results

How the methods compare

All — methods from the paper's results table, on the same — held-out screens and the same full-genome candidate pool. Sort any column, filter by family, or search for a method.

Family

Ablations

Reading the columns

Scaling

Performance scales with data, but not model size. AssayFormer models (without input screen descriptions) are trained to identify scaling of in-context learning capabilities. Top: Models are trained on varying amounts of data from 1 to all training screens, and average performance curves are plotted. Both SFT and RL show this performance increase, with RL boosting SFT performance. Top left plot: EF at k acquisition steps shows that scaling is most effective during the first few steps of the experiment. At 100 steps (10,000 genes sampled), models approach similar performance. This is expected because even a random model will find all hits if it samples the entire genome. Top right plot: Budget to acquire enough hits to satisfy a recall threshold m. For example, to find m = 50% of the hits, the ‘1349’ model requires sampling less than 30% of the full genome. Note that RL rollouts stop at 10 steps during training, so these curves reflect out-of-distribution gains for RL performance. Bottom: We also consider scaling model size from XS (0.28M parameters) to XL (10.97M parameters), given the full training set. Here, we do not observe meaningful scaling laws. We posit that larger pretraining datasets are required to show evidence of model scaling. Note that scaling models has shown useful performance gains in pretrained LLMs on the non-active learning setting of AssayBench (Figure 4).

Who actually uses the feedback?

Run the same ten-round loop twice: once with the previous rounds' hit labels in the prompt, and once with them stripped. The gap is a direct measurement of how much of a method's performance comes from adapting to the experiment rather than from its prior. Every LLM family gains from the feedback, but their in-context learning ability is worse than AssayFormer, which motivates the AssayLoop handoff.

Does it generalize to phenotypes it never saw?

Yes. We remove every training screen belonging to the phenotype category being evaluated and then score the resulting leave-one-phenotype-out models across the full 20-screen test set. AssayFormer falls from EF 4.83 to 4.59, a smaller drop than Screen-kNN (3.40 to 2.98), and the LOPO AssayFormer remains above Screen-kNN trained on all phenotypes.

Generalization to unseen phenotypes. Enrichment factor on the full test set for Screen-kNN and AssayFormer, trained either on all phenotype categories or with the evaluated category excluded. This replaces the earlier per-phenotype breakdown.

A dash in an EP column is a refusal to score, not a zero: a method whose picks are mostly unannotated cannot supply enough genes to calculate the metric. More on what those columns measure →