GOLLuM: Using LLMs as Uncertainty-Calibrated Optimizers for Scientific Discovery

A language-model framework trained with Bayesian objectives ranked first across 23 experimental-design tasks and matched conventional Bayesian optimization using over 40% fewer experiments, according to a Nature Machine Intelligence study. The method, called GOLLuM, reframes LLM overconfidence as a training signal rather than a defect and reports a near-doubled yield in Buchwald–Hartwig reaction discovery compared to expert quantum-chemical descriptors and prior state-of-the-art LLMs.

Architecture and training signal

The paper targets the central trade-off in experimental optimization: Gaussian-process surrogates provide calibrated uncertainty but require hand-crafted descriptors that rarely transfer across domains; LLMs carry rich scientific priors but lack calibrated uncertainty, which is disqualifying in sample-expensive settings. GOLLuM closes the gap by training the model directly on experimental outcomes under a Bayesian objective, reshaping the embedding geometry so that experiments with similar outcomes cluster. The reported consequence is that overconfidence becomes a precise learning signal rather than a failure mode — and that the same backbone, without domain-specific featurization, transfers across reaction optimization, materials science, and molecular design.

Reported empirical gains

Three measurable claims carry the empirical case. Starting from only ten low-performing experiments, the model generalizes across 23 heterogeneous tasks in organic synthesis, materials science, process chemistry, and molecular design and ranks first on average among all competing methods. It matches traditional Bayesian optimization with more than 40% fewer experiments. On Buchwald–Hartwig amination, the framework reports a 43% rate of high-performing reaction discovery versus 24–25% for both expert quantum-chemical descriptors and state-of-the-art LLMs. The domains are heterogeneous, which makes the descriptor-free transfer the load-bearing claim and the one most worth reproducing.

What to verify before adoption

Three checkpoints determine whether the reported gains hold outside the original setup. Compute the same top-performing discovery metric on a held-out chemistry benchmark not included in the 23-task suite; sample-efficiency gains on familiar tasks can mask transfer cost. Inspect the Bayesian training objective directly — the transfer claim depends on whether the model's predictive distribution is genuinely calibrated or whether ranking quality is being driven by a softer proxy. Re-run the ten-experiment warm-start protocol on a non-chemistry task to stress-test cold-start generalization, since ten points is an aggressive budget for any GP-style surrogate. If those three reproduce, the architectural pattern — specializing a foundation model through a richer training signal rather than more data — becomes a template worth applying beyond experimental design.

Adjacent on arXiv this week: a Fisher-information-guided framework reports parameter-efficient continual adaptation without explicit task boundaries, claiming competitive accuracy at lower trainable-parameter counts than advanced task-aware continual-learning baselines. Worth tracking as a second data point on whether information-theoretic objectives are displacing larger parameter budgets as the preferred specialization lever.