There is a way to think about a research paper’s bibliography that doesn’t get discussed much: it is an implicit encoding of the idea. The set of papers you cite tells the reader what intellectual territory you are operating in, what prior work you are responding to, and implicitly, what gap between those papers your contribution tries to fill. A skilled reader can often triangulate a paper’s central claim just from its reference list.
A new preprint asks whether frontier models can do the same.
Reconstruction is a blind benchmark by researchers at Stanford and collaborating institutions designed around exactly this task. Given only the anonymized bibliography of a pre-publication paper — references stripped of author names and journal details, identified only by random IDs — can a model recover what the paper argues? The benchmark enforces strict anti-leakage: temporal citation cutoffs mean no cited paper postdates the target, anonymous reference IDs prevent string-matching the bibliography to known paper lists, and each paper’s citation set is frozen at submission. The task is “idea recovery,” not retrieval.
The results across 643 papers in six scientific domains are not flattering. Individually, seven frontier models achieve Match rates between 3 and 15 percent. Some of that variance is domain-dependent — ML papers, where the bibliography tends to cluster tightly around specific technique families, are somewhat easier; natural science domains with broader interdisciplinary reference sets are harder — but the range is uniformly poor. Frontier models, including the best current reasoning variants, largely fail to infer the thesis from the cites alone.
The multi-agent result is more interesting than the solo baseline. The paper includes a reference pipeline: cross-model review (different models critique each other’s hypotheses) followed by a Swiss tournament that progressively eliminates worse hypotheses across alignment-scored rounds. This pipeline raises Match rates to 23–42% across all six domains — approximately 2.4× over the best single-model solo run. That’s a real gain, but it also means the pipeline is still missing at least 58% of ideas even with structured debate.
The intuition behind why this is hard is worth sitting with. A bibliography doesn’t just list influences; it implies a specific response to them. Paper A cites papers B, C, D, and E. That tells you something about the intellectual neighborhood. But which gap between B–E the authors identified, and which synthesis they chose, is not uniquely determined by the citation list. There may be dozens of valid hypotheses that would plausibly cite the same papers. The benchmark is asking models to identify not just the neighborhood but the specific door the authors chose to open — and that’s a much narrower inference problem.
One reading of the benchmark is that it captures something like research creativity: the ability to identify what is worth doing given what already exists. On that reading, the low scores are a direct measure of how far AI is from the kind of gap-sensing that drives original research. That reading is probably too strong — the benchmark measures one proxy for a multidimensional thing — but it’s a more coherent proxy than “can you solve this benchmark problem,” which is what most capability evals measure.
The multi-agent improvement is suggestive even if modest. Cross-model critique does appear to help: different models identify different plausible hypotheses, and the tournament filter selects better ones than any individual model would. Whether that structure generalizes to actual research ideation — rather than recovering a known-existing idea — is a harder question the paper doesn’t claim to answer.
What the Reconstruction results do clearly establish is that the bibliography-to-idea mapping is non-trivial for current systems. That’s a usable signal. For labs building research assistance tools, it identifies a specific capability gap. For people thinking about how AI handles scientific knowledge, it suggests that retrieval and recombination are not sufficient — there is something else in the inference from citations to claim, and models don’t reliably have it yet.