Neural Networks, Secretly Symbolic

A new paper from McCoy, Soulos, Linzen, and Smolensky shows that neural network representations implicitly realize symbolic structures — precisely enough to replace the network's entire representation process with a closed-form equation, and to intervene on LLM behavior by directly editing the identified structures.

Read more →

The Case for Learning Mechanics

Fourteen researchers across Berkeley, MIT, Harvard, and EPFL published a 41-page manifesto arguing that a scientific theory of deep learning is not just desirable but already forming. They call it "learning mechanics" and point to five converging research threads — solvable models, tractable limits, empirical laws, hyperparameter theories, and universal behaviors — that together look something like what statistical mechanics looked like before it became statistical mechanics.

Read more →