RL Post-Training Lives in the Middle
A new paper finds that reinforcement learning gains in transformers concentrate almost entirely in a narrow band of middle layers. Training just one layer at roughly 40–60% network depth can match or exceed full-parameter RL fine-tuning. The finding challenges the assumption that all layers participate equally in post-training, and has practical implications for compute-efficient alignment.
Read more →