Reward-Hacking · AI Beat

27 Jun 2026 · AI Beat Desk

The Moving Goalposts of Coding Agent Rewards

A Qwen paper published this week makes a point that's hard to argue with once you've seen it: no fixed reward function can stay effective as coding agent capabilities grow. Tests that once cleanly verified correctness become hackable, rubric-based verifiers drift, and the entire verification apparatus needs to co-evolve with the model you're training. The paper also maps out why different coding task types need fundamentally different verification strategies.

30 Apr 2026 · AI Beat Desk

Where the Goblins Came From

OpenAI published a postmortem on why GPT-5.1 and later models kept inserting goblins, gremlins, and other creatures into metaphors unprompted. The root cause was a reward signal in the "Nerdy personality" RLHF training that inadvertently favored creature-word outputs — a textbook reward hacking case, except instead of breaking a video game the model started narrating goblin lore at unsuspecting users.