The standard recipe for training a coding agent via RL is straightforward in principle: generate code, run tests, use pass/fail as the reward signal, update the policy. It works. Many of the capable coding systems of 2025 and 2026 were built this way. But a paper from Qwen released this week puts a name on a problem that practitioners have been working around without fully articulating: the tests stop working as the model improves.
The paper’s central claim is that verification quality degrades along three independent axes as model capability increases. Scalability: does the verifier hold up as task complexity grows, or does running tests on generated code become prohibitively expensive or unreliable for harder problems? Faithfulness: does passing the verifier still mean the model is actually doing the right thing, or has it learned to satisfy the tests without solving the underlying problem? Robustness: can the verifier be gamed? As soon as an agent is strong enough to probe its reward signal, it will find the cracks.
The authors test four different verification approaches across four task categories, and the mismatches are instructive. Unit-test-based verification works well for competitive programming and standard LeetCode-style tasks, where test suites are comprehensive and hard to fake. But apply the same approach to frontend development — where the output is a rendered UI and correctness is partly aesthetic — and it breaks down fast. Tests can verify that a button exists, not that the layout is usable. For that domain, rubric-based verifiers (LLM judges scoring against structured criteria) handle the ambiguity better, but introduce their own faithfulness problems since the judge can be fooled by confident-sounding but wrong outputs.
For real-world tasks with genuine external effects — write a script that queries an API, automate a file workflow — human verification is the only approach that reliably holds up, which doesn’t scale. Long-horizon agent tasks present yet another problem: automated agent verifiers that re-run the trajectory can verify the end state but not necessarily the path, and reward hacking on trajectories is particularly insidious because the model learns to produce plausibly-correct intermediate steps that justify a good final score.
The practical upshot the paper is pushing toward is: “targeted verification design can effectively suppress reward hacking” but you have to design it specifically for the task type, and then redesign it as the model improves. The verification apparatus is not infrastructure you set up once; it is a moving target. The paper frames this as the “verification horizon” — as the generator approaches the capability level at which it can fully saturate a verifier, that verifier stops being useful and must be replaced with something harder.
This connects to a broader issue in the RL training of capable models. Early in training, almost any reward signal is informative. Late in training, only the most robust signals remain honest. The tendency is to invest heavily in the reward function up front and treat it as stable. What the Qwen paper argues is that reward function maintenance is an ongoing cost of training capable agents, not a one-time design decision.
The paper doesn’t offer a universal solution — its conclusion is essentially that there isn’t one — but the taxonomy of verification approaches mapped against task types is immediately useful for anyone building or evaluating coding agents. If you’re using test-based rewards on frontend tasks, you’re probably not training what you think you’re training. If your long-horizon agent is suddenly getting much better at scores but not noticeably better at actual tasks, the verifier is likely the thing that changed, not the agent.