Agentic AI news, research notes, and industry analysis.
Tag
3 stories in this tag.
Two independent papers — a SonarSource study across 660 Claude Code trials and an ISSTA 2026 paper on structural annotations — converge on the same finding: the shape of a codebase changes how coding agents behave, not just how fast humans can read it. Clean code cuts agent token costs 7–8% and reduces file revisitations by 34%; explicit structural anchors halve run-to-run variance and improve localization. The environment is part of the model.
Read more →Snorkel AI, Princeton, and UW-Madison released Senior SWE-Bench, a coding agent benchmark that replaces precise issue specs with realistic, under-specified requirements and grades solutions on code quality as well as test correctness. Models that clear 88% on SWE-Bench Verified drop to around 24% here. The gap between those numbers is worth examining carefully.
Read more →ProgramBench, from the SWE-bench team at Meta, Stanford, and Harvard, asks agents to reconstruct real programs from only a binary and documentation — no source code, no internet. No model fully solves any task. The best performer clears 95% of behavioral tests on just 3% of tasks. The benchmark exposes a specific gap: AI agents can generate plausible code but cannot yet architect software at the structural level of real-world programs.
Read more →