Almost every team running agents in production has instrumentation watching them. Far fewer check whether the work was any good, which leaves a large and growing record that no one has graded.
An agent failure does not look like a failure. "A retrieval step can return the wrong document while every HTTP status reads 200," one survey of the tooling market put it. "An agent can loop through fourteen tool calls, burn thousands of tokens, and deliver a confidently wrong answer." (MarkTechPost, Aug 9) Every monitoring system an organization already owns reports that run as healthy.
By the numbers: Teams have responded by instrumenting. Across all 1,340 respondents to LangChain's survey, 89 percent had some form of observability and 62 percent had detailed tracing. Among the 57 percent already running agents in production those figures rise to 94 and 71.5 percent. Evaluation trails in both groups: 29.5 percent of all respondents run none, and 22.8 percent of the production teams. LangChain fielded the survey between November 18 and December 2, 2025 and published it on June 12, so it describes the ground these teams were standing on before this year's agent releases. (LangChain, Jun 12)
The big picture: Health systems show the same shape from a different angle. A UPMC and KLAS Research study of more than two dozen health system leaders, published on August 6, found more than 90 percent had deployed third-party AI and 92 percent evaluate it before deployment. Only 44 percent have a dedicated data platform for testing. (UPMC Enterprises, Aug 6) The rest check their tools through vendor protocols, limited pilots and informal approaches, which is evaluation of a kind, done without a place to run it.
Yes, but: One organization did publish a grade this week, and it read badly for the humans. Anthropic ran an evaluation across 1,053 paid testers in which a permission prompt was swapped mid-session for a clearly dangerous command. Only 13.6 percent refused it; the company reports its automatic mode would have blocked 89 percent, and it becomes the default for new sessions on Claude Code's Pro, Max and Team plans on August 14. Simon Willison accepted the finding and kept his reservation. "I absolutely buy that auto mode is a better solution than asking humans to constantly approve actions," he wrote. "But... I'd like to see more independent confirmation of this." (simonwillison.net, Aug 8) He then described a malicious-package attack he does not think any version of the feature could catch.
Between the lines: Grindr chose which of its own numbers to publish, which is a problem no observability tool addresses. Grindr chief executive George Arison told shareholders that AI raised engineering output 2.5 times between July 2025 and April 2026, work that would otherwise have taken roughly 200 more engineers and $60 million a year. On the earnings call he said the real multiple was 3.5, and that the company published the lower figure because the higher one sounded unreasonable. (Business Insider, Aug 7) The measure was lines of code shipped, which Goldman Sachs chief information officer Marco Argenti has called a poor proxy.