AI Signal 184
Simon Wilison's LLM cliché highlighter flags AI-generated prose patterns
↗
Illustration only Photo by Michael Dziedzic on Unsplash
Martin Fowler's September fragments highlight a new tool that detects common LLM writing patterns, along with research on AI detection, NVIDIA's long-horizon agent architecture, and the impact of agents on CI.
For engineers, the highlighter offers a quick way to spot AI-generated text when reviewing contributions or content. The post also argues that AI agents require moving verification before the push, a shift that aligns with Fowler's reminder that Continuous Integration is a practice, not just a server.
Written by elseif from the cluster below · every claim links back to a sourceThe three things worth knowing
Simon Wilison's LLM cliché highlighter flags patterns common in AI-generated prose when given text or a URL.
A 2025 study cited in the post shows humans distinguish LLM text from human text no better than random chance.
NVIDIA's AVO harness uses persistent memory and supervision to run long-horizon agent tasks like GPU kernel optimization over seven days.
THE READ
What the cluster adds up to.
The LLM cliché highlighter is a practical tool for engineers who review text, offering a quick way to flag patterns typical of AI-generated prose. It costs nothing to try, but it relies on pattern matching, so it may miss subtle AI text or flag human text that uses similar phrases. The tool stops working when an LLM is trained to avoid those clichés, making it a moving target rather than a permanent solution.
The post also cites a 2025 study showing that humans cannot distinguish LLM text from human text better than random chance. This suggests that tools like the highlighter are necessary, but they are not perfect. Engineers should be cautious about relying on any single detection method, especially when the stakes involve code review or content authenticity. The study's finding underscores the need for multiple signals and human judgment.
NVIDIA's AVO architecture for long-horizon agents is a significant development, using persistent memory and a supervisor to maintain progress over days. This allows agents to tackle complex tasks like kernel optimization, but it requires a harness and careful supervision. The cost is added complexity and the need for a supervisor model, which may not be feasible for all teams. It stops working when the task is too short or when the supervisor cannot effectively redirect the agent.
The CI discussion is directly relevant to engineers using AI agents. Paul Stack argues that agents break CI assumptions because they iterate quickly but still discover failures after pushing. Fowler agrees that verification should be automated before push, but reminds that this was always part of CI practice. The cost is changing pipeline design to include pre-push checks, and it stops working if agents are not disciplined or if the environment differs from the CI server.
Overall, the post highlights the evolving landscape of AI in software development. Engineers need to adapt their workflows to account for AI-generated code and text, and to design pipelines that handle agent-driven changes. The key is to understand the principles behind practices like CI, not just the tools, and to remain skeptical of both AI output and our own ability to detect it.
Written by elseif from the cluster below · checked for specifics the sources never containedTHE CLUSTER