AI Signal 121
ADK adds native live evaluation for voice agents using LLM-simulated users that speak audio turns
↗
Google's Agent Development Kit now supports evaluating live, voice-based agents by driving them with simulated users that generate spoken audio turns, scored within the same eval loop used for text agents.
Voice agents that perform in demos can silently break on prompt tweaks or model iterations, with tools failing to fire or context slipping between turns. Native live evaluation in ADK gives developers a repeatable way to test multi-turn spoken conversations before shipping, closing the gap between demo and production for graph-based agent workflows.
Written by elseif from the cluster below · every claim links back to a sourceThe three things worth knowing
ADK's live evaluation lets a simulated user speak turns as audio against a live voice agent, with scoring integrated into the existing eval loop.
Eval sets are JSON files supporting two test styles: conversation scenarios where a persona improvises turns, and fixed conversations.
The example workflow chains three single-purpose live agents on gemini-live-2.5-flash-native-audio, keeping the audio stream open and carrying session state across handoffs.
THE READ
What the cluster adds up to.
ADK previously could evaluate text-based agents but had no native way to test live voice agents that operate over open audio streams. The new capability lets developers run a simulated user that speaks its turns as audio against a live agent, score the spoken replies, and keep everything inside the same evaluation loop already used for text agents. This means teams do not need a separate testing harness for voice versus text workflows.
The evaluation system uses JSON-based eval sets with two distinct test styles. Conversation scenarios describe a goal and persona, and the user simulator improvises its turns accordingly. Fixed conversations provide predetermined exchanges. This split lets developers test both open-ended conversational behavior and specific scripted paths, which matters because real callers behave unpredictably while certain critical flows must always succeed.
The example workflow demonstrates a graph-based architecture where three single-purpose live agents run sequentially on gemini-live-2.5-flash-native-audio. The audio stream stays open across the entire interaction, and ADK carries accumulated session state and conversation history forward so each agent picks up in context rather than starting cold. This means the user never notices handoffs, but the developer can still isolate and evaluate each stage's behavior.
Built-in personas such as NOVICE control how the simulated user communicates, telling the simulator to share only high-level goals and wait for the agent to guide the conversation. This is a meaningful design choice because it lets developers test how their agents handle callers who provide minimal information, which is a common real-world failure mode for voice systems that assume cooperative, detailed users.
Because only one feed carried this story, there is no independent corroboration of the feature's scope or limitations. The material describes the capability and provides a worked example but does not discuss cost, latency overhead of running audio-based simulations, or where the evaluation stops being useful. Developers considering adoption should verify whether the simulated user's audio generation and scoring are sufficient for their specific conversation patterns before relying on it as a production gate.
Written by elseif from the cluster below · checked for specifics the sources never containedTHE CLUSTER