Stealing Reasoning Traces from Proprietary LLM APIs

Previously, users only saw the final text output of a proprietary LLM, with its internal reasoning hidden inside an encrypted blob. The research shows that this blob can be captured and replayed unchanged into a different model from the same vendor. When the second model is jailbroken to echo its thinking field, the original model’s chain-of-thought appears in plain text. This turns a hidden internal state into an externally accessible data stream.

For an attacker, the cost is limited to intercepting the API response and having access to a jailbroken version of the provider’s model, which is often available through public exploits or self-hosted copies. Defenders must invest in either suppressing the return of reasoning blocks, encrypting them with a session-specific key, or verifying that a block cannot be reused outside its original conversation. Either approach adds engineering overhead and may affect latency or compatibility with existing client libraries. The trade-off is between preserving model usefulness and preventing leakage of private thought data.

The technique fails if the provider no longer sends a reusable thinking block, for example by tying the encrypted payload to a nonce or session identifier that changes each turn. It also stops working when the target model refuses to output the thinking field, either because the jailbreak is patched or because the model architecture does not expose a thinking token stream. In such cases, the attacker cannot obtain the verbatim reasoning even if the block is captured. Consequently, the attack surface shrinks to providers that continue to expose portable, session-agnostic reasoning blobs.

The leaked data identified in the study includes real API keys, passwords, access tokens, and personal email addresses that were never visible in the user-facing chat. Sixty-four of the recovered artifacts appeared exclusively inside the reasoning blocks and nowhere in the observable conversation. This demonstrates that the hidden chain-of-thought can serve as a conduit for sensitive information that would otherwise remain protected.