Revision Prompting improves industrial LLM processes

Industrial prompting typically runs a fixed instruction over many inputs, such as extracting data from invoices or translating documentation. When an input record is edited, the naïve approach is to re-execute the full instruction on the new input, which repeats the entire computation. This not only repeats the token cost of the original run but also produces a new output that may differ in unchanged sections because the model is nondeterministic. The result is both expensive and potentially inconsistent with earlier outputs.

Revision Prompting changes the workflow by constructing a prompt that includes the original instruction, the original input-output pair, and a diff describing how the input changed. The model is then asked to produce an "output patch" that updates the old output to reflect the new input. The patch is applied to the original output, yielding the updated result without regenerating unchanged content. This approach shifts most of the token work from the output side to the input side of the prompt.

The practical impact is twofold: consistency and cost savings. Because the model only modifies the parts explicitly mentioned in the diff, all other text remains exactly the same, preserving continuity across revisions. Since the patch is typically much shorter than a full output, the number of output tokens, and therefore the compute time, drops dramatically, while the extra input tokens are cheaper to process. When revisions happen shortly after the original run, caching can further reduce input token costs.

Implementing Revision Prompting requires a suitable diff format. Simple line-based diffs (e.g., POSIX diff) work for plain text, while structured data benefits from formats like JSON Patch. Engineers must adapt their instruction to accept a diff and request a patch, and they need a reliable method to apply the returned patch to the original output. The technique shines when input changes are small relative to the whole document, such as a single field update in a translation or a minor amendment in a data extraction task.

The method's effectiveness diminishes when the input change is large, because the patch can approach the size of a full output, eroding the token savings. Additionally, the approach assumes the model can correctly interpret the diff and generate an accurate patch; errors in the patch could propagate downstream. Therefore, teams should monitor patch quality and fall back to full re-generation for substantial revisions.