Streaming without lying.

Streaming is the cheapest way to make an AI feature feel fast. Tokens appear one after another, the screen looks alive, and perceived latency drops even when the real latency has not moved.

So teams reach for it early, and they are right to — right up until they wire an action to the stream. Here is the tension nobody puts in the demo: streaming and correctness pull in opposite directions. The user sees content as it arrives, but a validated, structured result cannot exist until the response is complete. If you render the stream as though it were final, you are showing the user — and sometimes acting on — data your app has not yet verified.

That is the lie in naive streaming: it presents partial, unvalidated output with the confidence of a finished answer. This piece is about keeping the speed without telling that lie. The trick is to separate what you show from what you trust, and to never let a half-formed response drive a consequence.

1. Two clocks in one response

A streamed response has two moments that teams love to collapse into one.

The first is first useful output — the instant something worth showing has arrived. This is what makes streaming feel fast, and it is fine to optimise for it. The second is completion and validation — the instant the whole response is present, parses, and conforms to your contract. This is the only moment the result becomes actionable.

Naive code treats the first moment as the second. Everything downstream — enabling a Save button, applying a change, committing structured data, firing a paid side effect — gets attached to the stream instead of to validation. Then the final tokens change the meaning, or the response fails to parse, or a safety filter truncates it, and the app has already acted on something that no longer exists.

2. Show the surface, gate the action

Split the response into what the user may see while it streams and what the app may do only after it validates.

Progressive display is fine for the human-readable surface. Render tokens as they arrive; let it feel live. But commit is gated. Any control that causes a consequence — apply, save, send, purchase, replace state — stays disabled until the complete response has parsed and validated against your contract.

That single rule kills a whole class of bugs. The user still perceives speed, because text is moving, while the app refuses to let anyone act on unverified data. When validation succeeds, you enable the action and, if you need to, reconcile the display: swap the optimistically streamed text for the validated version. Usually they match. When they do not, the validated version wins, and the user was never able to act on the difference.

3. Structure is what makes this non-negotiable

If the response is free-form prose that gets shown and forgotten, streaming is nearly risk-free. The danger scales with structure. A response that has to yield an object with an action, an amount and a list of targets cannot be trusted until the closing brace has arrived and the object has validated. Partial JSON is not “a little bit of an object.” It is a syntactically incomplete string that will either fail to parse or — with a lenient parser — parse into something subtly wrong, which is worse.

So never act on a partial parse. If you want to give feedback during a structured stream, drive it from a separate, explicitly-partial text field, not from the machine object. Parse defensively, only on completion, and treat a parse or validation failure at the end as a full failure of the response — not a “mostly worked.”

4. Cancellation is part of streaming, not an afterthought

A stream is a long-lived operation on a device that gets interrupted constantly. The user navigates away, backgrounds the app, edits the input, or starts a new request while the old stream is still flowing. Streaming without a currency rule just reintroduces the stale-response problem in a new place.

Bind each stream to the request lifecycle. Give every request an identifier. When a newer request starts, or the user cancels, mark the older stream stale. If a stale stream later completes, it must not be presented and must not commit — its tokens are discarded and the event is recorded as a stale-discard under the same request id. The interface only ever presents and commits the current request’s completed, validated result.

5. Failing mid-stream without leaving a mess

Streams fail halfway. The connection drops, the provider errors after emitting some tokens, a safety filter cuts the response short. If you have been rendering optimistically, you now have partial content on screen and nothing valid behind it. Handle it explicitly:

• Keep partial display content clearly provisional until commit. On failure, discard it and show an honest failure or fallback — not a frozen half-answer.
• Commit nothing. No structured data, no side effect, because the response never validated.
• Make recovery a real state, not an exception swallowed into a spinner that never ends.

And guard side effects with idempotency. A retry after a mid-stream failure must not double-apply a change or double-charge a paid operation. The stream is a delivery mechanism; the effect has to be tied to a completed, validated, idempotent commit.

6. Model it as a state machine

A pile of booleans (isStreaming, isDone, hasError) drifts out of sync the moment two of them can be true at once. Model the response as a state machine instead, where every transition is explicit and observable:

From	Event	To	Effect
streaming	all tokens received	validating	—
validating	parse + contract OK	committed	enable actions, reconcile display
validating	parse / contract fail	failed	discard, show fallback, commit nothing
streaming	newer request / cancel	discarded	present nothing, commit nothing
streaming	mid-stream error	failed	discard, recover

The two states worth watching in production are discarded — are users routinely outrunning your latency? — and failed after validating — are completed streams failing the contract, which would mean streaming is masking a structure problem you would otherwise have caught.

Given–When–Then

• Given a streamed structured response still in progress, when the user views it, then a progressive preview renders but every consequential action stays disabled until the full response validates.
• Given a stream that fails contract validation at completion, when a preview was already shown, then the app discards the preview, commits nothing, and shows a clear failure or fallback.
• Given the user starts a new request or cancels mid-stream, when the older stream completes later, then its result is neither presented nor committed, and the discard is recorded under the same request id.
• Given partial structured output arriving token by token, when the app parses, then it tolerates incompleteness, acts on nothing until a complete valid parse, and never drives a side effect from a partial object.

When this is not worth it

If responses are short, streaming buys little perceived speed and the machinery is not worth it — wait for completion and render once. If the output is pure display text with no structure and no downstream action, optimistic streaming is essentially free, so skip the commit-gating. The full discipline pays off precisely when the response is structured, drives an action or a paid side effect, or lives in a UI where users routinely cancel and re-ask.

There, the goal is easy to state and easy to get wrong: let the interface feel as fast as the first token, while the application trusts nothing until the last one has arrived and validated.