Streaming and model output

Start with a supported protocol

Kastra has output adapters for OpenAI Chat Completions and Anthropic Messages, including supported streaming paths. A stream is not a byte-for-byte passthrough: policy may require buffering, filtering, or withholding generated tool calls.

Keep tool execution behind authorization

Do not execute a tool from partial arguments. Assemble the provider event according to the protocol and honor the completed policy outcome. A denied or held call must not be dispatched as ordinary output. Text already delivered to a client cannot be recalled by a later decision.

Inspect more than the HTTP status

A response may already have committed an upstream success status when post-inference policy acts. Read available X-Policy-Decision, X-Policy-Pre-*, and X-Policy-Post-* headers, plus the stream events and structured errors. Post headers can be absent when the stage did not run or headers were already committed.

OpenAI-style stream failures use an error event followed by the completion terminator; Anthropic uses its error-event shape. Treat an interrupted or rejected stream as an incomplete response, not as an implicit ALLOW.

HOLD timing

Waiting holds have bounded surface wait times and policy expiry. Keepalive events preserve a connection; they do not guarantee that a client, reverse proxy, or vendor will wait indefinitely. Async pre-request HOLD returns a checkpoint promptly; post-inference tool-call HOLD has a separate output/replay contract. Proxy approvals.

Verify your client

Test normal text, generated tools, a pre-request denial, post-output denial, HOLD, client disconnect, and upstream failure. Confirm that partial or held tool calls never reach your dispatcher. Client SDK compatibility must be established for the actual protocol path and error handling you use.