If it's the problem I think it is, the solution is to run two concurrent prompts.

First prompt validates the input. Second prompt starts the actual content generation.

Combine both streams with SSE on the front end and don't render the content stream result until the validation stream returns "OK". In the SSE, encode the chunks of each stream with a stream ID. You can also handle it on the server side by cancelling execution once the first stream ends.

Generally, the experience is good because the validation prompt is shorter and faster to last (and only) token.

The SSE stream ends up like this:

    data: ing|tomatoes
    
    data: ing|basil
    
    data: ste|3. Chop the

I have a writeup (and repo) of the general technique of multi-streaming: https://chrlschn.dev/blog/2024/05/need-for-speed-llms-beyond... (animated gif at the bottom).