One of the most powerful ways to integrate LLMs with existing systems is constrained generation. Libraries such as outlines[1] and instructor[2] allow structural specification of the expected outputs as regex patterns, simple types, jsonschema or pydantic models. Llama.cpp supports bnf grammars.

These outputs often consume significantly fewer tokens than chat or text completions.

[1] https://github.com/outlines-dev/outlines

[2] https://github.com/jxnl/instructor

Because most of the online models are neutered so much they are almost useless. I get the morals lecture several times a day now.

A simple "yes / no" would probably be valuable even for being a single token.

That's as fast or faster than most people can type.