Hacker News Reader

Seen a lot of things trying to do this by pressure testing the outputs, but all feel like anti-patterns. This is the first that seems like the "right" way to do it. Better to manage how the model is generating vs creating one more potentially faulty "glue" layer.

View on HN

Mathematically it requires less information to impose a certain prior on data in the process of generation than it does to read the data, do error detection and correction according to a prior, and then return it, if I understand correctly.

Something always felt incredibly icky to me about any kind of ad-hoc 'fixer' scripts that were part of a pipeline that was fully controlled by a user.

Can you elaborate about what you mean by pressure testing? Haven't heard this term yet.

Maybe not the right term... Just that a lot of other libs act like guardrails, i.e. let the model generate what it does (in full form text / GPT output), and try to then parse out what you want, error if output doesn't conform to standard format. As opposed to basically only allowing the model to generate into the already-built JSON form fields. Understandable why this guardrails/parsing approach is so popular though... can't do what this library is doing with OpenAI API. Need to be able to manipulate the token generation; otherwise you're forced to take full text output and try to parse it.

Reply on news.ycombinator.com