Hacker News Reader

You would also need to keep generating until the whole string is valid. And what if it gets caught in a loop?

Not sure how this can really guarantee 100%

> And what if it gets caught in a loop? Not sure how this can really guarantee 100%

It's not great but after some timeout you can just set the mask to only include closing brackets.

You would still have to ensure balancing somehow. Both "]" and "}" are valid "closing brackets" and the correct one to choose is context-dependent.

You can determine which brackets you need in which order by parsing the incomplete json which was generated so far.

That won't do it, also need to close other stuf

{"this": "is valid json so farrrrrrrrrrrrrr

But yeah the general idea makes sense. Once you hit a timeout, change the mask to things that will close existing open things in a valid manner (}, ), ], ")

Same problem with normal sampling - if it doesn't pick the <end> token, you're stuck generating until you hit some stopping heuristic (max tokens, timeout, etc.)

Reply on news.ycombinator.com