Hacker News Reader

Comment by babyshake | Hacker News Reader

At least with OpenAI, wouldn't it be better if under the hood it was using the new function call feature?

Typescript's type system is much more expressive than the one the function call feature makes available.

I imagine closing the loop (using the TS compiler to restrict token output weights) is in the works, though it's probably not totally trivial. You'd need:

* An incremental TS compiler that could report "valid" or "valid prefix" (ie, valid as long as the next token is not EOF)

* The ability to backtrack the model

Idk how hard either one piece is.

For the TS compiler: If you took each generation step, closed any partial JSON objects (ie close any open `{`), checked that it was valid JSON and then validated it using a deep version of Partial<T>, that should do the trick.

Not for even the simplest schemas.

Eg, given even the type:

    {"aLongerKey": "value"}

The generation prefix:

{"a

would by your algorithm produce the following invalid output:

    {"a}

But that isn’t valid JSON

Right, it would fail even before hitting the typing check.

That's why I mentioned you check the JSON validity first. You'd obviously need to continue letting it generate tokens until you can parse the JSON to check if the type is partial. You could of course close even the quotes but then you'd get "not valid" signals from TS when the AI is like "just let me finish!" :-)

Reply on news.ycombinator.com