Show HN: Anything To JSON – a language model for structured extraction | Hacker News Reader

Show HN: Anything To JSON – a language model for structured extraction

anythingtojson.com

18 pointsedunteman8 comments

Hey HN! Erik here from banana.dev

We’ve trained a small(ish) language model on structured extraction, and today we’re launching a playground for it at https://anythingtojson.com. Give it a try!

This model continues our work on structured generation, following last week’s launch of Fructose[1], a python client for strongly-typed LLM responses.

There seem to be two distinct halves of the problem intended to be solved by Fructose and structured generation:

1. the reasoning ability of the model, such as performing chain of thought, creative acts, and natural language tasks. In a way, the “business logic”.

2. the structured json response, to make sure the receiving code doesn't break

AnythingToJson is intended to solve the latter. No big-brain work, just find data in text and extract it. Constrained generation at inference time helps keep it on track, and our finetuning has improved the accuracy and consistency of extracted data.

There’s much more progress to be made (longer context window, hallucination to squash out, etc), but ship early and fast.

[1] https://news.ycombinator.com/item?id=39619053

Open articleView on HN

Nice, makes sense to chain models so you don't waste the attention of the big smart model on the grunt work of JSON structure.

I bet there are some good post-processing heuristics you could also apply for hallucination with a flag for "this should be in the text verbatim" & then string matching whether the answer it outputted was a string from the text or not.

interesting idea! yeah next big challenge for us is hallucination in null fields (IE: if you ask for a "name" from text that doesn't have a name you usually get "John") so need to add more sampling heuristics to doublecheck against the input blob for a sense of if it's an accurate extraction

Awesome, agree on the nulls! Any plans to add support for image inputs?

we have no plans for image or other modalities. I'd like to keep it just text-to-text so it can be as sharp of a tool as possible

How does it differentiate from using json output with OpenAI?

I think it can be useful for some cases, but you’ll always have the limitation of the model size to compete against large closed source models.

openai json mode will ensure JSON, but not strictly the right JSON schema to match whatever object you're expecting to return. Keys could be missing, fields could be improperly typed. We found gpt-4 reasoning capabilities could, most of the time, make it work, but that's overkill for many

can't speak to the actual service, but can confirm they have excellent bananas (not kidding)

hey, a vouch is a vouch!

Reply on news.ycombinator.com