Hacker News Reader

I get decent JSON from it quite well with the "assistant: {" trick. I'm not sure how well trained it is to do JSON. The template on ollama has tools calls so I assume they made sure JSON works: https://ollama.com/library/mistral-small:24b/blobs/6db27cd4e...

View on HN

And for anyone looking to dig deeper, check out "grammar-based sampling."

What’s the “assistant: {" trick? You just end your prompt with that?

Mistral supports prefixes: https://docs.mistral.ai/guides/prefix/

That’s cool. However it only shows a few odd. I’d imagine the model needs to explicitly support this (be trained with it). None are about json… do you use that trick with json?

I use it to get JSON pretty often. See also: https://twitter.com/simonw/status/1885091289554968975

Reply on news.ycombinator.com