Hacker News Reader

You might be interested in https://prismml.com/news/bonsai-2-27b

This looked exciting until I read that it gets stuck in loops and generally wasn't a useful model

Please cite a reputable source for this claim, and keep in mind we are talking about Bonsai 2 and not the original Bonsai.

Even on the original one, the authors have replied to such issues with working solutions: https://github.com/PrismML-Eng/Bonsai-demo/issues/131

for 32gb, this is my model of reference now, you have to run it with a patched version of llama and is still not available in lmstudio or omlx, waiting for that to streamline the experience a bit. But so far, the best i had till now.

It looks like upstream work is underway: https://github.com/ggml-org/llama.cpp/issues/29058

The sheer number of geniuses working so fast on this repo never ceases to amaze me.

Reply on news.ycombinator.com