Hacker News Reader

Something about having a very fast, very small harness written in C only to drive LLMs amuses me. Like putting a big ass spoiler on a minivan or something.

View on HN

Well, you'd think so but given how slow and resource hungry the ones written in javascript are it makes a lot of sense.

Are they? I use pi and it doesn’t drain more than other software.

The LLM can run elsewhere, such as the cloud.

For the part that runs locally, efficiency might matter.

Reply on news.ycombinator.com