Hacker News Reader

Comment by qwertox | Hacker News Reader

Let's see which company becomes the first to sell "coding appliances": hardware with a model good enough for normal coding.

If Mistral is so permissive they could be the first ones, provided that hardware is then fast/cheap/efficient enough to create a small box that can be placed in an office.

Maybe in 5 years.

View on HN

My Macbook Pro with an M4 Pro chip can handle a number of these models (I think it has 16GB of VRAM) with reasonable performance, my bottleneck continuously is the token caps. I assume someone with a much more powerful Mac Studio could run way more than I can, considering they get access to about 96GB of VRAM out of the system RAM iirc.

I bought a framework desktop hoping to do this.

And it can do it, right? I think AMD AI Max line the first realistic offering for this type of thing.

The Apple offerings are interesting but the lack of x86, Linux, and general compatibility make it hard sell imo.

Haven't started relying on it for anything but it does have decent performance at like 130b parameters. I also haven't tried to tune it at all. I saw some recommendations somewhere but I installed talos on it and didn't want to figure any of that out.

I just got my framework mainboard today. I haven't had a chance to set it up yet but from the research I've been doing it seems like Minimax M2 might be the best coding model for it at the moment. Similar performance to Devstral2 with only 10b active params.

my bet is a deepseek box

llm in a box connected via usb is the dream.

...so it won't ever happen, it'll require wifi and will only be accessible via the cloud, and you'll have to pay a subscription fee to access the hardware you bought. obviously.

Reply on news.ycombinator.com