Hacker News Reader

Comment by Game_Ender | Hacker News Reader

To help those who got a bit confused (like me) this Groq the company making accelerators designed specifically for LLM's that they call LPUs (Language Process Units) [0]. So they want to sell you their custom machines that, while expensive, will be much more efficient at running LLMs for you. While there is also Grok [0] which is xAI's series of LLMs and competes with ChatGPT and other models like Claude and DeepSeek.

EDIT - Seems that Groq has stopped selling their chips and now will only partner to fund large build outs of their cloud [2].

0 - https://groq.com/the-groq-lpu-explained/

1 - https://grok.com/

2 - https://www.eetimes.com/groq-ceo-we-no-longer-sell-hardware

View on HN

I deeply crave prosumer hardware that can sit on my shelf and handle massive models, like 200-400B at a reasonable quant. Something like Groq or Digits but at the cost of a high-end gaming PC, like $3k. This has to be a massive market, considering that even ancient Pascal-series GPUs that were once $50 are going for $500.

> This has to be a massive market

It's not - it's absolutely a vanishingly small market.

At home people would rather use the cloud.

Nvidia's working on it. 200B at $3k

https://www.nvidia.com/en-us/products/workstations/dgx-spark...

> I deeply crave prosumer hardware that can sit on my shelf and handle massive models, like 200-400B at a reasonable quant.

So, an Apple Mac Studio?

I have that irresistible urge too, but I have to keep reminding myself that I could spend $2000 in credits over the course of a year, and get the performance and utility of a $40k server, with scalable capacity, and without any risk that that investment will be obsolete when Llama5 comes out.

The Framework Desktop is one not absurdly expensive option. The memory speed isn't great (200 something GB/s), but any faster with those requirements at least doubles the price (e.g. a Mac Studio, only the highest tier M chips have faster memory).

hi! i work @ groq and just made an account here to answer any questions for anyone who might be confused. groq has been around since 2016 and although we do offer hardware for enterprises in the form of dedicated instances, our goal is to make the models that we host easily accessible via groqcloud and groq api (openai compatible) so you can instantly get access to fast inference. :)

we have a pretty generous free tier and a dev tier you can upgrade to for higher rate limits. also, we deeply value privacy and don't retain your data. you can read more about that here: https://groq.com/privacy-policy/

https://groq.com/hey-elon-its-time-to-cease-de-grok/

> So they want to sell you there custom machines

They stopped selling the hardware to the public, and it takes an extraordinary amount of it to run these larger models due to limited ram.

Groq was suing Grok at some point, but Elon Musk is basically untouchable now.

Suing for what? The name?

Groq's blog post about the issue was a shitpost, not an actual legal document.

Reply on news.ycombinator.com