With Nvidia Groq 3, the Era of AI Inference Is (Probably) Here (⌛ March 2026)

spectrum.ieee.org

2 points·mdp2021··4 comments

> Over the past few years, inference-specific chip start-ups were experiencing a sort of Cambrian explosion, with different companies exploring distinct approaches to speed up the task. The start-ups include D-matrix with digital in-memory compute, Etched with an ASIC for transformer inference, RainAI with neuromorphic chips, EnCharge with analog in-memory compute, Tensordyne with logarithmic math to make AI computations more efficient, FuriosaAI with hardware optimized for tensor operation rather than vector-matrix multiplication, and others

Let us add Taalas, which implemented Llama3 8b on hardware to achieve 17000 tokens/s at a small fraction of the usual power consumption. Quality test at https://chatjimmy.ai/

(General question: which Unicode can we use? To prepend a category icon to the article date - news but not the freshest -, I tried to use "Calendar" from the "Miscellaneous" set and it was not accepted; I then tried "Hourglass" from "Miscellaneous Technical" and it seems to work.)

HN submission titles don't need emoji.

The argument?

Not emojis but glyphs. There is the title, and there is metadata. Round brackets are already potentially in the title, as part of it (this submission is an example). Square brackets are for elisions, to complete the literal title. Other brackets do not seem to be immediately suggesting metadata. The article's datation is metadata. It makes sense, if one used the round brackets, to apply a tag to signify it's metadata, and to use a glyph is more efficient in more sense than writing a string (such as " (age/date/from: ...)" ).

Reply on news.ycombinator.com