Hacker News Reader

Comment by tancop | Hacker News Reader

Their numbers look too good to be true, they have no identified customers and the whole site is generated, but I think the principle behind it is good. If they can pull off the error correction needed to make analog reliable we might have a great new option for cheaper more eco friendly AI. Then again it could turn out to be a total scam.

View on HN

> Their numbers look too good to be true

Why? I have not seen anything outlandish for a NN implementation (vs a NN simulation).

> can pull off the error correction

There lie the issues that have not been mentioned, the solutions not explained. Analog computing means: * costly digital-to-analog at the input and analog-to-digital at the output; * sensitivity to environmental conditions such as temperature; * signal dispersion hence the need to boost it in the path.

Maybe checking the patents they registered?

Temperature is a relatively trivial issue because it can either be compensated or the chip simply kept at a constant temperature (they’re low power chips anyway, throttling could be skipped to simplify the design). The ADCs and DACs are indeed the main thing though. The whole analog compute game is about making those power efficient and scaled enough that the math still works out in your favor. The demos do work though, this company is far from the only one; see [1] or [2] for example.

Flash NAND can routinely be bought with 4 bits per cell, perhaps even 5 soon (QLC and PLC drives). Since it has been proven to store 4 digital bits at production scale, I’m willing to bet an analog architecture running an LLM should be able to yield the performance analog of a 4-8 bit quantized model. Where in that 4-8 range is pretty crucial, but it depends on the specific design.

[1] https://www.nature.com/articles/s41928-023-01010-1 [2] https://www.nature.com/articles/s41586-022-04992-8

(Moderation, the post https://news.ycombinator.com/item?id=49404099 from SkyPuncher is legit. Automated filter? Maybe you can give us the ability to vouch?)

My 15 second read of just the front page aligned with you, but when I saw replies pushing back, I went back and loaded News, then cross-verified some of the claims. It's real.

Like the numbers they claim could literally make LLMs 20x profitable. If it were true then why isn't every AI company trying to buy them out?

c.f. https://news.ycombinator.com/item?id=49403836, then from there, you'd need to see a couple of orders of magnitude before it's tractable for LLMs.

Bottom of page linked from HN (currently https://www.mythic.ai/) indicates they're hoping to demonstrate something that could that in 2028 or later, and both Nvidia and Cerebra are looking at 10x'ing models to 10T+ plus in 2027.

So they may never catch up on LLMs.

They're a good fit for the companies they're working with and have taken investment from, ex. Toyota, that aren't doing LLMs.

They plan to be able to run 1T parameter models next year

https://www.mythic.ai/vanguard

Seems big, IF true

Notice that they do not talk about SRAM when they present the M1 chiplets - but you have to store the kv-cache etc. somewhere to run LLMs.

The technology that could run LLMs should be the "Vanguard", but as the homepage says, "the M1 (scope: Edge/Cameras/Drones) is there, the Vanguard should be a reality in 2027".

I'm pretty sure the answer is that their chip is uniquely unsuited for LLMs, probably because the write speed might be horrendously slow. Most likely too slow for storing the context window in multi user workloads.

Bulk bitwise operations in a memory array driven by an out-of-spec controller is a thing, but I'm not sure why they're using NAND for this and not DRAM like everyone else, and why it's so small (just 80M weights). And the page is insufferable and doesn't tell anything about how it works. Everything about it seems off.

I went down this rabbit hole earlier in the year.

You generally don’t need error correction. Instead, you do a retraining of sorts where you basically fine tune the model on the target chip architecture. Because these systems are so complex, errors tend to localize without affecting overall output.

My understanding is the hardest part is actually noise in the analog system.

Reply on news.ycombinator.com