Hacker News Reader

Hacker News

Comment by JKCalhoun | Hacker News Reader

Parent Full thread

JKCalhoun

Apple should have done this yesterday. A local AI on my phone/Macbook is all I really want from this tech.

The cloud-based AI (OpenAI, etc.) are todays AOL.

View on HN

The die size is huge. This isn’t the kind of chip that would go into your MacBook, let alone an iPhone.

It’s for cloud based servers.

And computers used to be the size of a room. I think they can get it to iPhone size in the future, this is an early prototype.

Well, there's a limit to how small we can make transistors with our current technology. As I understand it, Intel is already running into those limits with their new CPUs (they had to redesign the fins IIRC). I can imagine that without an actual breakthrough in chip manufacturing the size could stay large. That's not to say that a breakthrough won't happen, though.

Yes, in 2D, but NAND has been using layers for a while. We call HBM interposers 2.5D. 3D breakthrough would be pretty easy but for those pesky problems like power delivery and cooling. (/s)

But give that time (e.g. microfluidics) - something interesting is that it would be extra hard to use all layers at once, but NN might be a good fit, imagining that computation will be sparse (subsets activating simultaneously)...

That's the part that people are missing: it won't get smaller. It already required heroic optimization to get 8B on one megachip. Taalas is more expensive but faster. It is cheaper per token when running 24x7 but not cheap to buy. It will never be small and never be cheap.

"It will never be small and never be cheap."

Will your comment age well? We'll see.

We might all be surprised if (somehow, ternary logic?) models come down drastically in size. It doesn't have to be the hardware getting more dense.

The hardware isn't there yet. Apple's neural engine is neat and has some uses but it just isn't in the same league as Claude right now. We'll get there.

They did do it yesterday.

And it produced fake headlines and summaries including the threat of lawsuits from involved person(s).

Apple usually waits until somebody else has refined a technology to "invent" it, but I guess they couldn't wait for this one.

https://developer.apple.com/documentation/FoundationModels

Reply on news.ycombinator.com