Hacker News Reader

Comment by w4ffl35 | Hacker News Reader

> but ain't no way we run models meant to run on 8 nvidia A100 on our smartphones in the next 5 years

m$ has been working on an AI chip since 2019 so i think we will.

An A100 is about the size of a brick, there is no way we're fitting those 8 bricks in a phone in the next five years, without even thinking about heat management

An A100 HGX server is ~6kW of power consumption (and associated heat), while an iPhone is O(1W). I agree that a 6000x increase in energy density or 6000x decrease in power consumption is unlikely in this decade.

I mean we could maybe put uranium in iphones...but not that many transistors

They still can't beat the laws of physics. It's just the kind of density that doesn't seem feasible to put in a smartphone.

Human brains seem to be of the same order of magnitude in terms of size, and seem more nuanced and capable than GPT4.

The human brain is also three-dimensional, heavily interconnected, and has built-in thermal management at every scale. Chips are much faster, but still operate on the essentially linear memory cells, and this limits how many matmuls you can do per second. If we can figure out true connectivity without doing tons of matmuls, then we should be able to massively cut computational demands of models.

It's not apples to apples comparison. There's a multitude of tasks that human brains are very bad at.

GPT excellence is in raw knowledge and answering machine. You won't find a single human brain that can hold the same amount of knowledge

And? Nvidia has been working on AI chips for years.

Reply on news.ycombinator.com