Comment by rerdavies | Hacker News Reader

Comment by rerdavies | Hacker News Reader

Parent Full thread

> Blackwell GPU has 208 billion MOSFETs[3]

Things us hard sci fi fans will insist on:

- You have significantly undercounted transistors. As of 2024 you can put up to 8.4 terabytes of LPDDR5X memory into an NVidia Grace Blackwell rack. So that's 72 trillion transistors (and another 72 trillion capacitors) right there.

- A GPU executes significantly faster than a neuron.

- The hardware of one GPU can be used to simulate billions of neurons in realtime.

- Why limit yourself to one GB200 NVL72 rack, when you could have a warehouse full? (what happens when you create a mind that's a thousand times more powerful than a human mind?)

You really need to separate the comparison into state (how much memory), and computation rate. I think you'll find that an NVidia GB2000 NVL72 will outperform a brain by at least an order of magnitude. And the cost of feeding brains far exceeds the cost of feeding GB2000's. Plus, brains are notoriously unreliable.

The current generation of public-facing AIs are using ~24Gb of memory, mostly because using more would cost more than can conveniently given away or rented out for pennies. If I were an evil genius looking to take over the world today, I'd be building terabyte-scale AIs right now, and I'd not be telling ANYONE. And definitely not running it in Europe or the US where it might be facing imminent legislative attempts to limit what it can do. Antarctica, perhaps.

> The hardware of one GPU can be used to simulate billions of neurons in realtime.

You mean simulate billions artificial neurons, right?

You can't simulate a single biological neuron yet because it's way too complex and they still don't even understand all of the details.

If they did have all of the details, at minimum you would need to simulate the concentrations of ions internally and externally as well as the electrical field around the neuron to truly simulate the dynamic nature of a neuron in space and time. It's response to stimulus is dependent on all of that and more.

Depends a bit on how certain you are on your preferred answer to the question: is the brain analog or digital? If you think it is predominantly digital, quite a lot of the details of the biochemical mechanisms don't matter. If you think it is predominantly analog, they matter hugely.

Do any serious theories on the brain being predominantly digital hold any water with neuroscientists? I feel like it's fairly well accepted that the brain isn't digital even just with what we do understand about neurons?

I don't think it's possible to tell at this point.

If you looked at the operation of a digital computer, you'd find an analog substrate, with some quite interesting properties. Deciding whether the analog substrate is an important semantic element of how the computer worked, or whether its operation should be understand digitally, is as much a (research-driving) leap of faith as anything else.

That's actually a pretty convincing point. If all we were able to understand was the flow of electrons in a transistor and not how they were connected, we wouldn't necessarily be able to conclusively claim that digital computers were indeed digital.

Action potentials are digital. Once the neuron hits the trigger threshold, the spike happens at the same amplitude every time.

The process by how the neuron determines whether it's time to fire or not is almost grotesquely analog, in that it is effected by a million different things in addition to incoming spikes.

How much of each process matters for what is the mind-generating part of the brain's work? We don't know.

Myelination (which allows very fast, and digital, communication between neurons) is universal in vertebrates, which points to it being important for intelligence.

As a counterexample, we have cephalopods, which accomplish their impressive cognitive tasks using only 'slow' neurons; they get around the speed limit by having huge axons, which reduce internal resistance. So we know that digital transmission isn't necessary for intelligence, either.

The nature of cognition is a fascinating subject, isn't it!

> Antarctica, perhaps.

Skull Volcano lair on an island. Geothermal energy to power the place. Seawater for liquid cooling.

Then of course, you also need the beautiful but corruptible woman as some high-up part of the organization. And a chief of security in your evil enterprise with some sort of weird deformity or scarring - preferably on the face.

Protip: If you catch a foreign spy on the island, kill them immediately. Do not explain your master plan to them just before you kill them.

1) There is not enough data in the entire world to train a trillion parameters large language model. And there is limited use for synthetic data, yet (we need an internal adversarial/evolutionary model like AlphaZero for that, which I think will be developed soon, but we are not there yet).

2) Models are not linearly scalable in terms of their performance. There are 8B parameter models (Zephyr-Mistral) routinely outperforming 180B parameter models (Falcon). Skill issue, like they say.

3) Even if you are an evil genius, you need money and computing power. Contrarily to what you might be reading in the media, there isn't much capital available to evil geniuses on a short notice (longer term arrangements might still be possible). And even if you have money, you can't just buy a truckload of H100s. Their sales are already highly regulated.

Antarctica, perhaps

Powered by what?

Solar panels in a place where the sun doesn't even rise for months at a time? That big beefy transmission grid that the Antarctic continent is so famous for? The jillions of oil wells already drilled into the Antarctic?

Oh wait, I know: penguin guano.

Pass the crack pipe, you've had enough my friend.

Nuclear reactors, of course.

Antarctica was chosen because the area on the mainland between 90 degrees west and 150 degrees west is the only major piece of land on Earth not claimed by any country[1], which makes it a good place place to locate businesses that do not wish to be subject to regulations on Artificial Intelligences. Plus it's much more inconvenient to attack than it would be if you located it in a volcanic island.

[1] Sez wiki.

Reply on news.ycombinator.com