Hacker News Reader

Comment by smusamashah | Hacker News Reader

Quote from article

    At the present rate, computers suitable for humanlike robots will appear in the 2020s. Can the pace be sustained for another three decades? The graph shows no sign of abatement. If anything, it hints that further contractions in time scale are in store. But, one often encounters thoughtful articles by knowledgeable people in the semiconductor industry giving detailed reasons why the decades of phenomenal growth must soon come to an end.

I don't know where to look to confirm that can current computers do 1 billion MIPS (Million Instructions per Second?) as predicted by this article?

EDIT: This Wikipedia article puts an AMD CPU at ~2 million MIPS https://en.wikipedia.org/wiki/Instructions_per_second as the highest one.

View on HN

The promise of Instruction per Second scaling forever did stop being super-exponential. Instead of CPU based Instructions per Second we have continued scaling compute with GPU based Floating-point Operations Per Second which don't exactly have a 1:1 equivalence with instructions but are close enough. Nvidia released the "first" GPU in 1999 so it's not too surprising that this guy wasn't thinking in those terms in 1998. That's why Kurzweil and Bostrom used FLOPS in their extrapolations. They also end up in the neighborhood of 2020s/2030s in their later predictions in the 2000s.

With that in mind we are currently at 1 billion GigaFLOPS for the largest super computer in 2023. And the trendline since the 90's has kept up with the prediction made in the post. Although we can't exactly say 1 GigaFLOPS = 1000 MIPS.

https://ourworldindata.org/grapher/supercomputer-power-flops...

When the article was written, almost nobody cared about GPUs or similar architectures being really good at parallel computation.

So while he wrote about MIPS, and those have indeed basically stopped significantly improving, FLOPS have continued to improve. And for AI in particular, for inference at least, we can get away with 8 bit floats, which is why my phone does 1.58e13/second, only a factor of x60 from a million-billion.

Today we'd include the GPU. If we include supercomputers, we really have to compare total global capacity, which is on the order of 1 million times that of 1997 [0]. That basically fits his trend line.

[0]: https://ourworldindata.org/grapher/supercomputer-power-flops

The graph in the article showing computing power per $1000, doesn't that make the comparison to supercomputers misleading?

> I don't know where to look to confirm that can current computers do 1 billion MIPS (Million Instructions per Second?) as predicted by this article?

An RTX 4090 has AI compute of up to 661 teraflops in FP16[1]. FLOPS and IPS are not interchangeable, but just for fun, that's 6.6 * 10^14 FLOPS, which is just short of 1 billion million (10^15).

1. https://www.tomshardware.com/features/nvidia-ada-lovelace-an...

I'm not sure he predicts a billion mips. The graph tops out at a million mips per $1000. He figues brain equivalence is 100 million mips. If you assume mips are equivalent to flops then that's 100 teraflops. That would seem roughly to match:

>"NVIDIA GeForce RTX 4090 Is The First Gaming Graphics Card To Deliver 100 TFLOPs of Compute Performance" https://wccftech.com/nvidia-geforce-rtx-4090-is-the-first-ga...

Currently about $1,800 on Amazon.

Nvidia claims the H100 GPU can do >3e15 8-bit operations per second (3 billion - million instructions per second):

https://resources.nvidia.com/en-us-tensor-core/nvidia-tensor...

M1/M2s are around 2-4 trillion if I’m not mistaken

~2.6 teraflops FP32 (CPU) for the M3 Max.

not even within 2 orders of magnitude.

Just rack-mount a hundred iphones. :-P

Reply on news.ycombinator.com