Hacker News Reader

Hacker News

Comment by Lewton | Hacker News Reader

Full thread

Lewton

Unless there's a page 2 I'm missing, this is a terrible article.

He seems to dismiss the singularity with 3 points

1. We don't understand human intelligence

2. "We've" been wrong about it before

3. Moores law is ending

1. Who says it has to be human intelligence? 2. That's not an argument 3. Moores law -might- be ending but datacenters are getting bigger and bigger

View on HN

Seems to be a lot of people thinking Moore's Law ending means that innovation in computers is over. It's really just the opposite. We can't rely on simple scaling anymore, so we need to take more innovative steps to push computing forward.

There's a limit to what you can do without "simple" scaling, meaning, without making cheaper, faster, smaller individual gates. That is, an architecture optimized for a given problem domain will give you maybe 1000x efficiency improvement over a more general architecture, but that's it. (Typically it will be <1000x, but let's say you're lucky; the point is, that's your last 1000x, and then you'll see no improvements, ever.)

So if Moore's law is indeed over, progress will be slow, domain-specific, and decelerating due to diminishing returns (specializing for a problem domain is worth it as long as that domain is not too narrow but at some point it's too narrow to justify the investment into building specialized hardware for it; you'll be better off using something less specialized. GPGPU, which does not provide amazing performance in an absolute sense, but does beat CPUs on a large range of problems, is an example to this - any accelerator more efficient than GPGPU needs to be justifiable in the sense that GPGPU is already there wherever there's a need for GPU doing graphics, which is where GPUs do provide amazing performance in an absolute sense [you can totally beat GPGPU with more specialized hardware on most benchmarks, while you can't beat GPUs in graphics.])

I'm a chip & accelerator architect, so it's not like I'm particularly happy about this, I think I'm realistic though. A higher-caliber architect saying the same pessimistic thing is Bob Colwell.

The one nice thing about really stopping at some manufacturing technology and not being able to improve any further is that the cost of using this technology will likely continue dropping for some years. Only when it reaches the bottom will progress have truly stopped.

"it's too narrow to justify the investment into building specialized hardware for it" FYI, if the only way to improve perf is to build custom hw, the cost and complexity of building custom hw will drop.

Erm... Not necessarily. The only way to travel quickly between 2 cities is a direct flight, but there isn't a direct flight between any 2 cities, because the cost of having such a flight didn't yet drop to the point where it's economical. So it will be with many types of custom hw.

The human brain needs about twenty watts, fits in a compact package and outperforms computers on a variety of tasks. Squishy wetware is pretty far from computronium. I think we still have a long way to go before we hit fundamental limits.

What do you mean by "fundamental"? Do you mean a theoretical configuration of atoms, or do you mean practically manufacturable?

I mean limits that we won't ever break for whatever reason, be it physics, engineering, or economics.

Interesting analysis, thanks. What do you think about the development of more radical redesigns like quanum computers, memristors, or anything else that might show promise to displace the traditional CPU? It seems that there's a lot going on in the more experimental side of computing right now that is reaching enough maturity where experiments might become practical products within the next decade. Even if we don't have pocket-sized quantum computers, the existence of cloud computing might make something like quantum annealing practical in the near future. Is there anything that has you excited?

Displacing CPUs is not the issue (the CPU occupies less than half the area of a modern application processor chip for instance), the problem is displacing CMOS. I don't know much about the alternatives, but I trust Colwell's skepticism.

1. We don't have to understand how it works to make something that works as well. It can always be found by trial and error.

2. That's indeed not an argument.

3. Whatever the human body does, it does not require any of its parts to be smaller than an atom. Transistors are already much smaller than a neuron. When Moore's law ends, companies will focus on making them more efficient so they can use more of them in the same time.

Reply on news.ycombinator.com