Hacker News Reader

Comment by komali2 | Hacker News Reader

People always talk about Moore's Law falling apart, and it seems like people didn't see it coming. I mean, I didn't, I'm just a guy that reads a lot of scifi. But I always assumed whiz-bang mathematicians had already figured out, I dunno, the threshold at which a processor can be nanosized, or like how big a processor could become before the distances became too great that due to the speed of light further computational advancement wasn't possible. I mean, I know nothing about this stuff, like I said, I just read a lot of scifi and have a high opinion of all the smart folk researching this stuff. Would love to learn more.

View on HN

> People always talk about Moore's Law falling apart, and it seems like people didn't see it coming.

People didn't WANT to see it coming. The data that it broke were back at 65nm but very much at 40nm and 28nm. RAM hasn't scaled in years.

However, the history of technology is littered with the corpses of those who bet against Moore's Law. Combine this with the fact that your job basically depends upon Moore's Law continuing, and you have something similar to the Wall Street collapse. I can bet against it, but I lose my job if I'm wrong, and I lose my job if I'm right. Better to just close my eyes and cash my paycheck.

For consumer evidence simply look at embedded microcontrollers. Note that RAM and flash haven't been scaling every 18 months for a while.

There's some truth in that. And there are also various approaches that can probably mitigate the end of CMOS process scaling for a while (special-purpose hardware, photonics, 3D stacking, etc.)

Some of these will cause some pain in other areas like software development.

That said, there's absolutely no guarantee that there's a true CMOS 2.0 on the horizon that will provides anything like the approximately 3500X performance gains we've seen from CMOS process scaling. There are huge macro implications if computing basically stops getting faster/cheaper/smaller from year to year.

I see this as a good thing. Now we can finally get down to what we should have been doing in the first place: better software, rather than just more of it and of mediocre quality. Moore's law has bailed out many producers of inferior quality software and now that that free run has come to an end we're going to have to take a really hard look at how we cook this stuff up.

But do you really believe that by taking "a really hard look at how we cook this stuff up" software engineers can double performance (in the sense of effective work) every 18 months? We can absolutely turn the dial toward more effort on software such as algorithms/compiled languages/etc. but color me unconvinced that there's exponential growth possibilities there.

Yes, I really believe that. And the reason why is probably not going to be very popular here but I believe that we took a wrong turn at some point in the past two or three decades that made us concentrate on the wrong thing. In the mid 80's there was a fairly strong push to substantially improve the quality of software, rather than the quantity. And when the web happened that all got flushed down the toilet and we now have about 500 ways to do the exact same thing, none of them very good, efficient or stable.

It's like the world just after the Cambrian explosion. Lots of room for expansion but no real plan or intelligence behind it resulting in a ton of repeat work and variations on themes that will ultimately go nowhere.

And once all that noise has died down the real work can finally begin, much more wood behind far fewer arrows.

I actually hope you're right. As you (approximately) wrote in another comment, "you canna change the laws of physics." CMOS has been a very special technology but it's obviously reaching the end of its life.

There are some exotic processes that give better performance but the economies of scale aren't there yet and the cost of production is probably always going to be much higher than that of Silicon, which is after all a very abundant element on Earth.

There was a fantastic video about IC manufacturing linked here on HN a while ago, I'm not sure if I'll be able to find it.

edit: Hah, found it :)

https://www.youtube.com/watch?v=NGFhc8R_uO4

>People always talk about Moore's Law falling apart, and it seems like people didn't see it coming.

First of all, let's be more precise. In this context, what people mean by the "end of Moore's Law" is the slowdown and, soon, probable end of CMOS process scaling. The issue is that CMOS process scaling has been such an incredible engine for performance that just getting those kinds of performance increases in other ways isn't easy. I wrote a bit about this a year ago.

http://bitmason.blogspot.com/2016/01/beyond-general-purpose-...

To answer your question, people did see this coming. There were disagreements about how far down you could economically drive feature sizes so there has been debate about exact dates. But pretty much no one thought CMOS features could be made arbitrarily small.

> But pretty much no one thought CMOS features could be made arbitrarily small.

Pretty much no one qualified, maybe. There are lots of laymen (including many True Believers in The Singularity) who operate under the impression that it's a fixed law of nature.

Have a look at this photo: http://m.eet.com/media/1169843/120906_intel_22_3.jpg

That's a fin of a modern transistor (Intel 22nm). The dots in the photo are atoms. It's fair to say that Moore's law is nearing an end or has already ended.

Atoms are too big, photons are too big, electrons are too big, and the speed of light is too slow. There's no longer plenty of room at the bottom.

Still, it's not like the physical size of semiconductors is the problem. A CPU today is maybe 100mm^2 of silicon. 4U of rack space can easily hold tens of thousands of CPUs, if you can power them, cool them, and connect them up in a useful way.

if you can power them, cool them, and connect them up in a useful way.

Good luck with that. Power costs go up linearly at best with cores if you don't have any more transistor scaling.

Power will always scale linearly with cores, of course, but you could make cores less power-hungry with reversible computing: https://en.wikipedia.org/wiki/Reversible_computing

Actually useful reversible logic is, as they say, an open problem.

My point was that a new transistor shrink would give you lower power vs. the same transistor in previously larger node. You get more compute-per-watt with small transistors. If scaling stops, then all kinds of things get more difficult.

I have heard that Electrons move about a millimeter second (although electromagnetic emanations propagate at the speed of light). This is allegedly the benefit of 'optical' computing.

Not quite.[1] Propagation in most interconnects is about half the speed of light, because real-world wires have capacitance. (As low as 25% for CAT 5 cable, as high as 90% for open-wire antenna feeds.) Optical fiber runs about 70% of the speed of light. On-chip interconnects run 30%-60% of the speed of light. So there's some propagation delay improvement with optical interconnects, but it's a factor of about 2, not some huge improvement.

[1] http://www.edn.com/electronics-blogs/all-aboard-/4426188/Rul...

People said similar things when we reached the wavelength of visible light.

Yes but that was a cost and engineering issue. The size of atoms is a lot more fundamental. EUV buys you a little more but you still bump into the size of atoms and quantum effects pretty quickly.

Think outside the box. Current chips have gates on a dozen metal layers measuring ~1µm, but you could scale to 10,000 layers at ~1mm. Sure there are many unsolved challenge to build 10,000 layers, but there is still PLENTY of space for Moore's law to continue in this 3rd dimension...

I assume you mean stacking transistors? This has been tried, and there are serious thermal issues. Where does the heat go in the intermediate layers?

If you could solve both the heat dissipation and yield problems that 3D chips have you would be a very, very rich man.

> "People always talk about Moore's Law falling apart, and it seems like people didn't see it coming."

EEs in the semiconductor industry have been talking about it for the past 20 years; they didn't know exactly when it would occur but they knew it was coming. It's only software developers who were blissfully unaware.

EDIT: The interesting thing about the end of Moore's Law from AMD's (and everybody who isn't Intel) perspective is that finally the playing field is moving closer to even. Before, Intel always had the most advanced fab processes because of the vast amounts of money they funneled into it, which meant that everybody else had to have a better architecture than Intel chips just to get equal performance. Now that Intel is losing that process advantage, it should become possible for everyone to compete purely on the basis of architecture again.

> "But I always assumed whiz-bang mathematicians had already figured out, I dunno, the threshold at which a processor can be nanosized, or like how big a processor could become before the distances became too great that due to the speed of light further computational advancement wasn't possible."

We're nowhere near the absolute limits imposed by the fundamentals of thermodynamics, quantum mechanics, etc. Those are relatively straightforward bounds to calculate but are completely useless for anything but extremely long-term projections.

What we wish we could compute is what the limits are on current manufacturing techniques and incremental advances from that. But we don't know precisely how many incremental improvements we have left to discover before we need to abandon silicon entirely.

But we don't know precisely how many incremental improvements we have left to discover before we need to abandon silicon entirely.

Well, it is not as if we will just suddenly stop using CMOS, but the end of improvements is in sight:

https://en.wikipedia.org/wiki/5_nanometer

If we are willing to putter along with no process improvements whatsoever, then yes, we can keep using 5nm indefinitely. And no doubt there will be incremental improvements in performance and yield at that node.

But we can't, and won't stop there to continue to drive down the cost of computing. It is just the end of the road for CMOS lithography.

What are your thoughts on the drivers for reducing the cost of computing beyond CMOS?

> But we can't, and won't stop there to continue to drive down the cost of computing. It is just the end of the road for CMOS lithography.

I mean, is there any reason to believe this is actually true? There's no law that just because something is super useful its price will continue to drop forever. Just check out this chart: https://www.nrdc.org/sites/default/files/import/switchboard/.... The blue line is the inflation-adjusted price of electricity by year. I think it's fair to say that electricity is even more vital to the modern economy than compute power, and that having more of it for cheaper would unquestionably raise the standard of living for everyone. There's also a functional and mostly-free market for its production. And yet its real cost has remained essentially stagnant over the last 40+ years.

Now, there are obviously reasons why Moore's law has been a thing for so long (probably a lot to do with the exponentially increasing number of chips we use and the experience curve http://www.economist.com/node/14298944), but the last 50 years in chips may have been more the exception than the rule.

I mean, is there any reason to believe this is actually true? There's no law that just because something is super useful its price will continue to drop forever.

The entire computer industry, well, the entire technology industry is founded on compute power becoming cheaper each year.

What happens to these billion-dollar companies when that is no longer the case? Will Intel's investors be OK with it when they announce that they won't be coming out with anything much better than what they've currently got? Or will they be punished (severely) in the stock market after such an announcement? [1]

The board of directors will respond to pressure from the investors (or get replaced), and will replace company leadership if needed to continue advancement in technology.

And so they (or someone else) will invest in new technologies to move beyond CMOS lithography, such as molecular nanotechnology. Which will disrupt everything else related to technology and biology as well.

[1] I read articles about gamers buying 5-year old Xeon processors for their game machines for $70 USD which are competitive with current-generation i7 desktop processors that cost much more. If Intel can't sell new processors that are effectively better than older ones, that's going to be a real problem for them, and the stock market will eventually notice.

> The entire computer industry, well, the entire technology industry is founded on compute power becoming cheaper each year.

No, we've just gotten used to it. So now, like any junkie with a bad habit we're going to have to get used to doing without that. Or deal with the increased electricity bill.

> The board of directors will respond to pressure from the investors (or get replaced), and will replace company leadership if needed to continue advancement in technology.

Company leadership being replaced is not going to magically change the laws of nature.

At some point even Intel investors will have to accept that there are such things as physical laws and nature doesn't care one bit about your quarterly earnings reports.

Company leadership being replaced is not going to magically change the laws of nature.

Well, that's just it, we haven't hit the physical limits of computing yet. But we are indeed close to the limits of what is practical with CMOS photo-lithography.

At some point even Intel investors will have to accept that there are such things as physical laws and nature doesn't care one bit about your quarterly earnings reports.

This acceptance process will be quite painful, and someone, some way, some how will step in to promise greater returns.

CMOS process scaling has been a pretty amazing unicorn. The fact that neither Intel nor anyone else has unveiled a CMOS 2.0 isn't for lack of research. There are various possibilities but they're either tactical and limited or very much still in the Labs phase.

I agree that the consequences of compute power no longer advancing year-to-year will be significant but I can't agree this is a case of nobody's trying hard enough.

I agree that the consequences of compute power no longer advancing year-to-year will be significant but I can't agree this is a case of nobody's trying hard enough.

I look at these yuuuuge teams at Intel working on their processors, and what kind of returns are we getting on that investment? How much are they, in comparison, investing in molecular nanotechnology related research?

I look at this, and it is mostly software-ish things:

http://www.intel.com/content/www/us/en/education/highered/re...

There is some hardware stuff, further down the page, and some talk of the post-CMOS world. But I don't see enough. In my view, they're not placing nearly enough emphasis on the foundational technologies they need for their business.

Not being privy to the details of Intel's R&D budget I don't really have an opinion on how their dollars are allocated. I will say that development tends to benefit from large teams more than speculative future research. I'm sure a very large proportion of their R&D budget goes to where revenue is going to come from over the next few years.

It's also the case that Intel has become less forthcoming on their semiconductor research directions so I wouldn't necessarily expect there to be a lot of information on their website.

I was doing some back of the envelope math for a blog post last night[1] and it looks like we're not that far away from the limits of what's possible - at least for non-reversible computing. Getting rid of a bit of information as in a Nand get is always going to cost at least 2.75 zeptoJoules and current transistor technology can do it for an attoJoule, so just another factor of 360 to go. Of course then we can try to transition to reversible computing to get around that[2].

[1]http://hopefullyintersting.blogspot.com/2017/04/the-coming-i...

[2]https://en.wikipedia.org/wiki/Reversible_computing

If I recall correctly, the lower bound on energy for destroying a bit depends (linearly?) on the temperature. Couldn't performance then be improved by cooling?

Yes, you're correct and I probably should have mentioned that.

I don't think it will matter much though. Whether you have to remove input power as heat or expend power to cool likely won't make a whole lot of difference. Just that the delta-t will be a lot larger and so it may actually cost more to keep that ultra-cool machine ultra-cool.

Maybe future data-centers will be used to pre-heat the water that goes into the turbines that power the electricity plant, in some kind of perverse version of a perpetuum mobile.

You'd need to make up for the difference lost on each cycle with extra fuel.

Or we could put future datacenters in space behind sunshades and get them really cold by default.

Bandwidth might leave something to be desired.

You need really big radiators for that. In space, waste heat is a huge problem.

Where did you get the number for attojoule per gate operation?

A factor of 360 is closer than I would have guessed. But that's still at least 8 doublings of compute efficiency. We've never had concrete roadmaps that far out—that would be like scheduling the introduction of FinFETs while Pentium 2s were first hitting the shelves. Nobody's ever been able to predict that far in advance which fabrication technologies will end up being economical at scale and required at which node. (Just look at how well the industry has coped with having to put off EUV so many times.) With the looming end of silicon scaling, those thermodynamic limits certainly don't look any less out of reach.

>Abandon silicon entirely

Abandon for what? In many series they hand-wave this away using words like "hyperspace substrate" or "dark matter" or "unobtanium." Do we have an idea of what we could use in reality?

I'm just curious, people. I knownothing about this.

The problem is at some point the gates start getting to the size of an atom. How do you get smaller than an atom? I don't think you can.

Well uh, can you squeeze the atoms closer together? I didn't think they were solid objects, but clouds of electrons? I guess eventually the nuclear power or whatever it is that pushes atoms apart becomes too strong?

Reply on news.ycombinator.com