Why CPUs aren't getting any faster (2010)
technologyreview.com
technologyreview.com
I work on the Dolphin Emulator (https://dolphin-emu.org/) which is a very CPU intensive program (emulates a 730MHz PowerPC core, plus a GPU, plus a DSP, all of that in realtime). We try and track CPU improvements to provide our users with proper recommandations on what hardware to go for. Here are the results of a CPU benchmark based on our software: https://docs.google.com/spreadsheet/ccc?key=0AunYlOAfGABxdFQ...
Wall 1991[1] is the classic paper, AFAIK, and saw a limit with "quite ambitious" (for 1991) imagined future hardware at around 5-7 IPC in a 'median' benchmark. I'd be interested to hear how close Intel is getting and whether anyone's updated that result.
[1] http://www.eecs.harvard.edu/~dbrooks/cs146-spring2004/wall-i...
I was big into building computers during the cpu race between AMD/Intel back in the late 90's early 2000's and the Intel Pentium 4 processor line is notable for pushing the envelope from the the high 2Ghz range all the way up to the 3.4Ghz and 3.6ghz (I still have a 3.4Ghz chip sitting in my home office ... those were the days!)
Wikipedia does a great job of chronicling what happened with the Pentium 4 line here http://en.wikipedia.org/wiki/Pentium_4 with hints to what I've just alluded to above
"Overclocking early stepping Northwood cores yielded a startling phenomenon. While core voltage approaching 1.7 V and above would often allow substantial additional gains in overclocking headroom, the processor would slowly (over several months or even weeks) become more unstable over time with a degradation in maximum stable clock speed before dying and becoming totally unusable"
It was after their failures with the brute force attempt at higher cpu cycles that Intel finally went a different way with initially the Pentium M line (code named Dothan and Banias) http://en.wikipedia.org/wiki/Pentium_M and eventually the Core duo/Core series that they've since built on.
This is an electromigration[0] and/or hot carrier injection[1] problem. Both are exacerbated by temperature and voltage, but are not the reason chips are stuck at 3-4GHz.
One actual limiter is pipeline depth. Deeper pipelines is an easy way to enable an increase in frequency. The problem in a nutshell- the longer the pipeline, the greater the overhead & the larger the penalty of branch misses.
while this nitpick does not change the overall argument, Intels high-end has hovered in 3.5-3.7 GHz area for some time now, with "turbo" hitting 4 GHz.
To "hide" this, Intel has refocused its marketing on power consumption, where gains seem easier to achieve (for now), as well as other pure marketing tricks such as calling what used to be "Turbo Boost speed", the "normal speed". I've noticed for example recently a Bay Trail laptop being marketed at "2 Ghz", even though Bay Trail's base speed is much lower than that.
AVX was just about floating point.
I always found calling it turbo boost speed odd. During the majority of applications that exist today that is the speed you will get.
By using Turbo Boost, they have their cake and eat it, too. They can do well in benchmarks thanks to Turbo-Boost, but also do well in power consumption tests, because most of the time the chip will not be running at Turbo Boost speed, and will be throttled to "normal speed" as soon as it passes its power consumption limit.
So Intel is being misleading by using Turbo Boost speed as the normal speed.
Also we're at the point where most of the very-requesting software isn't bottlenecked by the CPU, or where you can just throw more cores at the problem and solve it. And also software is starting to leverage GPU acceleration, which gives an huge boost when usable. And GPUs are getting a lot faster every new generation.
This limit doesn't immediately impact computations that are carried out entirely on-chip. I expect that's the great majority of computations on today's CPUs, which typically have large caches. So I'm not sure the GP's explanation is correct, although it's certainly interesting.
6cm now means that you only need to double twice to get to less than the die width.
Also, electricity does not propagate through copper at c, but at between 40% and 90% of it, depending on the inductance and capacitance of the circuit, so you are looking at even less of an increase before you start running into some really thorny problems as not only are you running up against relativistic limitations, but different bits of your chip can be doing it at noticeably different times.
Why isn't Intel able (or willing) to push the power envelope higher in desktops?
It really has more to do with the customers. The people buying these chips by the thousands are Enterprise Desktops, and keeping power bills down is their #1 goal. The customers don't buy GPUs in these machines either, they expect Intel's to be "good enough."
Unfortunately, consumer desktop purchases are basically a rounding error to Intel's multibillion dollar balance sheets, so they don't build parts for that segment. Instead, they blow some fuses on their enterprise parts and sell them as "Extreme Performance", "Enthusiast Desktop" or whatever other euphemism they want to use for "overclockable" these days.
Another big improvement has been moving certain functions to hardware - Intel's Quicksync is a great example of this.
Almost all the singularity-type techno-optimism was based on a curve drawn during the amazingly magical part of CPU development: shrink the die, double the speed.
We are now past that, although many techno-optimists are casting around for a reason it will continue.
Although that is an enormous if.
We may have to make due with what we've got.
Which, thankfully, is quite a lot.
Or, like you said, yep, we do have a lot.
In the longer term, we'll hopefully see advancements that let us fundamentally change how logic processors are constructed, such as possibly photonic logic chips. Only a major shift will let us break through the current single-thread performance wall.
On the software side, I've always understood browsers are pretty good at parallelism, which is a pretty major platform that gets performance benefits. That could also extend even further with projects like Mozilla's Servo (https://github.com/mozilla/servo), a browser engine built from the ground up with parallelism in mind.
The work we're doing with PJs, however, attempts to fix this problem :)
In the short term, it would be great if we had a breakthrough in x-ray power sources (for cheaper EUV lithography), a breakthrough in etch control (for vertical transistors), or a breakthrough in III-V materials (for a one-time mobility improvement).
But long term, we need to find an alternative to the transistor. Perhaps spintronics, perhaps non-von Neumann architectures, who knows. But there is no way that a paradigm-changing redesign will be competitive with silicon in the next 10 years. Silicon has a humongous advantage in manufacturing, supply chain, know-how, scaled production, etc. Even if we find a better technology (and we haven't) it may still take decades to get there.
Of course there's the well-known reasons, nonlinearity of power vs frequency scaling, diminishing returns in hardware design, etc. But there are others that we don't hear so much about.
Hardware design is still in pretty nascent stage, technology-wise. The languages used (say SystemC or Verilog) offer very little high-level abstraction, and the simulation tools suck. Sections of the CPU are still typically designed in isolation in an ad-hoc way, using barely any measurements, and rarely on anything more than a few small kernels. Excel is about the most statistically advanced tool used in this. Of course, CPUs are hugely intertwined and complicated beasts, and the optimal values of parameters such as register file sizes, number of reservation stations, cache latency, decode width, whatever, are all interconnected. As long as design teams only focus on their one little portion of the chip, without any overarching goal of global optimization, we're leaving a ton of performance on the table.
And for that matter, so is software/compiler design. The software people have just been treating hardware as a fixed target they have no control over, trusting that it will keep improving. That makes us lazy, and our software becomes more and more slow, by design (The Great Moore's Law Compensator if you will, also known as https://en.wikipedia.org/wiki/Wirth%27s_law).
The same problems we see in hardware design, huge numbers of deeply intertwined parameters, also applies to software/compiler design. We're still writing in C++ for performance code, for chrissakes. And even beyond that, the parameters in software and hardware are deeply intertwined with each other. To optimize hardware parameters, you need to make lots of measurements of representative software workloads. But where do those come from, and how are they compiled? Compiler writers have the liberty to change the way code is compiled to optimize performance on a specific chip (even if this isn't done so much in practice). To get an actually representative measurement of hardware, these compiler changes need to be taken into account. Ideally, you'd be able to tune parameters at all layers of the stack, and design software and hardware together as one entity. That is, make a hardware change, then do lots of compiler changes to optimize for that particular hardware instantiation. This needs to be automated, easy to extend, and super-duper fast, to try all of the zillions of possibilities we're not touching at the moment. There's even "crazy" possibilities like moving functionality across the hardware/software barrier. Of course it's a difficult problem, but we've made almost zero progress on it.
Backwards compatibility is another reason. New instructions get added regularly, but those are only for cases where big gains are achieved in important workloads. For the most part, CPU designers want improvements that work without a recompile, because that's what most businesses/consumers want. One can envision a software ecosystem that this wouldn't be such a problem for, but instead we have people still running IE6/WinXP/etc. Software can move at a glacial pace, and hardware needs to accommodate it. But this of course also enables this awfully slow pace of software progress.
Parameters like register file size are not determined in design (except where simple physics dictates). That's an architecture problem, which is constrained by the quality of simulation capabilities (which you did mention).
Another interesting thought- the design of a CPU starts many years before it hits shelves. So to accurately model performance (to make good decisions), architects must simulate a CPU running the code of five years in the future! There are some things that stay constant- tight loops, high level branches... but what about multithreading, for example? How multithreaded will the software of 2019 be?
> Hardware design is still in pretty nascent stage, technology-wise. The languages used (say SystemC or Verilog) offer very little high-level abstraction
There are a lot of things people outside microelectronics miss in this:
* Much (most?) of the research invested into HDLs is not related to the languages themselves. Verilog and VHDL and SystemC are really good enough for the things they describe. Really, they are; they may not look like much in terms of "high-level abstraction", but when you need to understand the relation between what you write and what gets done on silicon, that's actually a feature. It's turning Verilog code into silicon that's actually challenging and where a lot of effort goes. The benefits of being able to describe something in a functional manner and 10% fewer lines of "code" are belittled by the benefits of getting better output from a Verilog compiler.
* Some (not all, but some) ASIC design is actually done without HDLs. I don't know if Intel still does it. AMD stopped at one point and the results were tragic.
* There are, seriously, realistically, very few actual obstacles that stem from HDLs. Physical obstacles -- not just the laws of physics, but the difficult technological processes -- are considerably bigger obstacles. And then there's market presure:
> Sections of the CPU are still typically designed in isolation in an ad-hoc way, using barely any measurements, and rarely on anything more than a few small kernels.
"Barely any measurements" is not an accurate description of the ASIC design process IMHO. Yes, simulation/model extraction tools are used a lot more than in other fields of electronics, but that's somewhat hard to avoid when a) you're designing for a barely tested technological process (because it's brand new!) and b) prototypes are kind of expensive.
> Excel is about the most statistically advanced tool used in this.
I haven't done CPU design but I've seen some IC design being done (and wrote tools for people who did it). Maybe it's true for some CPU manufacturers, but I doubt it. I'm pretty sure that it especially won't cut it for mixed-signal designs like microcontrollers.
> As long as design teams only focus on their one little portion of the chip, without any overarching goal of global optimization, we're leaving a ton of performance on the table.
I'm not sure what you understand by "global optimization" but there's a lot of research done on topics like power-efficient caches, which is at least a form of optimization of more than one trait.
> The software people have just been treating hardware as a fixed target they have no control over, trusting that it will keep improving.
This is, unfortunately, an accurate description. There has been a time when software people weren't treating hardware as such, due to lack of abstraction-level tools, and I'm quite sure not even Wirth wishes to go back in time to then.
> To optimize hardware parameters, you need to make lots of measurements of representative software workloads. But where do those come from, and how are they compiled? Compiler writers have the liberty to change the way code is compiled to optimize performance on a specific chip (even if this isn't done so much in practice).
This is actually done quite a lot in practice, even within a single CPU family, like on x86, although granted not as much as it could be done. However, optimizing hardware parameters is somewhat meaningless as long as you need a general-purpose CPU. You can only go so far when considering potential workloads -- but that is being done, too. IBM's POWER cores, for instance, still try to improve sequential performance, because many of them still run single-threaded batch jobs for much of their functioning time.
> To get an actually representative measurement of hardware, these compiler changes need to be taken into account. Ideally, you'd be able to tune parameters at all layers of the stack, and design software and hardware together as one entity.
This has actually been attempted since the first reconfigurable logic arrays emerged in the late 70s/ early 80s. Unfortunately, it's plagued by the problem that the infrastructure required to dynamically reconfigure logic circuits is a drag. That's actually what makes FPGAs so slow.
I don't think FPGAs are the end of it, but "almost zero progress" is many light-years away from being an accurate description of the progress that has been made since MMI's PAL.
> Backwards compatibility is another reason. New instructions get added regularly, but those are only for cases where big gains are achieved in important workloads.
It's also done because practical experience has shown that, barring extreme stuff like OISC or perverted register-starved architecture design, instruction sets literally have negligible gain in terms of performance. Given enough time on the market, compilers that efficiently exploit any architecture will be designed. I mean shit it worked for the 80386!
However, not adding new instructions != not adding new features, not since microcoded architectures appeared. Granted, manual optimization is (sometimes?) still needed for extended sets (e.g. SSE), but their applications are niched enough that those applications that can't make use of them based on compiler inference alone are bound to be hand-optimized anyway.
Edit: compulsory disclaimer. I didn't do industrial level IC design, but I used to do research directly related to it and write software for that. This does mean I never actually did anything significantly more complex than a couple of current mirrors and logic circuits, but I did work with people who did. However, my experience is more on the analog IC side. I'm only casually familiar with logic IC design, so I may be unintentionally bullshitting you :-).
I'm mostly a software/compiler guy, where I know there can be a whole lot of improvement (especially in mainstream tools, not just toy/research projects), but my impression is that hardware isn't much better off. You've given me some good pointers for further topics to research, though. My email is in my profile if you want to chat more about this stuff!
Coming up :-). Cheers!
Not even- at early stages of the project, you're designing for a technological process that is purely theoretical!
Well, we estimate that when we figure out how to make these transistors a year or two from now, they will be about X fast...
There is also three economic reasons:
1. Fabs that make chips are much more expensive now.
2. Intel has no real competition, AMD limps along.
3. Most desktop users are happy with their computer speed, many still run XP
Dopped areas on silicon transistors have a minimum diameter around 10nm, that's the uncertainty about the position of charge carriers. Features a bit smaller than that will probably work, but much smaller ones won't.
That said, we are still far from the ultimate limits on CPU design. Those limits above apply to CMOS CPUs made of silicon what, despite describing all of the currently produced ones, is only one among many possibilities.
Probably some radical break through using some basic physics principle some guys worked on 30 years ago and left in a library somewhere ;).
http://research.microsoft.com/en-us/events/fs2013/doug-burge...
I'd say the end of Dennard scaling is why frequencies stopped going up around 2003.
What's killing Moore's law now is not Dennard's law ending, but the increasing price of lithography technologies. Right now we face the unpalatable choice between quadruple patterning (very expensive) and EUV (also very expensive). The industry is between a rock and hard place.
Now before the flames begin, let me caveat that as "We don't really need them to be much faster at single-threaded workloads." The article round-aboutly mentions this in the context of specialized CPUs, specifically GPUs: GPUs are basically hyper-concentrated thread runners. They're not very fast at running any single thread, but they have efficient shared memory and can run thousands of individual threads at once.
For larger workloads, we've gotten a lot more efficient through cloud computing. An individual CPU in the cloud is really not any faster than it was 5 years ago; but the advances made in energy efficiency (aka heat) and miniaturization mean you can fit a lot more of them in a smaller space.
While the technical hurdles to going faster are very real, I think we've built a technical infrastructure that's just not as reliant on the performance of any single piece of the system as it used to be. Therefore there is less demand for faster CPUs, when for many of the traditional "hard" computational workloads, more CPUs works almost as well and is a lot easier to scale than faster CPUs.
Just one of many possible examples, the minimalist ios7 interface has an embedded physics engine so that all the springy/sliding views feel natural to the user.
> In my experience a minimalist design often requires more effort behind the scenes.
Why ? When I say minimalist I essentially means less useless fancy features. Minimalist focuses on the useful, not on the feeling, so it can leave more room for other new features. The iPhone has a power of a 1995-2000 full fledged desktop PC or maybe more, yet it's using a lithium battery and it's 1cm thick.
This applies to all hardware: it's not worth it anymore to invest into new hardware if you're unable to exploit current hardware to their full potential.
You can throw money at chip technology, but at some point you also have to try throwing money at software and OS developers. Computers have always been about software, not hardware.
My take on a minimalist design means that life is made easier for the user. E.g. process data so the user has conclusions to interpret instead of raw data (that fits a minimalist design, but takes more processing).
I see now that your take on minimalist design is to remove unnecessary features from software. I'd argue that nobody wants "useless fancy features", unfortunately, what may be useless for me is necessary for you (otherwise why build those features anyway). So, it's all good to say let's keep to minimal designs (by whichever definition you mean), but in the end I don't see how that solves the problem that returns on hardware improvements is slowing, and frankly you haven't made the case that minimalist design solves the problem either.
I don't understand that, I think I meant the opposite.
> and frankly you haven't made the case that minimalist design solves the problem either
You gave the example of the iPhone. The apple platform is one of the least flexible platform. In many places it requires to learn an unpopular language, objc, which is not used on existing, non-apple platform for many reasons, first one being that nextstep is owned by apple. Then it requires approval from the apple store.
John Carmack talked about those "layers of crap", they're present in most OS.
And what do you mean by "solving the problem" ? Having a minimalist designs allows you to leave more resources for other things, it doesn't "solve the problem", it just leaves room for more improvement.
> life is made easier for the user
So basically you're forcing everyone to walk on paths, and forbidding everyone to explore forests. I understand that it's better for the mainstream customers because they're not able to learn how to walk in forests, but flexibility is good too: you should still enable it.
In the end, when you buy hardware, most of the time you use the default feature set of the OS, nothing else. Web apps just use a sluggish, unreliable networking protocol which was designed to view static webpages.
So to sum up, people buy hardware to use facebook and twitter, listen to music, play a game, and that's what it is, a fancy, expensive gameboy color with chat. Except the hardware is 1000 or 100000 time faster and batteries last just as long.
I'm not making the case of solving a problem here, I'm just talking about the absence of improvement.