Are Computers Still Getting Faster? [video]
youtube.com
youtube.com
Approximate cost per GFLOPS
Date | 2013 US Dollars
1961 | $8.3 trillion
1984 | $42,780,000
1997 | $42,000
2000 | $1,300
2003 | $100
2007 | $52
2011 | $1.80
June 2013 | $0.22
November 2013 | $0.16
December 2013 | $0.12
January 2015 | $0.08No underground city required
They did talk about more and more people using a computer as a window to the internet, but didn't talk about how much more efficient the software powering that window had become. Apple launched Safari in 2003 and Chrome/V8 was released in 2008. So even as we've wanted our web apps to do more, the engines running them have gotten so much better.
When most software that people were using was being written in C/C++, an increase in the amount of work the program wanted to do needed to be matched by an increase in performance. Compilers have gotten better, but nothing near the impact that modern web browsers have had. So older computers have benefitted from a huge increase in browser performance (the browser being the thing executing most "applications") over the past decade in a way that we never really saw in previous computing generations.
This has largely been offset by the fact that we're sending more and more data over the wire. The average web site is now 2.1MB in size.[0]
[0] http://money.cnn.com/2015/06/16/technology/web-slow-big/
s/site/page
Older hardware definitely often pays a price for supporting newer features.
Meanwhile chrome on that same machine runs great, albeit eating enough memory that switching applications is noticeable.
Trying to study rates of growth on a linear scale is basically impossible during exponential-ish growth periods. All but the rightmost end of the chart gets squished flat along the horizontal axis.
Similarly, charts of e.g. currency exchange rates, stock indices, comparative economic growth between countries, etc. should most of the time be plotted with a log-scaled vertical axis. Otherwise there’s no way to accurately compare slopes in different parts of the chart, which can be deeply misleading.
It’s in some ways too bad that slide rules have been replaced by electronic calculators. I feel like the general public coming out of high school used to have more fluency with logarithms, which until the last generation or two were the basic tool for all calculation in science and engineering.
Especially for a general audience, but even people familiar with log scales, it doesn't give an intuitive feel of the actual numbers.
I am lamenting that they are not, because for many, many purposes they are easier to read and facilitate more useful intra-chart comparisons than linear-scaled charts. In particular, they make the slope of lines meaningful in many cases where linear-scaled charts do not. They also fit much more useful information about relative magnitude and allow us to read a couple significant digits of numbers across a wide range of scales. Any time there is more than 2 orders of magnitude difference among values a chart, a linear scale becomes nearly useless.
Reading charts at all is an acquired skill, and takes a lot more practice than you might expect. Just like doing arithmetic with fractions, decimals, or angles of a circle is an acquired skill, or driving a car is an acquired skill. If you study young children or people from non-literate cultures, you’ll see all kinds of difficulty reading linear-scaled charts.
Edit: here, I spent a couple minutes making a linear-scaled and a log-scaled chart of the wikipedia $/gigaflops table from elsewhere in this thread, using Matlab. See how much useful information you can get from the linear-scaled chart:
http://i.imgur.com/A80Tr6V.png http://i.imgur.com/xG0mngX.png
(On the log-scaled chart, if we penciled in a grid, you could get at least one significant figure out of each data point, despite the data spanning 11 orders of magnitude. On the linear-scaled chart, you get 2 significant figures for a single data point, but there’s no way to make any distinction at all between the rest of the values, which span about 8 orders of magnitude.)
The video used a zooming chart, which solves both problems.
Sure it does. The number goes from 10^12 down to 10^-1, 13 orders of magnitude. You can easily read these off the side or count the number of grid lines crossed. (Whoops, I wrote 11 in my comment before.) If you’re used to reading a log-scaled chart, counting the powers of ten along the vertical axis is perfectly “intuitive”, or at any rate just as “intuitive” as any other way you could write these.
In distance scale terms, this is the same as going from nanometers to tens of kilometers. Or in time scale terms from milliseconds to hundreds of years.
Numbers with such a big difference in scale are to some extent inherently incomparable: We don’t interact with such a range of scale with most of our naked senses, but need tools to understand and compare such divergent numbers. (Though if you’re hunting for an analogy, comparing volume or mass might be slightly easier, as they scale with length cubed. 13 orders of magnitude gets you from the mass of a human egg cell up to the mass of a tank.)
Later improvements make it faster, easier to use, higher-resolution, better at multitasking etc., but none of that changed the yes/no question of "Will it work?"
We're currently on the cusp of the machine learning paradigm shift. As ML keeps getting better, it will start showing up more and more in consumer interfaces. I'm willing to bet that within a few decades natural language interfaces will become the norm. The desktop paradigm will simply go away at that point.
You'll just tell the computer what you want to do instead of having to juggle windows using the mouse and keyboard. You'll be able to say find that youtube video, play this song, or reply to that email.
The computer will most likely turn into a personal assistant, and these kinds of machines will require a lot more power than what we currently have in desktops.
Your phone/computer's speech recognition is done by your phone/computer recording your voice and sending it to Microsoft/Apple/Google, then Microsoft/Apple/Google transcribing it on THEIR computers, storing your data, and sending the transcription back to your phone/computer.
It doesn't require any processing power on your computer, besides the power to record and send your voice.
You seem to think that because you bought a computer, that means that it's going to be your "personal assistant" unconditionally. It's not. It's going to give your voiceprint, location information, and other personally identifying information to MS/AAPL/GOOG. F/OSS might get similar features but they will lag years behind, and will not have the same impact on society that large corporations with datacenters/server farms can have. Eventually, you'll start seeing friendly suggestions and helpful tips, like your traffic schedule (this is already the case).
You'll receive SUGGESTIONS on where to go. Friendly suggestions like, "Did you mean to go to the strip club/bar? How about going to church instead?"
This could amplify inequality. Imagine Microsoft Clippy saying, "Hi, I noticed that you need $350 for crack rocks. May I suggest Honest Achmed's Pawn Shop only 1.2 miles from your trap house? We compared prices of the Internet of Things enabled devices you have on Amazon and we noticed that your 202X Macbook has a resale value of about $350."
You'll eventually be CONTROLLED by some algorithm somewhere.
There's nothing PERSONAL about the assistance they'll provide.
Sure, however this approach has a lot of downsides, such as utter lack of privacy. As we've seen many times historically the pendulum swings between thin and thick clients.
Right now the computational power needed to do this kind of processing is very expensive so it's done on server. However, faster hardware will make it possible to do it on personal devices as well.
Also worth pointing out that privacy doesn't exist for most people already. The mode of interaction with the computer isn't going to change any of that one way or the other.
Do you think that MS/AAPL/GOOG will go out of its way to collect your user data in particular, when it can get millions or billions of people's user data?
Do you think that MS/AAPL/GOOG doesn't have voice recognition for other languages? Did you look it up at all?
Do you think that MS/AAPL/GOOG won't get voice recognition for other languages in "10 years", given how much personally identifying data they can scoop up?
No, but I also know very few typical users that use voice controls extensively.
"Do you think that MS/AAPL/GOOG doesn't have voice recognition for other languages? Did you look it up at all?"
Voice controls in Lithuanian (my mother tongue) on Android didn't work until very recently (2015 Sep, according to Wiki) and it still works very poorly. If your primary language is not a mainstream one, you are out of luck. I'm sure it will get better, but it's not among the reasons I don't use it anyway (see below).
"Do you think that MS/AAPL/GOOG will go out of its way to collect your user data in particular, when it can get millions or billions of people's user data?"
I'm very pro-privacy, but it's not my main concern when talking about voice controls. The main drawback and reason why I hate it, is that talking breaks concentration and disrupts workflow. Another reason is that I don't like people around hearing what I'm doing, I can imagine a lot of anecdotal situations regarding "Computer, load PornHUB" etc. Also, it annoys other people around -- imagine a shared, dormitory like flat where all people are using voice controls at once.
The primary bottleneck for intelligent agents is not voice recognition, it's semantic insight. They need to have an innate ability to learn the meaning of new things without being programmed to do so. As far as I know siri, cortana and google do not do this. Meaning must be programmed into them explicitly. That doesn't scale. Yes, they can learn new things, but they learn them slowly. You can even build advanced intelligent things like self-driving cars (with great effort) because the required level of understanding about the world is very constrained. However, you cannot scale it up to general purpose assistants because the amount of code required cannot be built in any reasonable amount of time. Until we have algorithms that create algorithms, software rewriting itself and evolving based on higher goals, we won't see the promised star-trek-level personal assistants come to fruition.
It could go either way. Let us hope you are right.
There's a massive jump you're making here, I feel. From where we are to "computer, invent a new novel for me" there's a huge range of useful mid-points.
One level is just understanding more about human-written unstructured data to answer questions.
Another is to be better at sending commands to other humans.
Third there's linking these things up. Understanding where there are gaps in the knowledge, who might be able to fill them in and then asking them the right questions. I know there was work done in this area as part of CoSY back in 2004-2008 http://www.cs.bham.ac.uk/research/projects/cosy/ which was followed with http://www.cs.bham.ac.uk/research/projects/cogx/
This then gets you to the level of essentially free personal assistants for a wide range of general tasks. That would be valuable to a very large number of people.
We're just starting to explore machine learning in any meaningful sense. I think the progress we've seen already bodes tremendously well for AI assistants.
>The primary bottleneck for intelligent agents is not voice recognition, it's semantic insight.
I absolutely agree. We won't see general purpose assistants in the near future, but that doesn't mean we won't see specialized assistants. Something like the desktop is a fairly constrained environment.
A lot of what you'd need to do is already being done by systems like Google search. You can type in a random term and it will often do an excellent job of finding exactly what you wanted.
Here's an example of just how good this tech is now. I forgot a name of a tv show one time and I typed in "movie with a clock and the key" into Google. Sure enough it found the show as the first result.
While we don't know how to make a general purpose AI right now, but there is a ton of work happening in that domain. I would be very surprised if we don't see a lot of progress there in the next few decades.
The AI doesn't have to be self improving, our brains aren't and we can do all the things that a digital assistant would need to do.
They have pretty bad discoverability. Users can only find out what works by a try->fail->try something else approach or by being told that it works like that.
Back in the 80s and early 90s, sound was a huge deal, going from synthetic beeps to full audio playback and you needed to fiddle with a dedicated soundcard to enjoy the latest breakthroughs. Nowadays, hardly anyone bothers because all computers have "good enough" sound out of the box.
I find it quite fascinating to think whether something similar might be possible for, say, graphics cards. It seems impossible now, but imagine a monitor at the max resolution your eye can see and real time 3D graphics with enough polygons to never see an edge and all the basic shading/lighting problems solved to Pixar levels... that might be less than 10 years away.
Related: http://www.forbes.com/sites/adriankingsleyhughes/2012/05/12/...
I'm using my 2011 Macbook Pro for Protools, and it feels just as fast as my 2014 model does for the rather paltry tasks I throw at it (I use outboard hardware for almost everything, mixing 16 tracks of audio was something a G3 Mac could handle easily). I don't see any reason I'll realistically need to upgrade it anytime soon (loaded with memory, SSD, etc)
Not to say there hasn't been massive strides of improvement over the last decade, SSD and GPUs in particular have been spectacular. But many consumers have the decision they'ed oven rather have a power efficient processor than a heavy duty CPU.
As developers it means being conscious of bloat and writing more efficient software. Already we're seeing many libraries that are closer to the hardware or 'metal'.
Most people simply don't need a lot of machine to do what they need on a computer. Where as early on computers were struggling to do things like display text and images the peak of user demands was the point where all computers could run Flash Player 9.
See the "circular argument" argument above.
But when good product comes out things will change, are you going to bet against virtual reality?
Always with new technology has come new ways to consume which required more processing power. Right now it's smartphones, but maybe soon it will be VR/AR.
Currently, a 5 year old computer can easily meet most demands. Today several times as many people that have ever bought a computer are buying smartphones. Yet they are less powerful.
http://www.extremetech.com/wp-content/uploads/2014/09/Dennar...
Since then there has been an explosion in number of cores. Seems like coding for distributed systems is the way to go. I suspect that increasing the transistor count will remain beneficial to things like scientific computing and machine learning.
Nowadays for 250€ you can only buy a i5-6600k which is around 25% faster than the i7-2600k in single core performance, but slightly slower in multi core performance due to lack of HT.
Only the GPU inside these new CPUs has improved greatly.
Cores have gone up, just not in the narrow spectrum of consumer processors since not a lot of software takes advantage of them. Knights Landing will have 72 cores, all 4 way hyper threaded, all out of order cores. GPU's internal cores have gone up as well.
Transistor budgets haven't stalled quite yet but lots of cheap journalism proclaims the end of moore's law because of consumer chip benchmarks. Intel has a roadmap to 7nm which will quadruple the amount of transistors we have now, after that it is unclear.
22 and 24 cores coming to Xeons in 2016: http://www.kitguru.net/components/cpu/anton-shilov/intel-xeo...
That graph shows no such thing... number of cores has flattened out from 2011-2015.
The reason is hardware designers have hit a wall, they cant make single threaded applications run any faster.
So your brand new gaming PC is no faster at running single core applications than your 10 year old computer is. You can copy over any single core application from your new PC to your old one, and it will run just as fast.
What your new computer has is more cores though. Ideally you want linear scaling. That is if you have 2 cores your program should run twice as fast, and if you have 256 it should run 256 times as fast. This kind of scaling is almost impossible to achieve. Most programs don't scale beyond a certain number of cores.
Your multithreaded program might run a bit faster if you throw an extra core at it, but very quickly throwing more cores at it doesn't make a difference. It might even get slower. You have all those extra cores idle, or worse they are busy waiting.
Most programs dont scale well to many cores because any synchronization between cores kill scalability. If you have ANY sequential steps in your application that will put a limit on scalability.
As an example say you have 4 cores. You have a main loop on one core, it posts expensive work to the other (3) cores. And when they are done, you collect all the results and use them somehow. This final step is sequential and it kills your scaling. Your application wont run 4 times faster. And it probably wont run any faster if you throw 16 cores at it. This is because that one core is synchronizing with all the other cores. So those other cores do a lot of waiting around. And any sequential steps in that one core becomes the bottleneck, as those other cores will all end up waiting for it.
This message is already too long. So long story short. Single threaded programs are no faster on new computers, and most multi threaded programs dont scale well to many cores. Its a huge wasted opportunity to let cores be idle or underutilized. We have to fundamentally change our programming style and tools to take make use of all available cores.
More info by Herb Sutter:
There is also some progress - http://www.cpubenchmark.net/singleThread.html
I'm writing this on a Core2Duo E6750 that has a single thread score of 1000 - most new CPUs are at least twice as fast. Also Instructions per cycle going up.
It's really a hard problem. You can't just spawn threads and put mutexes in front of you data structures. This is terribly slow, even slower than single core without locking for some tasks.
We need better lockless data structures that minimize synchronisation.
*overclocked to 4.2
Compared to the improvements we made in the decades before that, that's really poor.
I have a i7-2600k (quad core 3.4Ghz, 95W TDP, HT) that was introduced in 2011. Five years later, it still is not significantly slower than recent CPUs by Intel unless they take advantage of new CPU features. It sold for around 250€ back then. Today, for 250€ you can buy an i5-6600k (quad core 3.5Ghz, 91W TDP, no HT). The only big improvement in the i5-6600k is the GPU (which I don't use).
You can see it in games. Until recently it was equally possible to play on an overclocked Pentium G3258 as on an i5-4690, a way more powerful quadcore processor, because most games just did not use the additional threads the i5 provided. That is changing now, so far that even the Hyperthreading of an i7 gets very useful in games.
If that was true in games, that was probably also true for other software. Meaning that the new processors where not that more powerful for all those software not being able to use the multi-threading capabilities.
Today computing power in personal computers is basically a commodity, with companies now having to focus on other aspects to be able to sell their computers. Namely weight, battery life, network connectivity etc.
Having said that, the chips are still developing, but now the focusing is on the new user demands rather than raw speed. E.g. we now have native decoding of audio, video, networking, low power modes, etc to increase battery life. (And thereby decrease weight)
While the consumer GPU's are also getting faster, it's not due to more advanced use cases, but rather because computers now have screens with higher screen resolutions.
I believe that processing power became a commodity around a single core GeekBench Browser score of about 2500 (Think Late 2011 Macbook pro - note using browser score here to be able to compare apples to apples). Interestingly, this is now pretty much exactly where the newest iPhones are. (Android only just now reaching the 2k's).
In essence, with the latest and greatest chips on mobile, we are reaching the point where processing speed is now a commodity on this platform. However as with PC's we will likely see a bit of overshooting, so expect the mobile CPU race to continue until around a GeekBench Browser score of 3000-3200.
Hopefully this will result in companies starting to compete on battery life of the device as was seen with laptops. E.g. the newest MacBook Pro claims 9 hours of battery, compared to the 4 hours promised for the Late 2011 Macbook Pro.
The only obvious unknown I can think of, is if the average consumer embraces Virtual Reality, rather than it becoming a niche such as high quality PC gaming. Then we might once again see a new strong focus towards raw single core computing power, both on Desktop and Mobile.
And high-end consumer video cards are about x1,000,000,000 ... which is about, or a little less, what one of the Moore's law corollaries predicts for 35 years, 1981 to 2016, per $.
we are getting to a point where other aspects of performance are also getting harder and harder to notice. things like rendering, supported display sizes and refresh rates, the ability for machines to understand us (voice recognition and computer vision)
>The question I'm asking is why does a 10 year old computer basically run current software OK while that wasn't true in the past.
I've got a theory that it's a bit related to human brain IO limits. There's only so much text, sound and video we can take in, video being the highest bandwidth and computers got to a stage where they could do video OK a decade of so ago. So increasing the output dramatically does not have a huge effect on user experience. I mean 4k video is nice but it's a similar experience to 360p.
With unlimited computing power we could render real time highly interactive super accurate simulations which takes months to render. But we can't because we don't have the computing power.
The truth is, we are hitting multiple wall at the same time. It's harder to shrink gates any further, it's harder to increase clock speed anymore and it's harder to dissipate the heat generated by the cpus. All the increase we are now getting are incremental not exponential and that is why we can run a current software on a 10 year old cpu, if the software is not optimized for multicore cpus.
On the other hand if people really wanted performance they could get high powered desk top computers but the trend has been more towards using phones. About the most processor intensive thing I do personally is edit video and even that works OK on my 2 year old phone.
The adoption of mobile phones as computing platform is a very different event. It allows use of computing in a very versatile manner. That is why it's usage exploded.