Moore's law hits the roof
agner.org
agner.org
The big one is heat. As densities go up, so do power consumption and heat dissipation for active cells. Inactive cells are OK, which is why the highest density devices are now flash memories. That doesn't help CPUs.
The next is fab cost. As someone else pointed out, state of the art fabs cost billions now, and there are fewer ones with each generation.
Finally, of course, sizes are getting close to the atomic level.
Secondary limits include electromigration (the atoms won't stay where you put them as they're pulled on by electric fields) and even X-rays not being short enough for mask etching.
Heat is the biggest problem. CPUs aren't that big; you can fit a thousand in a shoebox if you can cool them. The Bitcoin ASIC business is a worst case. Those devices need no external memory and very little I/O, but most of the gates toggle on every cycle. Mining farms are now located in cold climates near cheap power stations.
I worked in many companies that would consider stuff like Java too "enterprisey".
In 2006 most employers said "There is an abundance of PHP coders, why should I switch to something more expensive?!"
Now they say this about JavaScript.
And devs think "Well, lets learn something to make money with. Since there are some much JavaScript jobs, lets learn it!"
I did so myself and made good money.
Instead, we should work on improved languages that provide better compromises between performance and convenience. Rust, for example, is a good step in this direction.
As dies get bigger, there's no law of nature saying that CPU power must be concentrated far from memory.
Actually, there is. The main reason why the L1 cache is so small, is that a larger cache would take more cycles to access. The L2 cache is allowed to have a higher latency, so the electrical signals have more time to propagate through a physically larger area.
3D stacked dies alleviate this problem, and the trend goes towards putting several dies side-by-side in a single package (a chip), enabling lower latencies, higher throughput, and less power consumption.
There's natural law that mandates the CPU power to be concentrated.
Unluckily this is not so easy: If you know a way, Yossi Kreinin would be really interested to know:
> http://yosefk.com/blog/the-high-level-cpu-challenge.html
And here is a list of ideas/problems that he identified within "the usual suspects":
While doing this, I was fumbling around from tutorial to tutorial, figuring out what's what (I still mis-type "backbone" and "bootstrap"). Reading through the questions that others are asking on stackoverflow, I saw a recurrent pattern. Someone would ask "How do I X?" Someone would answer "You do this, this, then that," explaining fairly-articulately the correct answer. A reply to that would be "I don't get it. Can you post a fiddle?" Sometimes inside of those, you'd see (what I think) is a mis-use of a framework. It would work, but either dragged an entire framework into the picture, or caused a tremendous amount of work to happen, when there would usually be a much simpler solution (i.e. do I -always- have to use jQuery?).
I feel like the entry level of programmer might need to change into a different title. "Software tradesman?" I'd describe these as people that know how to assemble a list of components into a working whole. They can replicate a pattern, filling in blanks where appropriate, but can't nip / tuck to change it, and certainly can't design and create a whole -new- component.
Dealing with the DOM isn't nearly has hard as it has/had been in the past, but it's still a bit more complex than jQuery's API... You can start with document.querySelector*, but then if you need an array, you need Array.from the results, then you can each/map/reduce until your heart's content... event binding is still quite a bit more verbose as well. Not to mention unbinding/cleanup when adding/removing nodes.
In the end, half the time I check "is jQuery in the project already" then I'll use it. More often than not, it is... that is except in "enterprise" projects where "enterprise" architects and the minions of developers have managed to introduce 3-4 different copies of varying old/ancient versions of jQuery and no idea which one is actually loaded. Because it's "just the front end" and have no concept/concern for actual load/performance... don't think about capturing state properly, and get weird behavioral patterns as a result.
Sorry for the rant, it just irks me that people are so dismissive of the front end or capability that the web offers because much of it is so easy to get results. On another note, if you get more into it, do yourself a favor and avoid Angular.
Taking my own learning as an example, I want to build a single-page application, so I decided that I needed a framework to manage it, since that's what all the cool kids were doing. While sipping on a frosty beverage, I was comparing Backbone and Angular, and then realized that Parse (which I'd already decided to use) built their JS API on top of backbone. That quickly simplified things for me, but I still see people that are trying to bolt together angular and parse.
Universities have always been under fire to provide courses "better suited" for market needs and it seems to me that they've been slowly caving in during the last decade or so. This, of course, probably varies a lot between country and even between universities in the same country.
The fact is: university courses that degenerate into professional training courses end up providing little value. Timeless CS principles are abandoned for marketable high-level skills that are obsolete even before students finish their studies, and that help nothing when the market itself changes (or needs to change).
I see Europe's Bologna process as a perfect example of this. It was supposed to make higher education comparable between EU countries but instead now provides an excuse for shorter and shallower courses, comparable only because they don't go any further than the lowest common denominator.
What you say is really evident in some circles: a magical view of computing platforms and a complete ignorance of hardware details critical for software performance. There is a blind faith on compilers/interpreters to somehow extract maximum performance from the hardware in the face of wastefulness.
As anecdotal evidence, I've more than once had to convince people that allocating new objects by the millions (in the JVM) has a real CPU performance cost. The usual response is that the application isn't limited by memory and the time required by the garbage collector is negligible, thinking nothing about cache locality and how memory access is now a major bottleneck for modern CPUs.
And Bologna helps a lot with your education, by making higher education comparable between EU countries, you can easily apply for student exchange programs and study subjects that aren't approached by your home university.
I studied in one of the best engineering university of Portugal and I studied for one semester in Sweden and although both universities were under the Bologna agreement, I could notice differences in both systems.
The only thing that Bologna imposes in universities is how many hours of effort is 1 course credit (ECTS). Course content, organization and etc, is up to each university.
This, by itself, isn't a bad thing.
However, the market perception is that these 3 year courses give the same level of qualification as the previous 5 year courses. It doesn't help that the name of the degree has been made the same on purpose ("licenciatura"). "They just removed useless stuff," they say.
This feeds back into students as "no need to go any further." Once they have reached the degree that the market recognizes (the "licenciatura") there isn't much incentive to continue for another 2 years to get a masters degree (btw, in the previous system a masters degree was 7 years).
Universities have to handle this somehow, and they do it by actually "removing useless stuff" from the 3 year curriculum thus reinforcing market perception and actually putting themselves on course to become training schools instead of universities.
These arguments are pretty specific to the portuguese case, and elsewhere things are bound to have been different but with the same general outcome. The general argument is this: solving the comparability problem between EU countries went too far and breached the universities' ability to resist outside influence (mainly political).
PS: The ECTS system is deeply flawed exactly because it is based on "hours of effort." Not only bacause that's a subjective measure on its own, but mainly because "effort" does not directly correlate to actual learning. A course may have very work-intensive assignments for very little knowledge.
That said, had a similar issue with logic in a simulation software where each virtual unit was its' own OO based object, with it's own threads for events... could only run about 4 simultaneous simulations on a server, and it still would bottleneck (GC was horrible). change it to a single event-loop that signaled each unit, and most of the concurrency issues went away. Streamlined the message passing (more functional approach) and memory usage dropped a lot... couldn't convince the manager to switch away from an SQL backend, which was the final bottleneck (normalized data structures), as it was then fast enough.
Had another instance where a configuration table from a database was loaded into memory (for performance), but then was kept in memory as a DataTable, with text based queries for each access for each key, that happened over a thousand times in a single web request (logins were taking too long)... Changed it to a HashTable, and lo and behold, concurrency was no longer a problem... similar issues with not understanding how static variables worked in a multithreaded application... OO ftw again (sarcasm).
Of course not. People should have the choice on how and what they want to learn. I just disagree when choice is turned into equivalence (i.e. formal education providing no value over self-learning / disguising knowledge to build bird houses as knowledge to build skyscrapers).
In some cases (some people) it may not actually matter, but in the general case, looks like it does.
But isn't that why we're supposed to have "Junior" and "Senior" level positions? I think that what software development really needs is a loosely structured guild system where you gain rank not by attrition or seniority but by reputation, where you gain/lose reputation based on who you back for seniority and how well they do. It seems that beyond CS education, it's a constant learning process. You cannot make a career only with what you learn getting a degree in CS.
It's true that a degree in CS doesn't make a career, and that it's mostly a constant learning process. After a while the difference between having a degree or not having one is difficult to measure and junior/senior levels become defined by experience. But everything else being equal, the person with the degree will have forgotten a lot that the person without the degree wasn't exposed to in the first place. Many people will be exceptions to this, but I'm just talking about the general rule here.
Once you enter an organization, you build reputation there. This is the only thing that matters. To transfer this reputation between organizations, the organizations themselves must recognize the transfer (a form of trust). When you have long and convoluted interview processes, it means there is no such trust. The organization you are trying to join is basically starting from scratch in evaluating you skills, disregarding infomation that says that even if you can't balance a binary tree now, you once did and so can do it again if required.
University degrees are supposed to provide this information. They provide knowledge but also provide a path that is known to be difficult and with a known level of "reputation" once completed. Experience alone cannot possibly provide this, because work at company A says litte to company B if the inner workings of A is an unknown quantity to B.
This is the theory at least. If universities aren't doing they work properly, this whole system collapses.
One of the reasons why I prefer to use "cloud" tools instead of LibreOffice is LibreOffice being slow...
Xcode on the MacMini I use to work is just terrible, it is SLOOOOOOOOOOOOW.
I wish I could code for iPhone with TurboC, the thing just worked, no loading time, if I hit a button, it do what I want instantly.
Games are also getting notoriously bad, I've been seeing 2D games, with no particularly important special effects, get extremely slow on my computer (that can run some very beautiful older 3D games easily), Starbound is a very bad offender at this.
But the one that pissed me off, is Epic Games launcher... Shadow Complex is free in december for PC, to get it you install Epic Games launcher, that then downloads and installs it.
My gaming machine is old, and has some parts of it failing, and playing shadow complex tipped it over some edge and now it is overheating easily... at least I thought it was shadow complex fault, I eventually learned it was the launcher's fault, if I close everything and leave only the Epic Launcher open, it uses 100% of my GPU, and make its temperature shoot from 56C to 92C, it is just absurd... it does not even have any 3D on it, I have no idea why it uses the GPU, and why it needs that much GPU power!
That was the past. This is the future. Always improving...
I haven't used turboc in over 20 years, but I don't recall it doing things like code completion and refactoring and all the other stuff one expects from a modern ide. (this is not to defend xcode)
You would think that with our recorded history, people would be smart enough to shut up about the "limitations of the future". There is not a single person alive today who knows what technology will look like in 10 years, let alone 100 or 1000+. All it takes is a single new discovery from a single person to completely change our understanding of what is possible with technology - or any other aspect of our lives.
The blatant arrogance with which people pretend to know what the future holds is infuriating. Why do so many find it so difficult to admit that we know an infinitesimal portion of total knowledge of the universe? We only recently discovered electricity. Computing is still in its infancy. A computer from the future - if they are even called that anymore - may be directly comparable to what we have today. Or the technology may change so much that the two devices are not even recognizable as being related.
People need to stop telling me that it's impossible for technology to scale as much in the next 30 years as it has in the past 30. What looks like a technological ceiling today could evaporate at any time.
Why does the human race insist on believing that the knowledge we have gained so far is "all there is to know", until proven otherwise? It strikes me that one has to be quite literally an imbecile to not understand that the future will unfurl much that we cannot even fathom today. This insistence that the world of computing 50 years from now must somehow be built on the shoulders of existing technology... it's absurd to be that closed-minded.
So I might not know what the future is like, but we can surely make lots of scenarios highly unlikely.
Agreed, nobody knows what's going to happen in the future, however...
>Computing is still in its infancy. A computer from the future - if they are even called that anymore - may be directly comparable to what we have today. Or the technology may change so much that the two devices are not even recognizable as being related.
... you should remember that 'nobody' also includes you. For computers to keep getting faster we'd need something faster than a MOSFET to build our computers on.
The fact that it hasn't been discovered in 90 years doesn't mean it's not going to be discovered ever, but it also doesn't mean it's bound to be discovered soon because of the unstoppable kurtzweilian march of progress.
Since we can't predict future discoveries because of the law of total expectations the best we can do is try to describe what the future looks like from here.
Also, people aren't going to start valuing craftmanship because you added more constraints. Adding a speed constraint won't make the pile of hacks upon hacks smaller, it will add a layer of speed hacks. It won't fundamentally rewrite the incentives for software, where companies that do the minimum amount of work to hit an acceptable speed are rewarded. It just culls companies naive enough to think they could write something elegant or concise.
I see that as good.
As someone who gets paid for writing software....
That sucks for business types, but this has a real upside for professional programmers. Despite propaganda, salaries are based on supply/demand, not impact.
Look how tiny the source code to Deepmind's general Atari player. It's a completely different model from traditional development:
The smartphone is the first universal tech for mankind, and CPU is not the bottleneck there, it's screens & radios, all batteries being equal of course.
That helps smartphones and electric cars.
For a given modulation scheme, the power required to reliably transmit one bit a specific distance is effectively fixed by the laws of physics.
Given that, earlier this year, Intel plunked down 16 billion to buy #2 FPGA maker Altera, this might turn out to be the most interesting improvement.
Purchasing a company is not necessarily "losing money" unless no synergy is possible between core competences. But in fact, Altera/Intel's respective core competences suggest programmable logic devices next to high-speed multi-core CPUs may be one way to keep Moore's law going (See http://www.eejournal.com/archives/articles/20140624-intel/ for example).
Many of the scientific supercomputing applications are migrating to GPU workloads with OpenCL and CUDA. One enormous benefit of making this migration is the extremely fast memory that's available on GPUs.
I've been waiting for this for 5 years, I don't know why it hasn't happened already.
- Optimized clock speeds, layouts, improved connectivity (i.e. increased number of RAM channels)
-- Reduced latency
-- Increased bandwidth
- Would there even be a need for an MMU?
-- CPUs wouldn't need to work with a variety of RAM types
-- i.e. Less complexity, fewer points of failure
The only cons:
- RAM size would not be configurable
Meanwhile, for better or worse, the majority of consumers can't re-configure their own RAM (Laptops/Smartphones), or on a desktop they don't know how. Businesses would either buy beefy hardware that suited their constraints or spend money on custom configurations. So this con only really effects hobbyists.
In a small way, this has already happened, though. The intel Iris Pro has 1.5 GB of basically 4th level cache, shared between the GPU and the CPU.
I could see memory on the package, though, which might generate some of the benefits you're looking for.
Given that RAM is relatively simple, I wouldn't have expected yields to be such an issue. Also, since the marginal cost of a CPU is relatively low, I figured again, it wouldn't be a large hit to Intel's bottom line to overcome the yields or to give out new chips through warranty.
Remember that for buyers, RAM and CPUs are marked up significantly. A combined CPU RAM unit, would be justified to have increased price over just a CPU. As long as CPU-RAM Unit cost <= CPU + RAM, buyers wouldn't mind and society would end up better as a whole. Potentially making it worthwhile for Intel, regardless of loss in yield.
I am not very educated on the subject though, so I'll defer to your knowledge on it, and agree.
You mean 128 MB?
Now, even if we take some specialized tasks that need more CPU (but are niche anyway):
- For image processing, we had Photoshop doing wonders since at least 2000. And with SSDs, we can view and edit 30+ MP images today faster than we could 8K images back in 2000 (and on 4K displays nowadays). We can very much cover all things web and print just fine.
- For video processing, a fast latest box can edit several streams of 4K with live filters. That's beyond what $200,000 professional boxes could do in the early nineties.
- For audio processing, a MacBook Pro i7 can run more audio channels and effects than what the consoles used for Thriller or The Wall had in million dollar professional studios.
We also need to understand the concept of diminishing returns. E.g. It's not like having an NLE rendering time down from 2 hours to 1 minute will enable us to make more movies. It will just save some editing studios and amateurs some time. Other processes involved in movie making (like just setting the lights appropriately for a shoot) trump this "optimization" many times over.
So it's not like people "need" faster processors. It's more like they would be "nice to have".
Now, for things like HPC for research etc, could indeed use faster processors -- but that's an even more niche market.
Don't you think all these machine learning oriented start-up would like more CPU? How about big data platform?
It's very narrow minded to think that the only market is people using their laptop in their home office.
I don't think society would have any serious issues, in fact we might just be better off, without more efficient HFT and big data.
In fact, I'd ask for more regulated and slower-frequency trading, and smarter (rather than "big") data. And especially when it comes to surveillance and consumer behavior analysis (for ads, etc.), the less data those scumbags get, the better.
I've found the opposite. The faster computers get, the more games are all about BS graphics and nice explosions. Back in my day we had real interesting adventure games, now get off my lawn.
You can definitely find examples of earlier voxel renderers that are also more efficient than Minecraft(e.g. Ken Silverman's VOXLAP), but memory limits and networking gaps would remain real and intractable, limiting its potential popularity.
Those don't have much to do with Moore's law though.
Tell that to Google, Microsoft, Amazon, Rackspace and others. HPC powers a large swath of the internet these days.
I read this wondering why the CPU prices aren't dropping - my i5 costs as much today as it did exactly 4 years ago, when I bought it - and I'm sad about this, because I could use a faster CPU for Kerbal Space Program, but I'm not going to pay third of a new computer's worth for a marginal improvement in clock speed.
It's convenient that multicore utilization is a scale problem at the same that utilizing networked compute resources (ie in the cloud) is scale problem resources because we might be able to treat them as similar problems. Yes the network introduces some other unique qualities (https://en.wikipedia.org/wiki/Fallacies_of_distributed_compu...) but apart from reliability assertions it's an interesting observation that they can be treated similarly if message passing mechanics are abstracted. eg they both fit asynchronous, event driven paradigms. both need elimination of shared state to not make errors in your code (required over the network).
Multicore concurrency in code isn't too hard but threads/locking are probably the wrong abstraction. Similarly, functional approaches might be a better "default" approach for a concurrent world.
That said, the fastest computers over the past 5 years or so aren't much faster, but the power usage is much less... For example, the i5-6200u is an absolute beast of a machine for its' power consumption. This is great for server-side computing, where virtualization and other isolation strategies for allowing shared resources work well.
I don't know that we really need too much more in terms of "faster" ... the problems to be solved at this point are absolutely in systems solutions that are distributed and those communication channels. Google can deliver results of a query amazingly fast... We're going to reach a point where communication networks reach nearly every inch of livable space, and all information is (virtually) within arms length.
(divergent rambling ahead)
There are bigger issues at hand... that being physical transportation... we (in the U.S. and many other places) have more pavement for cars that we do housing space. There are optimizations to be had in the physical world that are far more pressing than if computing can get another 4-5 cycles out of moore's law. The costs of food, medication are growing kind of insane the past few years. We're pushing for younger education to the point that little kids are washing out and doing worse, instead of being little kids. On the flip side, they're so isolated that they can't handle genuine pressure/stress later in life. It's a messed up world.
After using it for years, I can say it does work, but only with a high complexity overhead. Failure conditions, retry logic, serialization and delivery guarantrees are a concern of multi-host parallelism, but not really multicore (same process) parallelism.
I'm not sure I've ever seen an abstraction over memory-separated concurrency that doesn't add a lot of complexity. It may be that such a difference is fundamental and isn't meant to be abstracted over.
In any case, unless you are sure that you want to design for distributed parallelism, I recommend conventional thread pools, etc.
Also, https://www.chrisstucchio.com/blog/2013/actors_vs_futures.ht...
If the Erlang syntax is too weird, Elixir allows you to use ruby-like syntax on the Erlang VM. It also comes with an absolutely amazingly powerful lisp-like macro system.
Yeah.. seems overly specific and fatalistic.
Each time you introduce a new processor node, the cost of the fab goes up dramatically, and now we're talking about $14 billion for the next Samsung fab. (This is Rock's Law.)
If you go back a few years, there were a couple of dozen chip manufacturers around the leading edge, including Toshiba, Siemens, Sony, Fujitsu, NEC and Philips.
With each new step, one or two of these companies can't find or justify the money to build a very expensive fab for the next process, so they drop out.
Today, we're down to four or five: Intel, TSMC, Global Foundries (ex AMD+IBM), Samsung and maybe Europe's STM. So Toshiba, Siemens, Sony, Fujitsu, NEC, Philips and several others have all dropped out.
Which of the four or five leading companies have got both the money and the need to keep pushing on?
Looks to me like Intel and Samsung.
The only other option I can see is Apple putting a ton of money into TSMC, which current makes most of its money fabricating chips for mid-market companies like Mediatek. That's a possibility because it would stop Apple from being too reliant on Samsung.
Rock's law https://en.wikipedia.org/wiki/Rock's_law
On the other hand, having pots of money doesn't actually mean a company can do anything significant in a particular area. Look at all the knock-offs in the Google Graveyard for examples.
Since there's more than just silicon to a chip, those features can often be of metallic atoms, which can be smaller than Si, and hence more than 20 atoms wide.
If you mean innovation in specialized hardware, it's just too early. Those last iterations of fab improvement threw the prices over the roof, everything is expensive now. If Moore's law is really over, things will get cheap as they get stale, but not now.
If you mean innovation on fabrication, well, there's still some fat to cut on power consumption, and if one comes out with an absolutely flawless process, higher frequencies are still possible. But overall, one can not innovate against the laws of Nature.
The next tech boom may be in biology driven by cheap sequencing and its data.
graphene http://www.extremetech.com/extreme/175727-ibm-builds-graphen...
spintronics: http://www.technologyreview.com/view/428883/a-spintronics-br...
photonics: http://www.gizmag.com/photonic-quantum-computer-chip/38928/
memristors: https://en.wikipedia.org/wiki/Memristor
There's no theoretical breakthrough needed to use these technologies, it's simply a matter of moving them from the lab to mass production. So far there has been little incentive as companies are still squeezing out the last bits of juice from silicon, but as we start hitting the limits it's only a matter of time until these technologies get phased in.
It would be awesome!