Moore’s Law Is About to Get Weird (2015)
nautil.us
nautil.us
Ternary logic is really cool, but it only gets you about a 10% efficiency improvement. I think it would be awesome to go in this direction, but retooling our entire tech stack for that isn't going to happen. I mean, look at x86.
The only thing that will really make progress is molecular nanotechnology. I had always believed that reaching the end of silicon process technology would necessitate the research into MNT.
What's funny is that we have arguably already reached the end on the silicon process technology, if Intel's desktop product offerings are used as a guide. There are gamers who are building new rigs using chips that are 2 or 3 generations old, because there isn't really anything better available. What's really hilarious are the guys building systems with used Xeon CPUs from retired servers!
So I don't know what's going on any more. You would think that the profits and stock prices of all the silicon companies would collapse if they weren't able to continue producing improvements on their existing products. Intel is spending massively on R&D, but the gains are nowhere close to what we saw 15 years ago.
Note that keep referencing Intel, because they've been at the leading edge. AMD doesn't have 14nm in production, for example.
Chip design and manufacturing isn't something a garage tinkerer can do (at the level of cutting edge research to compete with existing silicon makers). Therefore, companies like Intel can afford to lax until their customers demand differently (in which case, they will probably put the gas on R&D).
I wonder if today it's easier than people imagine, but the lessons of Itanium are used as a signal to not try.
Sure there are plenty of apps which make low-level use of x86 features, and those probably aren't going to be updated. However a lot of modern software is written in languages that go through interrupters. Java, C#, Python.. if you can update the runtime, in theory the apps can be supported anywhere.
You'd need to spend several years writing a ternary RTL synthesis, place, route and verification suite before starting.
I like the Mill CPU design, and they're saying it will have a 10x power/performance advantage over traditional architectures.
But we're just about done with the times where we had easy gains in price/performance/power by just moving to a smaller process node.
AMD actually just released their first 14 nm products this summer in the form of graphics cards. They don't have any CPUs out in such an advanced process but I think that reflects the architectural cul de sac they've been stuck in. They're next architecture, Zen, is supposed to be 14nm and it doesn't look like it makes business sense to port a Bulldozer derivative to 14nm before Zen is ready.
1. That's not Moore's Law, which is specifically about density.
2. When you aren't getting density increases, your processing power / memory size / whatever increases linearly with die area. We've been used to living in an age with regular process node shrinks, meaning that more and more powerful computers don't end up being more expensive. Sometimes the cost goes down while the performance goes up. We're seeing the end of that right now.
In the end, the party decided that code compatibility with the west was more important than independent innovation, and the soviet computer industry slowly lost it's charm and died in the late 80s.
Never heard of a ternary computer since then.
https://news.ycombinator.com/item?id=12227555
What do you think about that advancement? Think the quantom dot circuits can get practical enough to go anywhere? Or even good potentially for low-volume, high-price applications we see with some S-ASIC's or ASIC's?
However, fluidics was/still is a practical technology for niche applications. Fluidic non-moving part logic devices have been used for for factory process control and integrated pneumatic controllers for aerospace for decades now. Fluidics can be very robust and reliable. In the time period of the 1960s-1970s fluidics was more reliable than similar electrical and electromechanical systems. One thing that still makes fluidics attractive today is its robustness in environments where transistors don't work. Fluidics can with stand high temperatures, high shocks, high radiation, high magnetic fields, etc. As long as the shape of the channel remains intact, fluidics still works. Fluidics has been demonstrated to work inside jet engines, nuclear reactors, and molten steel.
Because of fluidics inherent EMP resistance, fluidics has been considered for use in anti-ballistic missile guidance systems[0].
One interesting current day application of fluidics is kosher sound amplification[1]. Orthodox Judaism prohibits electronic sound amplification on the sabbath because it constitutes work. Using a speaker to produce sound constitutes labor because it creates a sound that did not exist before from electricity. Fluidics is able to get around this prohibition and can amplify sound in the range of human speech.
As far as molecular nanotechnology goes, some of the first molecular computers we may see might resemble the BZ media mentioned in the article. Right now we can make a whole range of interesting molecular devices including logic gates and motors among other things, but it is difficult to put them together into larger devices. It's hard to hook up a molecular motor to a molecular wheel to a molecular suspension to make a molecular car[2], much less wire up a bunch of molecular logic gates in a defined pattern to make a molecular computer.
However, it is significantly easier to make periodic structures. Metal Organic Frameworks(MOFs) offer a way to organize molecular machines into periodic structures. They are pretty great for this because they self-assemble and have a bunch of open space for molecular machines to move around, but still close enough that they can interact with each other.
Some researchers have proposed that we could use MOFs with molecular machines in them to make cellular automata with mole quantities of cells.[3]
[0]http://www.dtic.mil/cgi-bin/GetTRDoc?Location=U2&doc=GetTRDo... [1]http://acoustics.org/pressroom/httpdocs/132nd/2aaa8.html [2]http://www.cemes.fr/Molecule-car-Race?lang=en [3]http://pubs.rsc.org/en/Content/ArticleLanding/2012/CS/c1cs15...
Intel, Samsung and TSMC have all demonstrated their 10nm processes with TSMC expected to be the first to mass production. https://en.wikipedia.org/wiki/10_nanometer
I believe nVidia and AMD use Samsung and TSMC as their advanced fabs which is how they stay alive. They get to keep the expensive factories off their books and the factories can use their capacity to make more than just AMD and nVidia parts keeping their lines full. At least that's the theory...
If we're going to see any developments in alternative architectures and hardware, it's going to happen in the cloud computing space. From my perspective, this is the only sector that is in the unique position to sell specialized computing services to customers on a mass scale and make great margins in the process.
There are many interesting and useful things you can do outside of Von Neumann architectures and Turing Complete systems that were never practical on PCs. Simple services like Reddis could probably be implemented with alternative memory architectures like content-addressable memory. Many useful algorithms in the data analytic and machine learning space can be compiled down the specialized hardware using using a combination of transistors and memristors.
For companies like Amazon, Microsoft + Google, there's actually a financial incentive to harmonize hardware and software much in the same way Seymour Cray harmonized the software and hardware of Cray's supercomputers. These cloud companies don't want to be constrained to selling commodity virtual machines. They want to sell you the next generation PaaS solution like DynamoDB, SQL Azure or Firebase so they can lock you into their cloud platform.
If there's a future where developers are writing code in special DSLs that's compiled to gates on an FPGA, the cloud computing guys will be exploring it because they have a huge incentive to get their customer's into the walled garden that yields better margins. If you can create one platform service that developers love and runs on cheaper specialized hardware, you'll be able to destroy your competitors and signal an alternative hardware arms race.
There was also a (sadly failed) initiative about ten years back in Germany to build a custom ASIC for multigrid Poisson equation solvers. I'm sure there are more examples outside of my interests.
Big cloud computing players will almost certainly move this way. Actually surprised we haven't seen more of this already. But TensorFlow and the Tensor Processing Unit were smart moves by Google and I think this is just a preview of many, many things to come.
I think IBM probably belongs on the list of top contenders to make a move. Between SoftLayer, IBM's hardware and chip design and fab partnerships, their cloud/saas businesses, and their Enterprise penetration, they're well-positioned to lead here.
I think that you're right as well that all these folks will push to create developer-friendly services and APIs backed by hardware. However, I think the real wins will come from capturing market share in the Enterprise sooner than later. If I were them and I had to prioritize between the two, I'd target Enterprise customers first and foremost. You can sell services to help them migrate and that helps you get out to market more quickly than creating a fully robust self-serve platform. Moreover, if your custom hw-backed platform can offer either better price or better performance, Enterprises will pay big bucks for it. Harder to get it right with self-serve first, but you certainly need to open up to self-serve as quickly as possible over time.
I think this was what Google is trying with their TPU for running TensorFlow for machine learning applications.
https://cloudplatform.googleblog.com/2016/05/Google-supercha...
> "...Moore’s Law, which states that the number of transistors that can be squeezed onto a semiconductor chip of a given size doubles roughly every two years, has held true since the mid 1960s..."
I thought it was the case that Moore's "Law" hasn't held up since ~2012.
I believe Intel and others have stated that as of 2012 or so, with 22nm processes, transistor density now doubles every ~2.5 years and is likely to slow further within 5 years without more fundamental breakthroughs.
The related trend that processor clock speeds doubled every 1.5-2 years-- which is commonly mistaken as Moore's Law in lay press but is actually more related to Dennard Scaling, Koomey's Law and historical statements by Intel-- had ceased to hold as of the late Aughts as well, I believe.
Memsistors have been a promise for some time. I wonder how we will make the leap from current dense-transistor IC tech and processes to anything new, given the huge infrastructure investments in fabs, quality control processes, etc. It seems it will be difficult to scale out a new technology, whatever it is, and get it price-competitive with existing transistor-dense IC technology. Perhaps I misunderstand here though, and would be interested in thoughts from those with greater expertise in the field.
I thought it was before that, like 1999. I pretty sure I had an Intel Pentium III 2 GHz processor in 99. The computer I am using to type this comment is 2.5 GHz. More's law is long dead, we aren't going to see 12 GHz machines in the next 6 years.
https://en.wikipedia.org/wiki/Dennard_scaling#Breakdown_of_D...
Moore's Law is a limit that ultimately arises from economics. The costs to keep Moore's Law going are exponential (just like the gain), and fewer and fewer companies are willing to pay the increasingly high price.
I was always primed to believe the exponential gains were essentially "free" and came with a fixed amount of investment into the industry. Understanding that this isn't true explains why Moore's Law will end unless demand is exponentially increasing.
In the short term I really think we are entering the era of accelerators. Accelerators like Micron's Automata Processor (1) for graph analysis, accelerators for compute intensive applications like Convolutional Neural Nets, continued innovation in DSPs, GPUs, etc.
Accelerators that can either outperform traditional processors or use significantly less power I think represent one of the major next steps after Moore's law. What comes next may well be a slime mold computer but I think at this point is pure speculation.
(1) - http://www.micronautomata.com - actually I heard about this on HN!
I think the interesting question is what happens after that. Not too long from now, specialized systems will run up against the same physical limits that general purpose processors have begun to hit the past few years as well, as specialized fabs increase in capability and sophistication. We see this already with GPUs.
I think it's pretty clear that we're going to see the introduction of all sorts of specialized hardware that we already more or less know how to do but it just hasn't been worth the trouble before. But you're also right that, so long as we're still talking CMOS, this is probably a more or less one-time boost that's maybe worth a few CMOS generations depending on the specific case.
[0] https://www.microsoft.com/en-us/research/project/emips/
[1] http://www.theregister.co.uk/2016/03/14/intel_xeon_fpga/
Once the process is "mature" and isn't changing every few years, the error rate could be squeezed way down. That would make it practical and economical to make larger chips. There are cooling and power issues too but I'm sure those are solvable.
Make way for the 64-bit 128-core 1024gb RAM 12cm^2 SoC?
Call it More's Law? Or a BFC? (for Big F'ing Chip?)
This is already happening?
Intel Xeon Knights Landing already has an enormous 6.83cm^2 die size. It has 72 "cores," each of which has a 512-bit vector processor. In some sense, you could say that this chip has 9,216 "single precision cores." It also supports up to 384GB DDR4 RAM, which is not far from your 1024GB.
The term "core" is becoming vague, as NVIDIA refers to thousands of "CUDA cores" on its GPUs which are really vector processor elements.
China's Sunway SW26010 is also kind of a "SoC," 260 "cores" communicating via a Network-on-Chip interconnect. I think one could say that the Cell Broadband Engine processor had a similar design.
> make it practical and economical to make larger chips
I do wonder if die sizes can increase.
I'm not a hardware engineer, but my understanding is that latency is the main barrier to larger chips. The time it takes for a signal to propagate across a chip limits the clock frequency. One possibility is to replace the global synchronous clock design with an asynchronous chip design. However, from what I understand, this would make things worse because some form of synchronization is always needed - you could send back an "acknolwedge" message, but that would double the signal traffic.
On the other hand, supercomputers are regularly built with multi-socket systems and interconnects like QPI and PCIe between processors and nodes. It would seem logical to put multiple processors their interconnects on a single "SoC" and I don't know why that doesn't happen. Maybe it is because of error rates, as you say.
Only way to improve that is to make programming languages push either data locality or dataflow like designs, which may mean giving up or reimagining OO.
Error rates (yield) are a serious problem, but if your design is uniform enough you can very well get away with just disabling bits. There's also a less obvious problem with on-chip variation (OCV) - some physical properties have a gradient across the chip. At runtime, there's a temperature gradient as well.
https://scholar.google.com/scholar?q=Adamatzky+highways&btnG...
[1] http://spectrum.ieee.org/nanoclast/semiconductors/materials/...
Or even in term of Network connection speed - 100mbps, 1G, 10G, 25G, 40G (at lease in data center where it matters a lot.)
The simple usb connection speed also continue to increase at a fast rate.
Well, no thanks. No, it is not the same.
Intel's 3DXPoint has already brought memristor memory units to market. The beauty of memristor architectures is that they can be used for logic and storage, so you can create some radical new architectures with them.
I'm too lazy to link the talk by Stan Williams of HP, but if you google it it's the ~45min one.
This is the next architecture because it's already here. All that other stuff is vaporware.
It's also completely unclear whether quantum computers would make your Javascript engine run any faster.
A decent explanation can be found here: https://uwaterloo.ca/institute-for-quantum-computing/quantum...