How will memristors change everything?
highscalability.com
highscalability.com
This is not a theoretical failure mode. It has happened before in the computer industry. Multiple times.
For example it is why Transmeta died. Their goal was to have a simple chip that was so fast that they could emulate the x86 faster than the x86 could run. They failed. However one of the design goals was less heat (because heat was a major scaling barrier), which translated into having an emulated x86 chip that with much lower power. Given that they had a simpler architecture and had already solved heat problems that were killing Intel and AMD they hoped to iterate faster, and eventually win. But the investment asymmetry was so huge that they couldn't execute on the plan. And Intel was able to reduce their power enough to undercut Transmeta on the niche they had found, and Transmeta couldn't survive. (Intel was aggressive because they understood the strategy and the potential. Transmeta was always going to be something that either wiped out the existing industry or died with a whimper, and Intel knew which outcome they'd prefer.)
I'm a bit perplexed about the timeframe, though. I'd like to think 5 years, but my gut says it's more like 12-15 years before it's all different. And it will be very different.
Even after adoption, there will be major friction from the corporate and governmental sectors -- lots of money has been invested in doing things the old way. Early adopter consumers in 12-15 years. Rest of the world? Perhaps a good bit longer.
http://www.youtube.com/watch?v=bKGhvKyjgLY#
It has some tidbits I didn't expect. Teaser: "one of the guys in my group has...built a compiler to compile C code using implication logic rather than NAND logic and, interestingly enough, when we play with that the compiled code we get is always more condensed...by about a factor of 3"
If these truly are a viable storage system, building an interface that's reasonably easily adaptable to the current computing paradigm shouldn't be too difficult. Traditional HDDs and SDDs both play nice with SATA, for instance, despite using wildly disparate methods of storing bits.
That part is mind-blowing.
And I'm wondering, if this all works out, will the whole multi-core thing and all the trouble that comes with it (from a software standpoint) be pushed back for another decade?
And what implications would that have for programming languages? Seems like it would mean JavaScript wouldn't be so much worse than Erlang after all (cf. http://news.ycombinator.com/item?id=1304599 ).
(Yeah, I know, it's a little bit of a stretch.)
But Javascript as it stands today is even worse off, as are most languages. We don't really have much that could cope with this right now in a clean way. (Anyone know a language that really handles NUMA well? And I do mean a language, not a library for C or something. Something slick, not something that merely "permits" working with it.)
Depending on what you mean by well, any language that offers fork() as a primitive does work. The trick is that this allows you to create 2 copies of a process that share little enough that a scheduler can safely move them from one machine to another. By contrast with threading you have the problem that the scheduler cannot know when it is safe to schedule two threads on distant CPUs.
Of course this only lets you scale embarrassingly parallel problems.
For another approach that could be made to work reasonably well, try Go. Its central idea is that you pass messages to the processing job, and not vice versa. Figuring out how to schedule threads is a complex task, but run-time scheduling heuristics should do a reasonable job of that for most problems. (Writing algorithms that reliably avoid having bottlenecks will be an interesting challenge.)
And a third approach to watch is Parallel Haskell. Because of the guarantees it offers, the opportunities to rewrite the program at run-time based on what is appropriate are extremely interesting.
http://scholar.google.com/scholar?cluster=140343302019350181...
But I wonder whether it will be worth the trouble (at least to the majority) if you can basically get away with only worrying about one processor, and that one processor is no longer limited in its clock rate by the current leaking, massively heat dissipating MOSFETs we have today.
It's a collection of nodes, any of which can be either CPU transistors, bitkeepers, or both, at any given time - and change with the problem space.
While we may be able to target it with Erlang or JavaScript or whathaveyou eventually, it's going to need a completely different kind of instruction set than what any CPU currently has.
It sounds like version 0.1 of computronium.
Nevertheless, if and when this happens, it's going to be FUCKING AWESOME.
http://groups.google.co.uk/group/fca-t
Spread the word (I was kinda waiting until I had some more content before I publicised, but this seemed like a good opportunity).
And what implications would that have for programming languages?
Now these differences in representation and manipulation are large, as you don't need to 'get' and 'store' data from the memory to the CPU anymore, as the memory is the CPU. Calculations could basically be implemented as moving data from one part of the memory to another.
Memristors would implement physically what FPGAs could technically emulate using transistors. With this come the benefits of lower power consumption and the ability to keep state when powered down (which FPGAs can't do).
I think cellular automata like the Game of Life are a better mental model for memristor circuits (but I am probably hugely wrong here).
CPU with data is a bit like objects - except asynchronous, with true message passing. Like smalltalk or Erlang (or web services for that matter).
Brain images, with parts lighting up depending on how active they are, suggests that much of our brains aren't being used most of the time, but come online as needed. It's as if one part calls another part, except the "call" doesn't block. I'm not being very coherent here.
I find Kilometer much easier to say the American way "kill-ah-meter", rather than the correct way of "kill-Oh-meter". I suspect people will pronounce these things in a way that isn't painful to the tongue, and memri-stor is that way.
It's so hard to guess what this means but I wonder if I should start writing a memristor VM just to see what could been done.
Even if they are here in 5 years I bet i'll bet longer than that before we really, truly know what to do with them.
Also, in the 5+ years it takes to make them practical and reliable, transistors will make progress too. So it's unfair to compare the theoretical density of a new technology to the achieved density of an existing technology.
I'll dig around a bit to see if I can find the link.
The data, which is usually huge in comparison to the code, moves around slowly and transparently. The programming goes to the data. I'm willing to be the next step would be embedded memristor blocks in a human body, providing multiple PetaBytes of both storage and processing capability (and perhaps with some sort of primitive neural interface)
I think that's more than just a little while off, that sort of enhancement, if it ever comes to pass.
Ok, finally found it, posted it here: http://news.ycombinator.com/item?id=1322135
The question becomes how these embedded processors will change over time. I don't think it's too unreasonable to imagine the memristor tech being used and adoption rates going up. The neural interface, of course, was extremely speculative. But the rest of it seems trivial. At least to me.
Should be easy enough to do the math.
I would assume some sort of bio-friendly heat sink, perhaps a silver lace spread through the abdomen or something.
Fortunately getting rid of heat is already well understood.
Given the sugar-cube reference earlier, and that nothing's 2D, much less tons of stacked circuits, I assume they mean one cubic centimeter. Taking that assumption, I point out this problem with that storage claim:
Heat. Good luck dissipating that little cube. Heat's one of the biggest reasons we don't have way higher power machines now, you just can't continually stack things together.
* goes back to reading * very interesting article, though. I'll have to watch the video too. Homework first, though :|
Also, memristors are not stacked, they are deposited on a surface a layer at a time. This is a meaningful difference -- stacked chips are much harder to manufacture, and have much worse thermal conductivity than what is essentially a solid chunk of TiO2.
I have known about the memristor from the beginning--before all the "hype"--as I've interned at HP Labs and I drool at the thought of their vision for intelligent systems with in-memory processing. I mentioned this in another post, but I predict the memristors will make computer vision possible. This means autonomous vehicles, better airport security, etc. Similarly, anything involving sensors and machine learning, will lead to unimaginable progress.
Nanotech is real folks. The question isn't "if?", it's "when?" It will go through many iterations, but it's now real.
(1 petabit = 178TB)
However it should be noted that there is confusion floating around about those terms. The problem is that 2^10 is approximately 1000. Therefore sometimes people use factors of 1000 and other times people use 1024 in very similar contexts. This can lead to confusion and odd ratios.
What looks like it happened in this case is that someone consistently used factors of 1024 rather than 1000, found that a petabit is 128 terabytes (which technically should be called tebibytes), and then miscopied 128 as 178.
Nice recap of details I had read a bit of a few years ago... also, highscalability is a great blog that is likely of interest to many HNers.