I can't wait to see the programming paradigm of memory vs permanent storage begin to blur in the next 5 or so years. It's going to make some major assumptions about how you program stuff change quite significantly and it's really exciting.
Of course, all of this is pretty marketing heavy and not a lot of details have been released, so take this with a huge grain of salt. HP's CTO wrote a fluff piece on flattening the storage hierarchy here: http://www8.hp.com/hpnext/posts/end-necessary-evil-collapsin...
[1] http://arstechnica.com/information-technology/2014/06/hp-lab...
[2] http://www.businessweek.com/printer/articles/206401-with-the...
Inventors of anything submitted in patent applications should be named on the application. That is a potential route to finding who in a corp actually invented stuff.
(source: I'm a cs professor at carnegie mellon, and I was just chatting with some of those researchers at OSDI.)
Real revolutions in OS tech have for the last decades come from the keyboards of large institutional users and from hordes of smaller ones. Even IBM has pretty much acknowledged that. So we'll see if HP will be able to keep the door locked to someone doing a linux device driver for their new hot hardware or to add a few calls to the kernel that enable an application to use this tech.
The same has happened to infiniband (fast networking for clusters and storage) and tons of other esoteric hardware including shared memory between physically disjoint machines (DSM).
Still they're merging entire sub-systems that have existed independently for decades and someone has to actually make all the changes to support permanent in-place execution of code and the like that's what I mostly got out of the OS porting news.
And e.g. consider the changes to RDBMS's if you can do guaranteed committed writes of individual bytes. A huge amount of the work on RDBMS's have been around table types that are optimised for per-block read/writes, where the design tradeoffs might be very different if you can do "cheap" random access (though caches will probably still justify some degree of block-wise access).
But with respect to operating systems, I agree that it will take a long time to see any dramatic changes. The current mechanisms would still work: Open a file that maps to a memory region instead of a drive; mmap() it. Your app would just want to know whether or not the file content is instantly persistent on write or not.
I think HP's starting point, though, is that we don't really know. Both in terms of hardware and software. Take this DIMM drive, for example. If you look at current server motherboards, a lot of servers has no need for PCIe slots other than for SSDs. A lot of them will not need drive slots at all if you can stuff that much storage on DIMM slots. Suddenly it becomes viable to put really high performance systems in half depth 1U form factors. For a lot of our larger servers, 50% or more of the server volume is air because current designs are so geared towards front loaded hot-swappable drives. Maybe 75% of the weight of the servers are due to drive bays and larger boxes to accommodate that front-loading. That's a lot of wasted metal, and a lot of wasted rack space. And a lot of wasted electronics for SATA interfaces that could sometime become obsolete.
It'll be a huge paradigm shift in storage and IO speed.
It's unlikely that future technological advancements will give malware authors more tools to work with? That seems unlikely, if history is any indication.
Imagine when you have 1TB of non-volatile storage which operates at speeds close to those we currently enjoy for DDR3 memory. Most average consumers won't use all of that space. It seems entirely plausible that a virus will write itself to some area that is never overwritten. It becomes a latent piece of malware, invisible to any filesystem, but activatable on demand by some other hard-to-detect malware. This two-step scheme would be difficult to detect since the second component does nothing except activate the first component (stored in nonvolatile storage) under some specified criteria.
Or, how about key generation or password input? Remember cperciva's post about how it was extremely difficult or even impossible to zero a memory buffer correctly? Now add in the idea that suddenly "everything is nonvolatile" and you get a perfect storm for security concerns: if a program segfaults or is terminated mid-operation, then suddenly any sensitive data it had spit out into memory might be capturable by other programs. Or, since it's nonvolatile, those secrets will persist "forever" (until they're overwritten, which may be a long, long time if storage is plentiful) and you may inadvertently leak secrets when you sell your memristor-backed storage. Whoever buys it can do some digging and uncover whatever was written into that buffer, if it's truly non-volatile storage.
So, while the future is exciting, underestimating the security implications probably isn't a good idea. We can develop new methodologies to combat the new security concerns, but new security concerns will almost certainly come to fruition. Of course, calling the concerns "new" is somewhat unfair, since new concerns are almost always very old concerns manifesting themselves in new domains, but new domains usually enable malware writers with new toolsets to exploit.
True, even under this model, you could probably have a virus running "resident" in a core without the OS even being aware of it. Basically, it's be like the MMU was a hypervisor and the OS was just a domU within it, unaware of viral "domains." But those viruses probably wouldn't be able to do much, except waste electricity, because they'd need to either get the IOMMU to allocate them some hardware to do IO, or get the OS (which has such hardware) to agree to do IO on its behalf. (Though now we can get into things like intermittent side-channel attacks, where you can mine bitcoins on one cell for minutes and then only have to get the OS to leak a single packet when you have a (rare) success, which is much simpler than getting a whole stable TCP channel or something.)
> 1TB of non-volatile storage [...] Most average consumers won't use all of thatWhen we're creating fully immersive 4D environments in place of holiday snaps then I imagine we're going to be using quite a lot of memory waving through our holiday review with friends.
Funny, I'm the opposite. With all this bandwidth available, why keep stuff?
Locally, if you're using RAID like everyone else, you just have to trust your local electricity supply and your ability not to rm -rf /.
Remote is fine for backups, though.
The 'enables new opportunities that were previous infeasible' component is more exciting. But generally, I think most of it gets swallowed up in more of the old.
The only thing worth debating is whether the GUI design pendulum will have swung between Flat/glossy an even or odd number of times by then :p
I'm sure people will find something to do with it but I'm having a very hard time thinking of situations where you want massive amounts of RAM and also need to be very power conscious or wary of power loss. Certain types of database, perhaps? I wouldn't imagine HPC worries much about power loss. Video game textures don't need to be nonvolatile.
For most purposes you can keep shoving in DRAM and SSDs until you have enough space.
Yeah, of course, everything has security implications and some technologies even more so, but unless one explains them and states how they are specific to the technology being discussed (like you did, for example) then the comment doesn't really add any value.
Writable storage using a centralized access method (file system) is hard enough to purge and keep clean, having a memory mapped persistent chunk of storage that is user process level writable creates an entirely new class of problems.
Getting rid of viruses from BIOS flash is hard enough, this expands that concept to all of userspace.
I sincerely hope the 'x' bit will be off for such memory.
A simple way to counter this is to tag the memory pages as free/used. If the memory controller reads a page as free, any reads from it can return zeroes without hitting the actual memory bus. Any piece of code that wants to hide will not be part of any active object and its page being marked as used should be considered an allocation error and be easily detectable.
Used to get them regularly on my Windows 3 -> Windows XP/2000 days.
Haven't seen them much in Linux or OS X in the last 15 years.
There might have been one or two. But I doubt you can find 10. And the number of users I've seen reported as affected by one was also laughable, something like 1% of all users of less (half to around 1 million).
The non-hacker media of course always touts this or that trojan that actually needs user intervention to be installed as a "virus".
I just assumed they are integrating it into their new super computer
For ordinary computing, maybe a terabyte of flash will finally be enough.
I would be willing to wager that new models of laptops in 2020 with spinning disk will be uncommon, and, by 2023, except for niche verticals. basically non-existent. Same for Desktop 2 years later in each category.
2025 will mark the end of consumers buying spinning disk.
How about more like 2015. You're already forgetting MacbookAir/Chromebook/Ultrabook crowd are slowly taking over (which have no spinning disks).
Problem with Flash is the more you pack in same volume the more of it comes out bad, while the rest is fragile ticking bomb.
Replacing slow, but reliable HDD with 100 writes limited flash is not the answer.
Unless your writing at the machine level I really do not see the need for most programmers to ever concern themselves with storage, that should be the work of the underlying machine code and/or OS.
I really recommend Frank Soltis' Inside the AS/400 if you are interested in computer architecture and virtualization.
Aren't the latest versions of PCI-e SSDs already approaching early version DDR3 transfer rates?
Energy consumption, heat dissipation, all of this becomes a larger problem for smaller structures. It's just easier and less error prone to build larger structures, yields are higher, so costs are lower for larger structures.
I'm certain the tradeoffs will change, but it's all limited by some physical law and while we can sometimes trade one law against the other, we can't currently break them, so tradeoffs will always exist.
(That “currently” is just a weasel word, but this being the Internet I understand why you put it there)
It is highly improbable that this new technology cannot be optimized in any relevant metric by trading off optimality in a different metric (this is what you are actually claiming).
The laws of physics dictate what the ultimate limits of a techology are; they do not dictate which technologies you actually choose to use.
I never said so, I agree. I'm just stating that I'm confident that in the future, we'll still have tradeoffs. And I'm certain those will include "fast and small, but expensive" vs. "slow and cumbersome, but cheap".
We already reached the point where it's a problem that the clock signal on a die spreads so slow that it's a significant problem to increase the clock rate. So we're pushing parallel computing and other solutions, but those come with their own set of trades to make.
http://umumble.com/blogs/hardware/machine-perception-of-time...
They are, they're the laws of physics. According to known theories information cannot travel faster than the speed of light, so unless a very very radical new discovery is made sometime in the future, size and speed will basically be inversely proportional to each other.
Just look back at how things were before the discovery of GMR[0]. We had very different tradeoffs then as compared to now.
Which technologies we use for our caches and main storages will change, but we're never going to get around the need for caching.