I was thinking today: If persistent Flash memory has RAM-like characteristics, why are we still treating it like it's a spinning disk? Laptops and mobile devices should have a completely different memory hierarchy:
- Registers
- L1 and L2 cache
- Main Memory
- Persistent Memory
- Cloud Memory
The 1st three are also what we have today, but the last two are completely new. Persistent Flash stores are faster but more expensive than spinning magnetic platter. Cloud Memory is vaster and omnipresent, but much slower and less reliable than the hard drive. Together, they can fill in each other's weaknesses and create something vastly more useful than the old spinning hard drive.Apple is moving in this direction with their new Macbook Airs, which have taken the first step by taking the MLC Flash chips out of the pointless metal box and placed them on a form factor somewhat like a DIMM. They have moved even further with the iPhone/iPod Touch/iPad. Here, we have computers as powerful as workstations of the mid-1990s, but there is almost no notion of a Disk Drive. Instead, files are organized primarily by the Apps that create and manage them.
My prediction is that Apple is moving toward a world (at least for the average user) where there are only Apps and your data magically takes care of itself.
EDIT: Does anyone else also realize that a big reason our lengthy boot-up procedure even exists is because we had to have a spinning disk? Now that we have RAM-like stuff that's cheap enough to use in spinning-disk quantities, we don't really need the kinds of boot-up procedures we have now.
I'm not sure we are (other than low-level details like how the kernel does I/O scheduling and filesystem implementation details). RAM is transient; SSDs and magnetic disk are both persistent. Hence, both SSDs and magnetic disk can serve as stable storage. SSDs and HDs have different performance characteristics, but I'm not so sure that a radical rethinking of the storage hierarchy is actually called for.
a big reason our lengthy boot-up procedure even exists is because we had to have a spinning disk
Offhand, I would guess that a big reason for lengthy bootups is the time required to probe, detect and configure hardware (plus latency stuff like BIOS init). An intelligent OS will be able to arrange the disk I/O that needs to be done on startup so it is mostly sequential; SSDs and HDs are not significantly different with respect to sequential read performance.
Wow, somehow I missed this fact after 30 years of programming!
both SSDs and magnetic disk can serve as stable storage
Yes, but sometimes a quantitative difference can result in a qualitative one. (Like branching and merging and using historical information in Git. The fact that you can do this so quickly from the local Git repository means you can use it in ways that wouldn't be practical in something like Subversion.)
Offhand, I would guess that a big reason for lengthy bootups is the time required to probe, detect and configure hardware
Why doesn't this apply to hibernating laptops? (Or for that matter, the iPad?) In this day and age, to we always have to turn the power 100% off on a regular basis? Also, there are experimental OS like EROS, where there is no boot-up, only a reactivation of certain facilities like network cards.
The lengthy boot-up process is now living on a set of cultural expectations. There is no longer a technical reason for its existing.
Yay for sarcasm. My point is that changing the performance characteristics of the lower levels of the storage hierarchy does not necessarily mean we need to make radical changes.
The lengthy boot-up process is now living on a set of cultural expectations. There is no longer a technical reason for its existing.
Maybe so; my point was just that I don't see how SSDs vs. magnetic disk fundamentally effect the boot-up process. Hibernation vs. power-off is orthogonal to the type of storage used.
If you just treat the Flash as RAM, with volatile RAM as a very large L3 cache, the notion of hard drive can just disappear. Everything just becomes orthogonally persistent.
I'm guessing this is a troll. Most programmers I talk to in person just get this without so much exposition.
Of course, dropbox is most likely going to say thanks but no thanks to these early offers, and the big guys will kick themselves down the road.
The first is open source (librsync) and the third is pretty much a garden-variety hierarchical, AJAX-enabled database browser. You'd have to surmise that companies like Google or Amazon, with so much in-house talent, would be much better off writing the proprietary, closed-source portions from scratch than paying a hefty premium for them on the open market.
So Microsoft has made three attempts to do what Dropbox does and has still not succeeded. I think there's more to it than what you're proposing.
Specifically, I think you're missing key ingredient #5, file conflict resolution.
It was the dropbox of its day. The issues began after Microsoft purchased and then neglected it.
This idea is just so obvious and stupid simple that it boggles the mind that nobody has really done it. You can look at all the major players and they actually all have deliberately crippled half-implementations. For example SkyDrive should be a Dropbox killer, but for inexplicable reasons MS has never made it so you could just 'mount it as a drive' and thus it is nearly entirely useless. The trivial work it would require for MS to do that and the wildly useful properties of having that feature can only mean that MS has deliberately decided to cripple it. Gdocs is the same - in theory you can shove anything into your gdocs account and get it out again. But Google, who could so easily do it have never released a utility to mount that space as storage.
I'm really curious about why every player is half doing this and then stopping short. It can't possibly be that they are all too dumb to see how useful it would be - it can only be that they see significant risks or negative consequences in doing so.
The computing industry, as a whole, is undergoing a massive shift, and I'm afraid that really innovative companies like Dropbox have simply provided the template by which the filesystems of the future will operate.