Hacker News Reader

Comment by Expez | Hacker News Reader

Over the last decade SSDs have increased their storage capacity at an incredible rate. This growth has mostly been fueled by adapting better process technology. I read a while back that the move to ever smaller scales (now 19nm) came at the costs of increased errors rates and that the limiting factor going forward would be error correction. Has this situation changed, or is SanDisk just throwing chips at the capacity problem, along with better controllers, and targeting an enterprise segment willing to pay the price?

View on HN

One of the interesting changes has been folks have been thinning out silicon wafers. Typically 700 microns there have been examples of working wafers as thin as 5 microns. If you stack 5 micron silicon you can put 100 chips in the same place you put 1 before, assuming you can figure out how to talk between them. Thru-silicon-vias (TSV) is the current poster child but there are better alternatives coming. Should be interesting when that comes to market.

This is a precursor to the next revolution in chips, 3D. It's funny how chips have of course always been three dimensional but the heat removal issues have so far stopped true 3D from happening (as far as I know). But SSDs have a completely different use pattern and likely don't run nearly as many Watts per volume of silicon so you could probably stack them way higher even if you didn't reduce the thickness of the layers all that much. Then it becomes a matter of routing and placement rather than heat dissipation.

Xilinx is already doing something like this: https://www.semiwiki.com/forum/showwiki.php?title=Semi+Wiki:...

Eventually this might even converge on something like a spherical shape, that's when the sum length of the connections is minimal (the surface area of the sphere is too, and that again is a heat removal problem).

Jim Gray was famous for predicting that the end point of computers were "smoking hairy golfballs". His reasoning was that spheres give you the shortest path length, there will be thousands of wires (hence the hair), and it will be difficult to dump the heat (smoking). I loved his image of a computer that looked like a a box of golf balls :-).

Hehe, cool, that's a very close description of the mental image I had when I wrote that, I was afraid that it would be dissed as ridiculous.

I read the paper you refer to, it in turn credits Frank Worrell as the original person coining that particular metaphor in ... 1985!

The paper quoted:

"To minimize memory latency these 4B machines will likely be smoking-hairy-golf-balls3. The processor will be one large chip wrapped in a memory package about the size of a golf ball. The surface of the golf ball will be hot and hairy: hot because of the heat dissipation, and hairy because the machine will need many wires to connect it to the outside world."

With a footnote:

"Frank Worrell used this metaphor in 1985."

The paper is here:

http://research.microsoft.com/en-us/um/people/gray/papers/su...

It's very interesting reading, especially when you look at how much he got right.

We don't quite have those smoking hairy golfballs just yet but we're slowly getting there.

I read that as "smoking hairy goofballs" and thought that perhaps he meant a bunch of weird old men with a cigar habit. Then you said spheres and it was all kinds of confusing.

That wasn't quite a prediction, it was the prevailing spirit of the times.

Ideally for volumes, yes, one would go for spheres. But in reality more often than not that translates to a cylinder because of technology trade-offs, with perhaps some endcaps. And not surprisingly, computing has already been there and done that.

If you take a bunch of compute nodes, roll the rack row up to minimize cable length, you end up with ...a classic Cray design from the '80s. And yes, the wiring in the middle of a cylindrical Cray was as 3D as it could physically get...

Looking at the picture, the technology of the era was an obvious dead-end:

https://upload.wikimedia.org/wikipedia/commons/f/f1/Cray-1-p...

and the trend is clear, just as it was back then with VLSI. Now the race is on to integrate those wires at a very large scale... (Cray also had a classic, visually appealing design approach to dump the heat, not unlike the enthusiast PC market today with commercial liquid coolers and some high-end HPCs)

I'd use RF from the controller to the different layers then you would only have to run power. Could even have 2 or 3 transmitters in phased array with little islands of storage on layers of silicon. This would also enable redundant storage (multiple recv) and simultaneous read and write (mimo). The distances in are cm and the signals wouldn't leave the drive package. With each layer exposing power and ground to the layer above the architecture would be scalable in Z just by adding more independent wafers.

https://eeweb.ee.ucla.edu/publications/journalVwaniRoychowdh...

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.159...

Quite hard to do this reliably with 1000's of signals in a constricted space. It also requires considerably more circuitry than wiring would, the very first trick of routing is that you try to do your placement in such a way that the routing of signals is trivial. This is of course not always possible and it's a very complicated problem at chip scale but I don't see how adding 1000's of RF based signals would help here. You simply have to make sure that stuff stacked vertically lines up where you want the interconnects to be. After all, RF would not only require a lot of bandwidth, it would also right away require error correction and all kinds of transmission lines.

Optical would probably be even better than RF (and that too has its problems).

The wiring is localized and free since it is lithographed and short run. No long wires for transmission line effects and high power driving. Optical is option but I think harder socially and technically in the now-term.

60Ghz CMOS is already a thing http://www.seas.ucla.edu/brweb/papers/Journals/RJan06.pdf

http://www.home.agilent.com/upload/cmc_upload/All/Webinar_60...

Also http://en.wikipedia.org/wiki/Ultra-wideband

Watch out for the "Hazards of Prophecy"

Intel has been working on voxel, light-based vertical interconnects for nearly a decade.

I recall when I was there in '99 talking with people about stacking cores. Then a few years later some friends there were doing 64 core research in ~2003-4-ish...

I'm sure they have evolved the voxels since then.

Yes. When scaling down semiconductors your oxide shows weird behavior. In transistors this was one of the crisis that hindered miniaturisation, there were alternatives (like exotic high-k dielectrics) but semi corps tend to steer away from such things.

Miniaturization happens not only on the surface (length and width) but also depth. This is where silicon oxides come into play. Flash tech relies either on charge trapping in the oxide itself or in a floating gate. Both become sensitive to effects like tunneling when you scale them down in the 10nm magnitude.

Here is a paper from two years ago talking about the the theoretical limits of SSD technology.

The Bleak Future of NAND Flash Memory: http://cseweb.ucsd.edu/users/swanson/papers/FAST2012BleakFla...

The next doubling in capacity will likely come about through 3D NAND which allows for much larger feature sizes than current planar NAND. Samsung already have 3D NAND on the market (in 128Gb / die capacity), and other manufacturers intend to go down this route too.

Using larger form factors like 3.5" or 5.25" can possibly help.

You can fit 403 MicroSD cards into the volume of a standard (9.5mm) 2.5" laptop hard drive. The largest MicroSD currently on the market is 128gb, meaning you can fit 51.5 TB of MicroSD cards into a 2.5" drive.

SSDs need additional circuitry, chips and memory, but MicroSD cards aren't 100% flash either. This shows that the ceiling is very high even with current lithography.

My guess is that there's just not much market for a $200k+, 50 TB 2.5" drive, but it would be possible to make in high volume production.

If you need storage capacity, traditional hard drives will easily beat it in terms of TCO.

If you need IOPS, it's much better to use an array of smaller high performance drives.

I could picture it being useful in some military and space exploration applications but that's about all.

>space exploration applications

Doubful, since they are not radiation hardened.

I could rather see a market for smaller size SSDs which contain lots and lots of chips and thereby have their own RAID subsystems.

radiation poses problems, but it has been since found that the radiation hardening measures taken on past IBM chips have been largely overkill for the application.

one can get near the same performance for much cheaper by using consumer electronics with redundant systems and a voting mechanism between systems.

That suggests an interesting alternative path: WORM style flash, where you simply keep on adding at the end.

In fact today the main problem with the 2.5" form factor is being limited to the performance of SATA controllers, not how much you can pack into the space.

There's little reason to pay a massive premium for huge SSDs if your controllers limits you to a small percentage of the potential capacity of the drive. So instead you get things like the PCIe cards with 4+ controllers that gets RAID'ed together by the drivers.

As the controller bandwidth catches up, I'm sure that'll change.

Then why the entry level Mac Book Air still has paltry 250 GB! I hope they catchup with the rest of the world soon.

These new drives from SanDisk are still over $1/GB in cost.

The entry level MacBook Airs (and retina MacBook Pros) only have 128GB SSDs. I get that Apple wants to keep the price down so they can keep their margins high, but it's a bit small for a primary computer in 2014.

Actually, it's more than enough for me, and I dualboot. Not everyone has the same usage habits :)

Where do you store your pictures and home movies? A movie of an event is near 1GB in HD.

Some people don't take pictures or videos. I take the occasional ephemeral photo of my dog doing something stupid. I have maybe 20 photos that I know of from the last year or two and they're all of stupid crap. On my phone. At least 25% of those photos are of my dog mid-yawn. Not kidding.

However, I do have something like 6TB of active desktop storage (not including NAS) at capacity because of software, audio samples, disk images, virtual machines, backups, documentaries and shows, etc.

Everyone just uses computers differently. My mom's storage is something like 2TB and I think about 70% of it is home movies and photos. The remaining 30% are duplicates of some of the same files, likely on the same disk in different directories.

Flickr. 1TB for free (although I'm a grandfathered pro user at $25/year).

I have a NAS with 3TB of RAID storage. When iCloud automatic backup complains that my phone is running low, I download the camera roll to there.

If I'm making a home movie, I'll just copy the files to the SDD temporarily. I have 30GB+ free most of the time.

Yeah, you are unusual in my view. :) My mom, who barely uses the computer - and has a laptop, was able to get up to 200 GB within 6 months. How? Simple -- videos and pictures she takes of the kids from her smartphone, that she insists be saved and archived. That's my mom.

I, on the other hand, am running out of free space on my 4TB capacity desktop, and will be purchasing larger storage devices soon.

I'm an engineer & I've been using a single 60gb SSD for over a year. I run a few virtual machines and when I'm done I delete shit I don't need or store it elsewhere. What do you use 4TB for? I could only imagine using it for watching TV but I don't like doing that. Everything else I just rent cheap servers.

Music, games, movies alone easily take up like 3TB out of the 4TB. Big software (3d cad software, some gfx software, etc.) take up a TB. One of the NI instrument collections (http://www.native-instruments.com/en/products/komplete/bundl...) is like 100GB (I forget which one, I think it's the Piano one with the soundbank) -- that's just a single one. If you're working with media stuff, it's way too easy to find even 20TB to not be enough for your needs. And I'm just an amateur for goodness' sake!

Oh, and I have an external 3TB drive where I house all of my backups and captures of old desktops. It's completely full, and I will be purchasing another ~3TB drive in the near future for backup purposes.

Yeah I understand it now! Those instruments are "tools" :-) A music collection is also understandable, but I generally stream, since various services I can run simply from my work environment (Emacs). I'm sort of scared to even get a 1TB disk because I'd worry about introducing distractions. I've liked the constraints 60gb has imposed on me. However, admittedly, as I've started contracting around various places each which has different environments, it'd probably soon upgrade to a 128Gb in order to do the "context switching" between the various virtual machines with Vagrant I need to get work done in various places, in the past I've managed to solve that by just having a development server at cheap places like Online.net, but it's nice to have the virtual environment locally to get around the typing delay caused by ping time in a terminal.

You have wayyy too much data. You're a straight up digital hoarder. You couldn't even sift through that shit in a lifetime, let alone use it all.

Making full disk backups and archiving huge collections of media assets you'll only ever use 1% of is just a huge burden of mind and a huge loss of portability and flexibility. Not to mention a waste of time and money.

I know you say you're an amateur as if professionals would have even more, but in reality, they've probably realized what a time sink it is and focused on just the small set of stuff that actually matters.

I'm not really sure you're getting downvoted, since your point of view is pretty reasonable (even if many will disagree with it), and you've expressed it fairly respectfully.

It's not just an opinion, it's exaggerated to the point of being factually wrong and bearing little resemblance to a reasonable point. I think the downvoting is reasonable. Here, I'll go through various points:

"You couldn't even sift through that shit in a lifetime, let alone use it all." "media assets you'll only ever use 1% of"

Assume someone's daily routine involves 3 hours (below average) of TV/movies, in HD. 5-6GB a day, 3TB is reached in under two years. 1% of 3TB is reached in a single week. Even hanging on to a small fraction of the media they watch for future viewing/sharing would fill up 3TB in a handful of years.

"Making full disk backups and archiving huge collections of media assets you'll only ever use 1% of is just a huge burden of mind and a huge loss of portability and flexibility."

Managing full disk backups means buying a disk and pointing an automated program at the folders you want. Average time per week: 0 minutes.

Managing huge collections of media assets: This means navigating services to click on movies and shows to download, and occasionally moving a season into a different folder. Average time per week: 5 minutes.

Burden of mind: Remember to buy a drive every several years, otherwise none.

Loss of portability: Keep it on an external drive if you want portability? Not sure what is meant here.

"Not to mention a waste of time and money."

Tell that to anyone with basic cable, as far as watching the content goes. Storing it after watching has negligible additional time/money cost.

Now do you see what the problem is with that comment? While it's certainly possible to be a digital hoarder, having two shelf's worth of movies on a hard drive is extremely weak evidence toward that.

I'm just going off my 3TB disk - its only 2TB full, though. I do have it backed up three times over at three different locations when I get the time to truck my external drive around to do so.

75GB of TV 300GB of Movies 80GB of Music 9GB Pictures 8GB Code 140GB Games 84GB OS Images 64GB Installers (I rip game cds so I don't have to keep the discs around) 60GB VMs 50GB Containers 2GB Books 2GB Comics 200MB Documents 4GB Misc Audio 14GB Misc Video 240GB of cross-device backups

This stuff adds up. If I actually ripped my entire music, tv, and movie collection I'd easily fill up the rest of the disk. Mostly I'm just impatient about only 20MB/s reads off old cds and dvds.

This is exactly me -- ~2TB of 3TB storage full, similar content to you. The difference is that I store all that stuff on a RAIDed NAS device and stream it over wifi when I need it.

I don't back it up, but it is RAIDed with 1-disk fault tolerance, and that's pretty much good enough for me. I mostly only ever add new content to the NAS or read existing ones, so I'm not worried about versioning.

Best of both worlds imo.

Yeah, it adds up if you're too obsessed with archiving to actually cull some of it. Most movies are only worth watching once. Most music collections are too big. Most photos are duplicates and/or out of focus. Most games are replaced by much better ones.

I deleted a few hundred GB of movies a while back and realized I didn't even need a multi-terrabyte drive at all. Feels good man.

Media. I can probably about as much data in 1 minute with a DSLR as I can create in a lifetime on editing text files.

Your mom should never be storing 200GB of data on a hard drive. Thats Wrong(tm) because baking up properly is difficult enough when you have a team of sysadmins.

Instead, she should be using a private youtube for her videos, and Flickr for photos. The rest should fit in a dropbox account. Not only will her data have a team of professionals trying to prevent the inevitable fuck-up, and she'll enjoy 'cloud' benefits, but also she'll have a chance to curate her collection during upload, because no one has enough time in their life to look at 200GB of home movies and photos every 6 months (it's turning into a landfill)

> Your mom should never be storing 200GB of data on a hard drive. Thats Wrong(tm) because baking up properly is difficult enough when you have a team of sysadmins.

It is much thanks to me that she does not up it on the cloud. I don't trust cloud services, so that's what I suggest to her -- I'm happy to sysadmin her devices whenever she needs me to though.

I hope in the future there are truly Easy-to-use-for-moms storage devices, along with the software to go with them, that gives them all the benefits cloud may have.

Doesn't Time Capsule count as a backup? It's not difficult. https://www.apple.com/airport-time-capsule/

Sure, and when Youtube decides to change their policies, and Dropbox goes bankrupt, then she has.... nothing. Hard drives are cheap. Run a backup service and you have no worries.

I've been running a 256gb SSD in my Macbook and haven't come remotely close to hitting that limit.

In my case it's my secondary computer, so I don't see a need to store much in the way of media (Movie, Music) etc on it, which is about the only way I imagine I would use that much space.

The OS and Applications takes up a decent chunk, as does VirtualBox, but other than that it's just documents and source code.

Does your mum need to carry around 200GB+ of movies and pictures? would she get by fine with a multi-TB drive on the network at home and a smaller laptop hard drive.

In theory this setup would be perfect. In practice it is a nightmare because the software makes managing media split across multiple file systems that may or may not be available at any given time very hard. It is particularly bad within the Apple ecosystem. iPhoto really wants your entirely library in one folder on one disk. The iCloud photo stream options do well at syncing but cannot handle the model of store all my photos in the cloud and cache the recently taken/commonly used ones locally.

There are cloud services that solve this but they have their own set of issues. Many are expensive compared to basic direct/network attached storage, others require photos to be public, others come with dubious guarantees that they will still exist next year.

I get a call from my sister with a 128GB MacBook Air every month or two asking to remind her how to move her baby photos/videos to her external drive. Every time we do it I am surprised how complicated and error prone the process is (we've managed to accidentally permanently delete photos). If anyone has a good recommendation for an alternative to iPhoto (and not cloud based) that understands the concept of archiving photos to a network/usb disk that is not always present I would love to hear them.

Aperture can do it, and as iPhoto's big brother isn't too much of a leap for people used to that.

Have you tried Time Machine and found it wanting?

It supports both network and impermanently attached usb disks.

>Apple wants to keep the price down

Are you being sarcastic?

If they really wanted to keep the price down, they wouldn't strive for 2x or higher profit. MacBooks (any Apple hardware) feature component-set easily sold at half the price. You are simply paying for the Apple logo.

You do realize they just dropped the price of the Macbook Air across the board by $100, right?

Which tell you all you need to know about how they want to "keep prices low".

Funny how often this is repeated, yet no other laptop company has managed to walk in and take over.

Almost as HP, Acer, Asus, Fujitsu, Toshiba, Sony and co. don't want an easy few hundred million dollars. Or as if you're wrong and there's more than a logo involved.

He's wrong of course. Apple is operating at a scale far above these other companies.

But they weren't always. The major laptop players used to be IBM, Toshiba, Dell. Apple came in with more expensive laptops and killed them.

    Apple wants to keep the price down so they can keep their margins high

OP clearly means that Apple keeps their costs down, not the retail price.

>You are simply paying for the Apple logo.

You have absolutely no clue what you are talking about.

I'm paying for a high quality machine with many features already built in, integrated with a well designed, power-sipping, user security and user privacy optimized operating system that uses modern best practices under the hood that scale to current and future multicore processors, with a solid UNIX at its core with hardware drivers that just work. And amazing support like free replacement motherboards, hard disks, batteries, chargers, keyboards, screens, etc. under AppleCare if anything goes wrong.

I only have 24.5Gb of data including family videos and Nikon NEFs. 128Gb is plenty for me.

Plenty enough for me, my gaming rig at home needs over a TB but I would use a macbook for development, don't need more than 50GB or so.

These days, I get most of my entertainment online and store my code repositories and other permanent data on a home server. I wouldn't know how to fill 128GB on a laptop.

You could fill it up quickly with big-sized software and games.

Aha. You can spot the Apple fanboys rushing to contend this just by saying they don't use 128GB to begin with.

The entire point is that pretty much any other (read: non-Apple) laptop/desktop you buy nowadays is going to come with at least 500 GB, and generally 1 TB nowadays.

No need to get up in arms and whine that Apple is doing everything just right for your tastes.

edit: I obviously wasn't referring to 1 TB SSDs. The parent's comment said that 128 GB is pretty paltry for storage capacity nowadays, as the thread is concerning the rate of growth of SSDs.

The entire point is that pretty much any other (read: non-Apple) laptop/desktop you buy nowadays is going to come with at least 500 GB, and generally 1 TB nowadays.

Is that the case? Most machines I've seen around the cost of the Macbook Air still come with hard disks or 128G SSDs - maybe a couple might push that to 256.

Indeed, I don't think I've even seen a laptop with 1TB of SSD, aside from the very-top-end-I-upgraded-it-especially Retina Macbook.

Well, I'm no fan of Apple at all and haven't owned an Apple product since the II GS. And I'd argue against storing any significant data on a laptop or desktop drive, as this is one of the most likely components to fail. Data should be on a storage server with RAID. All you should have on your primary drive is your OS, applications, and active working set of data. Any archival data or larger files should go on the storage server.

Glancing at amazon.co.uk's results for 512gb ssd laptops the cheapest non Apple on page 1 was £1,676.55 (Lenovo Yoga), Apple was actually a bit cheaper (macbook pro £1,213). Not really a scientific survey but they don't seem too bad.

The new MacBooks boot in 1 second.

I'll take that any day over extra storage I don't need.

> The entire point is that pretty much any other (read: non-Apple) laptop/desktop you buy nowadays is going to come with at least 500 GB, and generally 1 TB nowadays.

Which laptops are these which come with 500GB SSDs in the base models? Or are you suggesting that Apple should make a laptop with a 500GB spinning disk in the base model? Because funnily enough, they have one; the non-retina 13" MBP. Of course, no-one buys that, because it's worse than the cheaper Airs by all metrics but storage capacity, but it's there if you want it.

>The entire point is that pretty much any other (read: non-Apple) laptop/desktop you buy nowadays is going to come with at least 500 GB, and generally 1 TB nowadays.

That's for non SSD drives. So every other laptop/desktop you're gonna get with those storage sizes also has rotating rust disks, with are more volatile due to moving parts, have much worse read/write speeds and are noisy.

>That's for non SSD drives.

Yep, clarified in my edit.

>So every other laptop/desktop you're gonna get with those storage sizes also has rotating rust disks, with are more volatile due to moving parts, have much worse read/write speeds and are noisy.

That's cute. So now hard disks are so obtrusive, fallible, and out-dated that they're literally not an option anymore? I think some people have such a love affair for the acronym SSD that they forget, disk access is still hundreds of thousands of clock cycles regardless.

See: http://imgur.com/pQ8BriQ

SSD benchmarks are clearly better than HDD all around, but not by the orders of magnitude that you seem to believe.

>That's cute. So now hard disks are so obtrusive, fallible, and out-dated that they're literally not an option anymore? I think some people have such a love affair for the acronym SSD that they forget, disk access is still hundreds of thousands of clock cycles regardless

A time N improvement is still better that no or marginal improvement, even if it's not 2 orders of magnitude better.

It makes all the difference between waiting for 5 minutes for some IO process to finish and waiting for 1 minute. 1 second would have been nice, but 1 minute is already a game changer.

I don't understand how keeping the price down keeps margins high. Perhaps you meant cost?

You're right, that statement was poorly phrased. Apple wants to meet a certain price point for their notebooks, but they also don't want to sacrifice on their margin to reach that price point, so they reduce the cost of manufacture by using less expensive components, like lower-capacity flash storage.

These hard drives are enterprise targeted and, I assume, rather expensive. Microsoft was selling Surface Pros with 32 GB not long ago, with over half occupied by Windows.

Reply on news.ycombinator.com