8Gb USB Flash Drive Endurance Test (2017) | Hacker News Reader

8Gb USB Flash Drive Endurance Test (2017)

goughlui.com

74 pointsdaxuak52 comments

Heating flash media can repair the oxide layer, depending upon temperature and duration.

In 2012, research on short bursts of 800°C heat could vastly extend media lifetime. Supposedly 250° for several hours has the same effect (assuming the packaging can withstand it).

In view of the shortages of recent years, we should mandate some form of this technology so we don't waste foundary output on throw-away parts (planned obsolescence).

https://m.hexus.net/tech/news/storage/48893-making-flash-mem...

>we should mandate some form of this technology so we don't waste foundary output on throw-away parts (planned obsolescence).

That's not planned obsolescence. Planned obsolescence is INTENTIONALLY rendering a product unusable/undesirable in order to sell more product. A prime example is changing the chapter questions in a college textbook to force new students to buy new books that teach the same thing.

Things that are NOT planned obsolescence but people mistakenly think they are include cheap inkjet printers (they last as long as any reasonable person should expect).... or NOT including an expensive oxide layer regenerating heater in a THUMB DRIVE that's only intended for casual file transfers, not long term storage of frequently changing data.

Ans really, does it make sense to MANDATE a complex heating scheme for flash drives? The way drive capacity and usage continuously scales upwards, old drives would be pointless long before they died. Do you REALLY still use old 32 megabyte flash memory in today's age of terabyte drives?

What do you think of the fact that 100K SLC used to be the norm, 2-bit MLC has an endurance of 10K, 3-bit "TLC" (awful misnomer) gets ~1K, and 4-bit "QLC" (even worse misnomer) in the low hundreds from what I've heard (they are very secretive about it)? That's an exponential decrease in reliability for a mere multiplicative increase in capacity.

"The new product stores 4x as much but lasts less than 1/100th as long" doesn't sound good, but that's the reality of what NAND flash industry is like. Meanwhile what little SLC is still available but priced far above what the cost/capacity would suggest, encouraging use of the cheaper and less reliable stuff. Manufacturers playing tricks with things like "SLC cache" and the like. Doesn't that seem suspiciously like planned obolescence?

It's not so secret.

The intel 670p (using 3D4 QLC) can do 370 TB of writes for the 1 TB model. [1] That makes it terrible for a DB, but perfectly fine for a media server. And other than the limit on total writes, it's overall a good performer at an affordable price. It also does clever things in converting some of QLC to SLC as a variably sized cache, so that smaller, more ephemeral written data doesn't affect it as much. [2]

If you want high endurance, you can get an Optane drive, like [3] which can do 17.52 PB on a similarly sized drive.

[1] https://ark.intel.com/content/www/us/en/ark/products/204109/...

[2] https://www.tomshardware.com/reviews/intel-ssd-670p-m-2-nvme...

[3] https://ark.intel.com/content/www/us/en/ark/products/147529/...

That 100x longer life span is not useful to most consumers.

Not all consumers have the same use case. I personally enjoy using flash drives for live OS.

I’m sure the market will create endurance drives for those with the need.

If it has I haven't seen it. It wouldn't be true SLC either, just TLC operated in SLC mode.

Neither are paper shopping bags.

Are paper shopping bags cheaper for grocery stores or are they being forced by the government?

Are you really asking that? In EU paper shopping bags are mandated due to environmental reasons https://www.theguardian.com/environment/2019/mar/27/the-last..., just like longer product lifespan https://www.beuc.eu/press-media/news-events/european-parliam.... None of those are directly beneficial to consumer at the moment of purchase in an obvious way, aka you pay more so almost no one would pick that option on their own accord, but have wide ranging long time ecological and usability ramifications.

Knowing about a solution but not implementing it is pretty darn near close intentionally rendering the product undesirable if not exactly that.

For the case of those USB keys: Would you prefer for everyone to bear the seemingly high cost of having such a system, whereas you probably never had a thumb drive fail nor know someone that has ? (You may, but if that's the case you are sadly part of the outliers)

I've had 6 or so USB disks fail inexplicably. Sounds like the proposal is a heater and a timer that are embedded on the stick. That doesn't sound like high cost. I suspect theres a timer and thermometer already on the sticks, that leaves just a heater, which are super cheap.

Failing inexplicably does not mean that it failed because of the issue talked about here. It could as well be bad contacts or soldering whiskers. I'd prefer a USB stick that does not fail too, but in fact, I have never had any fail on me so I don't know the prevalence, and know even less which parts come from this specific issue. Also, if your USB drive starts having a heating element, you need to trust it to not fry everything due to a regulation failure in badly designed units that fail closed. I also doubt a process of heating "at 250 degrees for hours" could be done at 50°. Even of it could, it would literally take days at a time.

> Do you REALLY still use old 32 megabyte flash memory in today's age of terabyte drives?

The tiny ones are great for physical deployment of certificate with key. Takes seconds to do a multi-pass wipe to prepare for the next delivery. Sadly, I wore out the drive I was using this way.

At 250°C, all components would desolder from the PCB. Provided that the components are attached at one side, one would need to be very careful while heating and cooling the device.

If it's not disturbed it could survive this, but ideally you can reduce the temperature just a bit further and increase the time period (though for some processes there is a temperature floor) so you don't liquefy the solder. If you just heat it and don't disturb the surface it should survive, but I'mma have to look up tin oxidation again. It might foam.

Would be interesting to have media with a self-heating cycle.

Playing with temperatures over 150 C on electronic parts is a lottery. Even if they survive, you may get in trouble later.

> Provided that the components are attached at one side

Unlikely with the tiny size of these devices. I am sure there are tons, but that most will have components on the A and B side. However… the same reasons that this works to assemble in reflow, are the same reasons it MIGHT not be an issue here.

Surface tension keeps the hot upside-down parts where they should be when their solder is molten. So if you have the right conditions, and you have the correct side-up, it should be no more of an issue to reflow a two sided part than a one. This is likely how it was made to begin with, the boards run down the line twice and the second time the upside parts have molten solder.

Obviously there would be issues for things touching components like the enclosure or tape/sticker bridging multiple parts. And fwiw to anyone… you determine “which side is up” by the weight of the components. If one side has a heavy part that won’t be held in place by surface tension, that’s the top side.

I wouldn’t do any of this for some internet-rumor that your drive will last longer, but I do know enough about two sided reflow to offer the above.

Edit: lol classic

Perhaps "annealing heaters" could be integrated on-die, and the bursts of heat controlled to be short enough that they don't change the external package temperature much if at all.

The article mentions lifetime/longevity/data retention, although not an 8GB drive that had been written to multiple times... I recently purchased a 2GB Sandisk Cruzer Titanium (nostalgia) thumbdrive off eBay, new in package.

Since drives at the time typically had preinstalled software like U3 or some app to encrypt your files, I wondered if a drive as old as this one would still be readable, never having had power since it was packaged up.

It was.

https://i.imgur.com/0dNo5ni.png

14 years.

It outlived nearly every CDR and DVDR I had.

Retention specs are usually for "end of life" conditions, i.e. after the blocks have been cycled the specified number according to endurance. If they were only written once, and the device kept at room temperature, I would not be surprised if it lasted a century or more --- the retention is an exponential function of both temperature and cycles, if I remember correctly from the last time I researched this stuff in detail.

I'm using the Bar Plus 128GB plugged into my RT-AX58U to host Diversion, a dns-based adblocker and it wasn't until recently that I realized that Dnsmask was writing a log for every domain resolved... Wonder how much of the drive lifetime I burnt through there.

I'm wondering about this for a long time and hope if any experts can explain this: is there any difference in append-only & small-sized log writing vs sequentially filling an SSD/USB flash drive/SD cards?

I guess there are significant differences due to flash wear-leveling and write-amplification but I'm not sure. What I know so far is that there's difference in wear-leveling because apparently cheap USB flash drives/SD cards might not do global wear-leveling but rather only limited local wear-leveling. This shouldn't be a problem for append-only logging because it's not writing to the same block repeatedly.

However for write-amplification I'm a bit confused. Does append-only logging to an existing NAND block require reprogramming as well? If so, small appends will leave to very large ratio of write-amplification and significantly shorten the NAND life time, right?

This is particularly important for devices like Raspberry Pi which rely on USB flash drives/SD cards for logging.

It probably worked out to only a few GB total, so probably not very much?

I am waiting for my raspberry pi running home assistant on a 32gb MicroSD to die. It's running now for about 3 years nonstop. I am amazed it is still running.

I had an 8Gb drive plugged into my Raspberry pi and it managed to kill it even without any writes happening but I guess I had not disabled the file access time stamp updates

I do wish access times were disabled by default. In my experience, they don't generally provide any useful information, but place unneeded stress on the storage.

Their original purpose was to help administrators identify unused files to migrate to auxiliary storage.

This is discussed in a BtrFS post on the atime impact to snapshots (quite harmful). Definitely mount BtrFS with noatime when snapshots are in use (likely best with ZFS also).

https://lwn.net/Articles/499293/

The last to fail was the Verbatim Store’n’Go which achieved an impressive 9751 cycles.

Nice to see --- I have a 64MB Verbatim SnG from many years ago with Samsung SLC that has probably been rewritten many times over, yet only a tiny fraction of the rated 100K cycles has been used, and actually has a full binary size (131072 512-byte sectors are accessible).

So what is recommended for long term storage? (not many writes, I mean, for offsite backup).

They make archival quality optical media for all types. Here's some Blu Ray with 1000 yr shelf life. https://www.amazon.com/Archival-Verbatim-M-Disc-Inkjet-Print...

Tape drives kept in climate controlled storage, however the upfront cost is expensive

Question, anyone knows a proven industrial grade usb-stick ~16G for server 24/7 usage?

Look for "SLC USB flash" on eBay. Definitely confirm the part specs with the OEM, and maybe test one to confirm longevity.

https://www.ebay.com/itm/201270936497?hash=item2edcaec3b1:g:...

>Definitely confirm the part specs with the OEM

>https://www.ebay.com/itm/201270936497?hash=item2edcaec3b1:g:...

That's especially true for the listing you linked. ;)

According to one source[1] the drive is actually MLC, not SLC.

[1] https://www.tweaktown.com/reviews/5549/lexar-jumpdrive-p10-6...

Hey thanks for the tip, i knew about SLC's but i meant something that is really proven, but also small so you can putt it into pizza-form-factor servers.

Something like that:

https://www.atpinc.com/de/products/industrial-ssds-usb-drive

But available and more then 2GB ;)

EDIT: Nice i found something

https://www.swissbit.com/de/produkte/produktsuche/produkte/

I see this listing for Oracle/Sun SLC media.

When you say "really proven," do you mean a lifetime of 100k writes, as is normally expected of SLC devices?

p.s. I've bought SwissBit before, and had no problems.

https://www.ebay.com/itm/165027448860?hash=item266c67081c:g:...

>When you say "really proven,"

I mean something like:

We used them for every esxi servers since ~5 years and never had a problem with it.

I just have a really hard time trusting any usb-stick.

But thanks the Oracle hint/link is a good one.

I was not going to comment but then you clarified what you were looking for and I can say “we’ve used Sandisk Extreme Pro USB 3.0 in 32GB - 128GB in every ESXi server for probably over 5 years without any failures.”

I’ve heard they released a bad SKU that they then retracted - I think it may have been the first 3.1 variant, but it’s no longer for sale. The only problem is they are bulky, but I believe there is good reason for that.

Does that help?

>Does that help?

Yesyes..thanks for the tip, will check them out...or like 10 of them.

Note that the Oracle flash drives are used, and likely won't have the full 100k remaining lifetime.

"This item was removed from a working machine."

Useful keyword is DWPD(drive writes per day) but I don’t think there are USB sticks rated that way.

I would use a decent SSD in a USB to SATA enclosure as a boot drive in a server were I can't fit an additional internal drive.

No place in a pizzabox for that.

There should be room to velcro an M2 enclosure to the front of even a 1U box

Yeah no, customers are not happy with velcro tape in a esxi server ;)

Some people posted a few companies in this post from April 2018:

https://news.ycombinator.com/item?id=16776344

If you want to read the entire thing:

https://news.ycombinator.com/item?id=16775768 "Raspberry Pi microSD card performance comparison" - April 2018

Swissbit does a lot of that sort of thing. A bit pricey.

This is a bit of an aside but when using dd, larger block sizes aren't always faster.

On modern systems 512k tends to be optimal, smaller sizes around 128k may be better on older/slower systems.

The 8M block size used here is certainly excessive.

Could be an attempt at matching the erase block size of the NAND. Of course due to the FTL and other abstraction layers in between, it probably didn't have the intended effect.

Reply on news.ycombinator.com