Hacker News Reader

Comment by ChuckNorris89 | Hacker News Reader

>whatever failure damaged the SSD is statistically likely to compromise the reliability of the rest of the board

Sorry, but how? That's just reads like BS.

SSDs have limited lifespans due to the limited write cycles of the NAND chips. So no, what damages NAND does not damage the rest of the board. The more you write and swap to flash, the faster it wears out, taking your Macbook down with it when it dies.

Granted, I expect the controller to keep moving your data around to the cells with the least ammount of wear and isolate the ones that have worn out, reducing your storage space and speed of the drive over time.

View on HN

In practice, exhausting NAND write endurance is usually not what causes SSDs to fail in the field. It is especially unlikely to be the underlying cause of unexpected and seemingly premature catastrophic SSD failure.

Also, it is not possible for an SSD to decrease your usable storage space over time. That would break any partition table format or filesystem that expects an ordinary block device. It also wouldn't win you any significant increase unusable lifespan, because when your drive starts retiring large numbers of blocks, all of your blocks are on the verge of failure. (Though keep in mind that "failure" here is defined as "only able to retain data for one year" for consumer SSDs).

> It is especially unlikely to be the underlying cause of unexpected and seemingly premature catastrophic SSD failure.

One of my graduate students experienced this, the hard way: writing up his thesis, his mum plugged in "the wrong USB C" charger. I don't know exactly what happened, but the board was utterly fried. The problem with the soldered-to-the-motherboard, everything-utterly-encrypted approach is that it's almost impossible to do data recovery. On the M1+ models, you literally can't do anything except semi-fix the motherboard as the flash and its controllers are integrated and the keys are cryptographically securely stored. This video [1] shows what data recovery is really like -- I don't often link to YT videos, but it is an hour long, involves an awful lot of BGA de-balling and re-balling and the scouring of spare chips from other donor dead machines, as Apple doesn't let manufacturers sell the chips separately and datasheets are all NDA'd.

We're very much in the part of the curve where you'd need a lab and a lot of serious work to get data off these things without either sacrificial macs or luck (e.g. one blow voltage regulator). Of course, the flip side of this is that your data is really, really secure. You just have to be damn sure that you back it up correctly.

[1] https://www.youtube.com/watch?v=110x5rMHCIw

I work for my university's helpdesk and a lot of students learn this the hard way every day. We always require users to assume full risk of data loss whenever they check in for a repair. For most computers, if they aren't okay with that, we can just tell them to go to a third party shop and have the drive pulled out and imaged first. For recent Macs we basically just have to tell them it's gone for good if we can't get the machine booting without having to swap the system board.

> soldered-to-the-motherboard, everything-utterly-encrypted approach is that it's almost impossible to do data recovery

Well, that's kind of the point.

> You just have to be damn sure that you back it up correctly.

Exactly. And that's as easy as plugging in an external hard drive occasionally, or storing critical files in the cloud.

>> soldered-to-the-motherboard, everything-utterly-encrypted approach is that it's almost impossible to do data recovery

>Well, that's kind of the point.

Not that I don't appreciate a security-minded platform, it just seems overkill for 99% of the people who'll purchase them, and do nothing but cause heartache when the internal SSD fails. And fail they do. it's rare, but I've had several SSDs (Toshiba and Crucial, if it matters) fail within 1-2 years of moderate usage. No warning there was an issue, the drives just disappeared one day and I was left looking for backups.

> Not that I don't appreciate a security-minded platform, it just seems overkill for 99% of the people who'll purchase them

No man is an island. If my mom is insecure, then all the messages I sent her are leaked, too, no matter how secure I try to be. If my boss is insecure, then I'm even worse off than that.

Also, if high-security stuff is normalized, then you don't wind up stuck in a place where you have to choose between "the secure option" and "the option that can actually run the apps I need." E2E encrypting the whole world is also the best defense against the government passing laws that make E2E encryption illegal.

>E2E encrypting the whole world is also the best defense against the government passing laws that make E2E encryption illegal.

The government doesn't need to pass laws to ban encryption (at least in America) since they design the encryption standards themselves. It's basically common knowledge at this point that everything NIST cranks out is vulnerable to differential cryptanalysis beyond the domain of public understanding. Apple, Google, Facebook and the other top dogs all help create the illusion of choice in exchange for keeping the SEC off their backs.

>It's basically common knowledge at this point

What is asserted without evidence can be dismissed without evidence.

And what's really annoying is that you are doing a bad job of arguing for a position that I actually kinda agree with. NIST has published a backdoored elliptic curve-based RNG[1]; don't trust them. Encryption algorithms need some sort of verifiable provenance for where those numbers came from.

[1]: https://en.wikipedia.org/wiki/Dual_EC_DRBG

On that note, it's unfortunate that T2 can only create ecdsa-sha2-nistp256 secret keys. Right now I use Secretive but I might resort to a different utility that generates ed25519 and stores it within keychain, if there is one.

> 99% of the people

I think you underestimate this number. Most businesses are going to want security/encryption over ability to recover data to prevent leaking secrets in the case of a lost/stolen laptop.

as for everyone else, backups really need to become the norm for everyone because as you say any HDD/SSD can fail for any reason anytime beyond any possible recovery apple laptop or not.

I recently learned that my hard drive was encrypted with Bitlocker, had I not been attempting to get Genode running off a USB stick, I might have gone on for a year or to in blissful ignorance of how remarkable insecure my stuff was as a result of this.

It took about 12 hours to decrypt it, and restore sanity.

Availability is part of security, if it's encrypted, and you don't (or can't) boot just the way Microsoft wants, all your data is gone.

If it is so, your data will go. Backup backup backup. Otherwise unencrypted data cannot save you.

Well, the main reason I was upset is that if I dual boot something else, even if it supports NTFS, my data is unavailable to it.

> it just seems overkill for 99% of the people who'll purchase them

There are many places in the world where information on your laptop even criticising the government can end up in you going missing e.g. China, Hong Kong, Saudi Arabia, Myanmar etc.

> Well, that's kind of the point.

Being able to yoink the SSD from the original machine, plug it into another compatible one, and decrypt the contents with your password, is not a vulnerability, it's a feature.

I think that’s a joke?

I mean, why even encrypt it in that case?

It’s not like the drive is a web service that will detect and block your password attempts…

No it's not a joke. If the password is sufficiently complex, it can't be brute forced in a reasonable amount of time. But it's still a way for you to get your data back having only the SSD on which it is stored.

Or, well, fine, let there be an active element. But let me export the master key then. It's my device, I have root access to it, I should be allowed to do that. And, yes, do still put the SSD into a slot.

> I mean, why even encrypt it in that case?

Yeah because your time machine backup drive that's encrypted with "just" a password (you did encrypt it, right?) is somehow more secure than an internal SSD encrypted in the same exact way.

> But let me export the master key then.

You can do this encryption shindig with the ability to export by not using FileVault and using your own encryption software - Apple just doesn't allow this by default. In fact, why only store it on the computer? Rclone can upload and replicate an encrypted (via a long salt + password) copy of your data to dozens of cloud providers. Your encrypted data could be on 3 cloud providers in 10 physically separate locations on 30 different hard drives at once, possibly even in separate hemispheres. Or chuck it into S3 replication and be in all 81 availability zones if you so wish.

I mean, just turn off encryption if that’s what you want.

I don’t think you grasp how the encryption keys are generated, derived, and stored.

And I doubt you are using a password the length of the key. Well, maybe you create a USB human interface device that fits on your keyring to type it for you. I do have something like that for one time codes, but if that is your sophistication level, then we wouldn’t have this discussion really.

I feel for the “it’s mine, I bought it” logic, and I miss the days when our rights were more absolute. But that’s not today.

There are key-derivation functions like PBKDF2 that are computationally expensive enough that it would make brute-forcing infeasible at best and impossible at worst. Perhaps only if your password is 1234 or qwerty, but even then it would take literal years.

> I don’t think you grasp how the encryption keys are generated, derived, and stored.

And you're wrong.

If data is not encrypted with a piece of secret information (usually a password) with enough entropy to keep it secret in the face of offline attacks, then it is as good as not encrypted, period. The hope that your hardware design is sufficiently complex and non-repairable to prevent your attacker from disassembling it and performing such an attack is not a replacement for proper encryption.

And if it is encrypted with a secret with sufficient entropy, then making your hardware design non-repairable serves no reasonable purpose.

You can easily limit password attempts by requiring a sufficiently difficult password hashing algorithm and then just use a secure passphrase. Nobody is breaking into my encrypted drives via a guessing attack (other attacks exist, but they ~all also exist on macs).

Encryption is based on math, not on some hardware gimmick. I'd absolutely clone my laptop drive and move it to a new one if I want to, having exactly the same setup.

> > soldered-to-the-motherboard, everything-utterly-encrypted approach is that it's almost impossible to do data recovery

> Well, that's kind of the point.

Great; how do we opt out?

with different machines? The framework laptop doesn't work this way, afaik

Too bad i switched to Mac OS from Linux. The OS matters to me and OS X is still the least annoying.

Apple is not about giving you options. Either you go the Apple way or you leave the ecosystem entirely, and except for the very beginning when Steve Wozniak still had some influence, Apple has always been like that.

It has advantages and disadvantages. The disadvantage is obvious: few options. But the advantage is that they can make a consistent ecosystem, with sensible choices, and not dragged down by compatibility issues. It "just works". Not my thing but I totally understand those who love Apple.

The way you do recovery with Apple is via Time Machine backups, use that.

Desktop Linux is the opposite, lots of options, but perhaps because of it, it doesn't have the same polish as Apple, and even Microsoft products, or, using your words "it is annoying".

That really just shows that everything has tradeoffs. I'd Love if Windows could run on top of the Linux kernel (as in, nvidia drivers and all the Windows compatibility while being able to do stuff like `lspci` to see my real hardware - WSL2 doesn't do full PCI device passthrough yet) but that's unlikely to happen with how much the Windows team is seemingly trying to simplify their testing infrastructure (removing CPU support and removing user choice).

Back it up to some cloud vendor that also holds the encryption keys... Brilliant bit of security there...

If you don’t want that, back it up to a cloud service that doesn’t hold the encryption keys. Eg, use tarsnap. It’s your computer and your data.

Thanks, I don't remember trying tarsnap. I use $OTHERBIGVENDOR and its been sticky, because I have ~60T of large files (miniDV .mpg captures/4k video/etc) mixed with an insane number of .5k files from git repos/etc.

That combination has broken just about every modern cloud backup application I've tried to use over the ~10 years that meets a short list of minimal features I require (locally stored encryption keys for one). It actually breaks $OTHERBIGVENDOR in its default configuration as well, but I've collected a pile of tweaks/etc that keep it functional although I've managed at one point to cause my account to go into a reindexing mode on the servers that didn't complete for months too.

So, maybe at some point. I've got some experience in the space :) and I've considered writing my own when I eventually give up on $OTHERBIGVENDOR. So many of them are written in "modern" languages and the clients are outrageously slow, or get exponentially slower as the data set grows.

I've looked at tarsnap in passing in the past, but haven't gotten around to trying it because from their description of what appears to be a traditional referenced counted global dedupe. I suspected it of having problems when the hash map need to track 50T+ unique hashes from all those video files across backups.

PS: That doesn't mean there aren't good tools, my local backups are via rsnapshot to an offline USB+RAID I plug in once in a while.

If you have 60T of files, then you're already free from the problem since Apple doesn't sell a machine with 60T of soldered on SSD.

the motivation for soldered everything is profit margins and the ripoff prices for upgrades.

Not security. There is no additional security here

Of course there is additional security.

You can't just physically remove the SSD from the machine, place it in another and brute force the encryption.

of course you can desolder and move the chips - all you need is a hot air station and some flux. It's of course harder but nothing even remotely close to breaking the encryption itself, like orders of magnitude easier - any hobbist can do it.

It's another story, a part of the encryption is stored in yet another place on Macs, which it'd be still stored there if the SSD could be moved in/out.

> or storing critical files in the cloud. No. I refuse to let my vacation photos, hard drive images or hell, even my wallpaper collection to leave my house. No. I refuse. My data is my data.

Encrypt it? There are some strong algorithms that allow you to enjoy the most highly secured, fault-resistant buildings on earth without letting the capitalists that own them view your data even if they were to hook your storage up to their entire GPU/ASIC/etc farm.

What if your house burns down?

i keep a copy of my most important files in a safe deposit box

I keep my FLAC collection in a safe deposit box. More important files get stored on HDDs and MLC flash. Storage is cheap.

Nothing stopping you from putting NextCloud on a Raspberry Pi

I don't know about the mac, but I tried to do this for the wife/kids i-devices, and it was an utter failure. 3rd party backup apps are at a huge disadvantage on ios because apparently they can't run in the background long enough to keep things synced unless they also require _GPS_ service notifications (or at least that is how the nextcloud app works around it). Which in turn eats even more battery to work around ios's inability to flag apps as trusted system services and provide extra warnings/whatever during install (or app store acceptance might be a better plan).

> or

There was another simple option. You're free to use that one.

> You just have to be damn sure that you back it up correctly.

Most somewhat serious NAS appliances offer being a TimeMachine server which is more than enough for most people.

These days I don’t understand why people don’t sync their important files to cloud. Dropbox has a free layer that’s perfectly fit for a theses.

Maybe because not everyone likes to share their most important files with a third party? Or because of providers scanning files or stuff like: https://www.reuters.com/article/us-apple-fbi-icloud-exclusiv...

If you seriously consider a state actor to be a threat, then you likely make many other compromises to your ability to conveniently perform disaster recovery.

There is not much overlap between these use-cases:

* I need an easy way to recover my family pictures because I didn't make backups

and

* I have state actors interested in my data

If you really have state actor threats, you probably like that M1 Macs are very hard to perform data-recovery from.

If you are a state actor you probably love M1 Macs. You literally can't restore them without phoning home.

Everone should consider state actors as a threat because they are. They maybe are not specifically after you. But every now and then some big box gets popped leaking all kinds of personal information (including yours) and the actors are probably state sponsored. Also it is not state actors scanning your data, many cloud providers do so.

If you have a state actor as threat model you probably want to remove iCloud backups (see link):

"Apple’s iCloud, on the other hand, can be searched in secret. In the first half of last year, the period covered by Apple’s most recent semiannual transparency report on requests for data it receives from government agencies, U.S. authorities armed with regular court papers asked for and obtained full device backups or other iCloud content in 1,568 cases, covering about 6,000 accounts."

Your concerns are not unfounded, but there is a very significant functional difference between being generally concerned about data privacy posed by any third-party bad actor, and having a threat profile that includes specific state actor threats.

If you think it is a realistic scenario that the US would serve a warrant against you, then yes, your iCloud backup will probably be disclosed to the authorities. But if this is a real concern of yours, then you probably have much bigger things to worry about than whether your SSD is soldered to the board or not.

It feels like you're using "state actor" as a motte and bailey. It usually means targeted espionage affecting very few, with strong motives for why security services are interested in them specifically. Whereas that article is ultimately talking about a dragnet surveillance system, which is technically a state actor but affects literally everyone.

I don't mean anything special by "state actor", I plainly mean: actions taken by a state.

Whether you consider those actions to be a threat would depend on an individual person's or organization's assessment of the impact of those actions.

Using your loose definition of "state actor" means that every single person who uses mainstream cloud solutions has "state actor" as a threat, whether they acknowledge it or not. It's not just the occasional person super focused on security as you're making it out to be - in fact people less in touch with security have a larger threat from their system being hijacked and used to distribute contraband bits, thereby running afoul of cloud scanning.

No it doesn't mean that, because threat assessments consist of multiple variables, including the footprint, probability, and impact of a potential threat.

That's a fancy way of saying "I've got nothing to hide".

No it's not. Consider the risk that a street criminal could throw a rock through my window and steal my computer off of my desk. Do I put bars on my window? No. Why? Because I have nothing to hide? No, because I don't consider that attack vector to be a high enough priority that I need to take steps to mitigate it.

Threat assessments are purely an engineering exercise. This is orthogonal to the ethical, philosophical, and political discussion of civil liberties. If you have ethical, philosophical, or political concerns about this topic, those are something to be addressed in their respective realms. It is naïve to believe that engineers have power to protect anyone from abuses of civil liberties. Apple hardware engineering team is neither capable nor equipped to be addressing these concerns in their designs.

Then use tarsnap or Arq or some other client that encrypts locally and backs up the encrypted files to the cloud.

That’s fair but if someone is concerned about it they should invest into backups. It’s unconscionable to entrust days of someone’s work to a single device.

Seriously. It's 2021, you should have a remote backup. Use backblaze, tarsnap, anything. SSD failure is one of a thousand ways you can lose access to files on your laptop.

You always do backup (another hard drive), GitHub (these days free private repro), Dropbox, iCloud, … etc. Totslly rely upon a hard disk … a big lesson to learn.

> Also, it is not possible for an SSD to decrease your usable storage space over time.

It totally could be possible. For Windows/Linux, you could have a daemon that creates a 'ssd_wear' file which grows in size as the SSD wears out, and messages to the SSD which physical blocks it occupies. Mac could obviously do a deeper integration directly into the OS.

> all of your blocks are on the verge of failure

A failed block still stores some information (in an information entropy sense). It's totally possible to store one blocks worth of information across two, four, or eight failed blocks (with extra error correction information).

Combining these two techniques, an SSD should never fail. Instead it slows down and gets smaller. Sadly as far as I'm aware, no consumer drive vendor has implemented these.

You should read up on Zoned Storage for NVMe SSDs, and some of the more recently added error handling features like Get LBA Status. The gist is that it is only practical for the SSD to be in charge of determining what has failed-not the host, and that retrofitting NAND retirement into the traditional block device/LBA model is not worth the trouble when there are also other reasons to migrate to a different, more flash-friendly abstraction.

I used to work on SSD firmware. I did implement something like the above for a custom solution for caching. When the flash storage is running low on space, it reduced the reported storage capacity and notifies the host which will need to trim some data to bring it below the reported storage capacity. The striping of data across multiple blocks also has been implemented. You could fail an entire NAND die and it would still function though a little slower assuming you still have enough spare blocks.

>> It also wouldn't win you any significant increase unusable lifespan, because when your drive starts retiring large numbers of blocks, all of your blocks are on the verge of failure.

That's not true. Blocks degrade when written to. Most files are written once and read many times.

You're forgetting that SSDs use virtual addressing so writes are distributed relatively equally over the whole drive. Even data that is written once and never changed logically will probably move around on the SSD due to garbage collection. The virtualization algorithms are kept relatively secret, but I assume good SSDs will move full unchanged blocks around occasionally to distribute write load more equally on the physical blocks.

Yep, wear leveling actually works. On some SSDs, there are SMART indicators to tell you the average, max and min block erase counts, so you can see for yourself that the erase counts are all about the same across the entire drive.

>> I assume good SSDs will move full unchanged blocks around occasionally to distribute write load more equally on the physical blocks.

I hadn't considered that. Make sense. Better to move never-changing data to blocks that are still good but not worn out so those available writes can be utilized.

> Though keep in mind that "failure" here is defined as "only able to retain data for one year" for consumer SSDs

So the M1 Mac lasts a year before the owner has to shell for an apple authorised repair to replace the disk ? Me thinks that may not be the story for apple or dell

What am I missing

> In practice, exhausting NAND write endurance is usually not what causes SSDs to fail in the field.

+1. The part that fails on consumer SSDs is typically the controller.

https://tesla-info.com/blog/tesla-mcu1-emmc-failure.php

Do you have a point? Linking to a story about an embedded system that experiences flash wearout is not really relevant here. You're trying to make a comparison against a far smaller and lower-quality storage module than typical consumer SSDs, subjected to a vastly different workload. There are no useful conclusions to draw about consumer PCs from that case.

I think you are using old information. Any modern SSD should be good for a few hundred to several hundred TB and may reach a PB. It would be very hard to use a personal computer that heavily. Most people would take 20 years or more.

"Any modern SSD should be good for a few hundred to several hundred TB and may reach a PB."

Actually, even more than that - here are some examples:[1]

Intel SSDSC2KB019TZ01 ("S4520") with 8.8 PB of lifetime writes

Intel SSDSC2KB019T801 ("S4510") with 7.1 PB of lifetime writes

Seagate ZA1920NM10001 ("Ironwolf[2] 110") with 3.5 PB of lifetime writes [2]

Samsung MZ-76P2T0BW ("860 PRO") with 2.4 PB of lifetime writes

[1] https://www.rsync.net/resources/notes/2021-q3-rsync.net_tech...

[2] https://twitter.com/rsyncnet/status/1402397708329967618

To be fair, those are unusually high-end products compared to typical consumer SSDs, and three out of four are enterprise drives where endurance is measured according to different criteria that inflates the number a bit further.

Those numbers aren't out of reach for consumer SSDs either.

Video editors have workloads that can write a ton for each project in 4k and 8k resolutions, if its pro res even more. First thing they do is grab the largest fastest storage they can work off externally.

That is a fairly write-heavy use case. But SSD endurance is measured in drive writes per day (over a typically 5-year warranty period). Rewriting the entire contents of your drive every workday for 2.5 years is only enough to put a typical consumer SSD out of warranty; actual failure due to write endurance will come much later.

Right -- but work externally is key here. Sure, you often have some stuff you'll do locally, but having external disk arrays is the name of the game if you're doing large volumes of video, not just because you want to be able to have a backup if your internal dies, but because lots of other people often touch your stuff and it just fits a workflow better to use fast networked storage when possible.

I finished the life of a SSD in a single year of regular use; it was great to find it should not happen, but it did.

What drive? What was the end of life behavior you observed? Catastrophic failure, or SMART data showing wearout?

Not the parent but I had this happen to me. It was a top of the line Samsung SSD at the time. Catastrophic failure. Was working fine then one day I rebooted and it never came back. I usually plan for failure of each drive every 2-3 years and buy them on Black Friday. But if that took out my entire 3.5k-4k m1 max laptop I'd be rather disappointed. It's $200 to replace a SSD.

So, that failure mode makes it pretty much impossible to conclusively attribute it to wearing out the NAND write endurance. You probably hit a firmware bug or failure in a different SSD component, otherwise you would most likely have been getting warnings from your OS about impending drive failure based on SMART data.

SSD wrote endurance is an unavoidable ticking time bomb. But it is nowhere near as unpredictable or imminent as many people assume. It's a failure mode that doesn't sneak up on you, and usually takes real effort to trigger. Other failure mechanisms are much more important.

I've had two SSD's fail on, both were "here today, gone tomorrow" type failures without warning.

Granted, been a while, but I do nightly full-disk backups due to the first such experience (so was well prepared for the second).

Intel 535. I was showing some alerts and the Intel SSD tool said the remaining life was ~ 5%, then it got to zero so I replaced it.

That model had a known issue with excessively high write amplification, for which Intel published a firmware fix: https://www.intel.com/content/www/us/en/download/18821/26452...

In general, you should be aware that the "life remaining" indicator may merely be the "warranty remaining" indicator. If the unused reserved block count isn't approaching zero, a drive can usually keep operating long past the official write endurance rating. However, in addition to the above bug, Intel is also known for putting a kill switch in their SSD firmware tied to the "life remaining" counter, so you might have actually experienced the SSD going read-only when that hits zero.

You're forgetting swap. Not to mention professionals in domains like photography, design, video, etc. who might chew through big files frequently.

> You're forgetting swap.

8 Gb should be enough for everyone :)

Swap and other system activity is factored. Most heavy workloads wear other parts of the system as well. As another user pointed out, truly heavy storage and scratch system based workloads want to use large externals anyway, for other reasons.

Yes. Don't RAM-max at your own peril.

That's not how SSD wear works because nobody completely writes to their SSD, erases it, and writes again.

You can't wear-level against blocks that are in use, only blocks that are free.

If every person I've helped socially and professionally is any indication, people keep their storage anywhere between 80% and 95% full.

At 95% full, all your write cycles are focused on just 5% (plus spare provisioned blocks.)

Think about how browsers consume all available memory and then readily push the system into using a lot of swap. I have to restart my browser regularly because of how much ram it uses even if I've closed every tab.

Think about how whole-system file content indexing is the norm, and all the write operations that occur every time a file is added or updated.

Think about how a lot of applications are just electron apps and they routinely update themselves.

Think about all the filesystem metadata changes.

Etc.

> You can't wear-level against blocks that are in use, only blocks that are free.

It only takes a bit of arithmetic to prove that this cannot possibly be true, or else all of those 95%-full drives would have long since failed, and QLC SSDs would have been a disaster and never would have been adopted by system OEMs that provide multi-year warranties. (And if it were true, then SSDs prior to TRIM wouldn't have been viable.)

You can include static data in wear leveling, and all SSDs do. It does lead to some write amplification, but that's already factored in to a drive's endurance specs.

I did not say SSD wear works that way. All of that is factored into the usage I cited—-swap, system writes, etc. do not wear the drive as much as you appear to think for most users.

How many SSDs have you worn out? I've never managed it.

I've had hundreds of enterprise SSDs fail on my servers, but only a couple had any pre-failure indicators in SMART, including write volume which was pretty low. The post-failure indicator was almost consistently no longer visible through the disk controller.

Failure rate was much less than enterprise spinning disks, but at least with the spinners failed, you could still see them and get most of the data off, if you needed to. And they had reliable pre-failure indications.

TLDR: take regular backups and plan for storage to fail (which for soldered storage means plan to get a new device, I guess)

If your running that big of a shop, I would call your channel partner/support group and bitch until they figure out whats failing. The usual expected failure modes for SSD's should be that they go into read only mode. If that isn't happening its either a firmware bug (probably) or a controller/vr failure which could be a process/heat/etc type of failure.

Particularly for "enterprise" equipment the expectations isn't that it fails, but that the failures are more graceful and understood.

So complain... that is why your paying the "enterprise" tax.

This was in managed hosting, we'd open a ticket and get teh SSDs replaced quickly, which was good enough for us. For the most part, the failure rate was about 10% of the rate of the hard drives, and concentrated in the systems with many disks, which were setup to handle disk failures, so it wasn't a big big deal.

Although we did get a couple of bad batches on our single disk systems, almost all of those failed within 2 weeks of install and we had the rest replaced.

I also suspect firmware bugs, but it wasn't convenient to upgrade them, so who knows. I did have to upgrade some of the same disks on single disk systems with low usage; the disks would sometimes timeout for no reason and confuse some of our OSes. The upgrade procedure was more or less PXE boot the manufacturer upgrade utility and hope it worked with whatever disk controller was there (eta: 5 minutes) or let the hoster firmware upgrade script run and hope it upgraded the ssd firmware and didn't mess anything else up (eta: 40 minutes per run, probably run it twice).

Apple SSD module failures on portable Macs are quite common. A buddy has a computer repair shop and they do brisk business replacing them. Mine failed about 5 years in.

SSDs are wear items. Especially when they're quite full, it is very easy to blow through spare writes and brick the drive.

There was a problem with some of the SSDs where they corrupt themselves on powerloss.

That’s a totally different reason, and afaik, they should be using proper caps to let the controller write/flush their buffers.

My last MacBook Pro was 7 years old, almost always full, and still works fine. Granted I’m not a video editor, but I had quite some write cycles.

Did those modules fail readonly, and the SMART data showed that their write limit had been reached?

Or did the entire module electronically fail, as distinct from the SSD "no more writes" case?

One, small capacity X limited cycles = short life span.

Limited write cycles will produce data corruption, but what the article explains is that Apple is using the Flash in M1 macs in a similar way to other ARM devices, that is, they have the firmware there.

I don’t know how much time do you expect your mac to last, but write cycles on your firmware should not be much of a concern.

Right, but I get the impression that the firmware vies for space with the rest of / et al.

Sooo that means the wear leveling had better be ahead of whatever lifetime writes you throw at the system, or...

Easy to account for in drive firmware. Not to mention the file system if that were actually being used. Way back when, master boot records were duplicated at know locations for kinda similar reasons.

> So no, what damages NAND does not damage the rest of the board. The more you write and swap to flash, the faster it wears out, taking your Macbook down with it when it dies.

Except as explained by the parent, when the SSD dies on an Intel Mac it can be recovered via Internet Recovery via EFI.

Only way to brick that it is to have a bad flash onto the EFI ROM.

On Apple Silicon when the SSD dies, it cannot be recovered as the firmware is also on the SSD and remains bricked which that is that.

Heh, I wrote pretty much exactly the same thing, should have read all the comments first.

that's true, but it's far more likely for the mainboard to get soaked/smashed than the NAND running out of write cycles.

> That's just reads like BS.

Because it partly is. It's just a lame defence / justification of soldering SSD, RAM and other parts, the major advantage of which is to make modern electronics more complex, hard to repair and prevent upgrades (i.e. "planned obsolescence"). If SSDs were reliable beyond 5+ years, manufacturers would be highlighting that and even offering guarantees for such a period. (Hell, most RAM today come with "lifetime" guarantees, even though RAM modules too fail occasionally).

The modular design for computer hardware was chosen for a practical reason - that hardware can, and do fail. While electronic manufacturing has advanced a lot, minimizing failure rates to a great extent, the pros of replaceable modules absolutely still outweigh any of the alternative hardware designs so far.

Except soldering everything on can give you HUGE performance boosts. Like, laptops come with soldered-on RAM that you can't get in a DIMM configuration because the soldered on stuff is so fast it needs a fast path to the CPU, which DIMM slots can't provide. And now with Apple building all system RAM directly into the chip die, it's just a whole new ballgame.

So you can either go modular and have a slow PC, or you can have decent performance and just accept that everything is going to be soldered on and irreplaceable.

This argument doesn't work with Apple. They built laptops that could literally just use 50% of their CPU because of insufficient cooling. Not too long ago they released a Macbook with a CPU cooler that doesn't even have direct contact with the CPU. And when LTT modded the cooler to fix it they realised the performance was still not better because the power delivery is insufficient as well.

Also our PCs are not slow, modular or not. In many cases hardware is not the bottleneck.

Maybe that CPU cooler wasn't really a CPU cooler after all, if it didn't have direct contact with the CPU.

Maybe the fan was more of a case fan and it simply ran air through the whole machine INCLUDING the CPU heatsink, which is AFAIK exactly how it worked.

Sorry, what? Citation please. The speed lost moving a DIMM from a 3cm path-to-CPU to a removable DIMM slot 6cm away is infinitesimal. You would not notice it.

For laptops, soldered vs socketed DRAM is mostly about power, not performance. But it is true that soldering DRAM makes it possible to reach higher bus speeds than are practical to achieve through a DIMM slot; this is why GPUs don't have upgradable RAM.

Laptops that have soldered RAM (including Apple) do not come with overclocked RAM though. I don't even know of any system with soldered RAM that would let you try pushing the speeds in practice, so this advantage is never leveraged.

But we do know how far DIMMs can go. Mainboard memory trace layouts have gotten ridiculously optimized. DDR4-5200 is a totally "normal" XMP spec now, there are even 5400 kits out there I think.

> Laptops that have soldered RAM (including Apple) do not come with overclocked RAM though. I don't even know of any system with soldered RAM that would let you try pushing the speeds in practice, so this advantage is never leveraged.

This is unsurprising given what I already pointed out about power being more important than maximizing performance. Sure, you can overclock the memory controller on a desktop platform and with sufficiently expensive DIMMs and probably a bit of an overvolt for the memory controller/uncore you can reach higher bus speeds than any laptop CPU (though still not anywhere near as high as GDDR used with high-end mobile GPUs).

But take a look at the fastest memory speeds available through SODIMM slots vs the speed of LPDDR4x supported by the same CPUs. Within the constraints of laptop power levels and using standard grade memory parts rather than requiring premium binned chips, soldering currently corresponds to a 33% performance advantage.

Who is saying the soldered ram is never overclocked? You cant overclock changeable ram by default because you don‘t know what the customer will use. But by soldering the ram you know you will use very specific parts, so you could (maybe they do?) tune the settings because it does not have to work with thousands of combinations.

The speed lost due to distance would be infinitesimal, but is that the only speed loss there would be? I'd expect that parasitic inductance and parasitic capacitance would be higher in a socketed system which would impose speed limits.

I've read that nowadays because they effectively have 8-channel or 16-channel RAM, there would just be too many connections to make them removable.

Imagine 8 DIMMs next to each other, and then fitting that into a laptop form factor.

Channel != DIMM. Many small channels is just how LPDDR works, as opposed to DDR which has one big chungus channel. Well DDR5 has now split it in two. The total number of traces is the same.

I have seen many comments parroting this line but none of you provide a source to back this assertion. Your claim that modular PC's RAM is slower is quite ignorant 'cos the bandwidth for the RAM bus has been consistently increasingly (currently DDR5) and remain the same whether it is soldered or modular.

Reply on news.ycombinator.com