Update Firmware of Samsung SSDs in Linux
blog.quindorian.org
blog.quindorian.org
edit: Windows PE won't do .msi, but .exe should work: https://docs.microsoft.com/en-us/windows-hardware/manufactur...
I remember in the Windows XP era, there was a third-party piece of software called BartPE.
With BartPE you could create a preinstalled environment with WinXP Home even.
Here is a page someone wrote with some details about BartPE: http://www.optimizingpc.com/miscellaneous/manual_bartpe.html
Wikipedia:
> As with Windows Preinstallation Environment, BartPE operates by loading system registry files into RAM, and not writing any registry changes back to boot media. Thus, neither operating system requires an operational hard drive or network access. This also allows them to be run from non-writable media such as a CD-ROM.
> […]. The Bart PE Builder application interprets and condenses files from a Windows setup CD to create the BartPE installation. It can built from a Windows XP Pro or Home Edition CD, or from a preinstalled Windows XP version (without a CD).
From the http://www.optimizingpc.com/miscellaneous/manual_bartpe.html page linked above:
> WinPE (Microsoft's tool Windows Preinstallation Environment) is used by system builders to install and configure the Windows operating system on new computers. WinPE is commercial software and only available for system builders. Bart Lagerweij saw the advantages of this tool and decided to develop his own PE, named BartPE.
A year later the drive developed some bad sectors. It was still under warranty so qualified for a free replacement. All I needed to do was get an RMA number and ship the drive to them.
But to get an RMA number you had to run their diagnostic software which, if it decided the drive really did have a problem, would issue the RMA number.
That diagnostic software was Windows only.
I ended up temporarily moving the drive to my Windows gaming box to run the diagnostic. I'm not sure what would have happened if I had been a purely MacOS and Linux household.
Passthrough to a VM, though I suppose the Windows license could be an issue.
The big issue with a VM approach in this specific case was that this was before Macs switched to Intel processors. My Mac was had PowerPC.
A VM might have worked on my Linux machine, although my recollection is that back then (early 2000s) VM passthrough was not very sophisticated. I remember trying Windows VMs for a few firmware things that involved proprietary vendor commands and finding that it was hit and miss.
Ref: https://www.howtogeek.com/244678/you-dont-need-a-product-key...
You get a nag every now and then, and the desktop background keeps resetting to black. Can't remember if there's a watermark as well (10 has a watermark).
Microsoft stopped giving a shit about piracy from consumers. Its a far cry from the days when windows cost €300.
I had to get the state consumer fraud bureau involved before AMD would honor their promotion.
I tried to explain that I use it with a NAS and I cannot install Windows on it just to do such diagnostics test. No luck to convince the support and I had to hack some pieces of old hardware together just to run that tool. Funny enough the tool was able to tell only something like "the drive is broken". I had a much more detailed report from NAS though, with details from SMART but that was not accepted. Basically they didn't believe me and asked for a confirmation from their tool, like it's more trustworthy.
Of course I'm not going to buy any Seagate after such experience.
There may be a difference here between the consumer drives and the enterprise drives, or maybe they changed the system to not require it in any case now.
I was not an enterprise or even a business customer anyway, so they had that procedure. The process may change since that moment, I just checked it was in 2019. Maybe I should give them another try
Windows shouldn't be a requirement for getting a faulty product replaced.
In my case, I was the only one in the company with a Thinkpad, so my process for upgrading the docking station firmware was:
"dd" off my drive to another box. "dd" on the copy of the disc I had saved when I got the computer. Boot it and set it up. Run all the updates. "dd" my drive image back onto the box.
The company I'm at now prefers buying Dell, but I had special requested that Thinkpad. It was kind of disappointing. It had two batteries, which made it slim, but sometimes one of them would be flaky. But not reliably flaky, so sometimes it'd just power off, but most of the time both batteries reported tons of life. And the docking station was basically useless (don't remember the issues, I just never was able to get a satisfactory docking experience from it). It had the ultra high res screen and that was nice.
For me, it was less than 10 days ago. The battery on my Thinkpad X1 (that has 5 years onsite warranty) stopped charging. I run Debian, and had got rid of the Windows partition entirely. They demanded a battery report from Windows. It took a long time, but I was able to convince them the report from their extended battery test in the BIOS saying the battery had failed sufficed, and the battery was replaced.
The new battery lasted about 1 month, then it developed the same fault. Same story - after a day worth of arguing, they agreed to replace the battery.
That battery lasted 3 days, then developed the same fault. The story went the same way, right up until they said they fix it by replacing the battery to which I responded "I don't think that's a good idea". (By this time I had developed suspicion the USB-C chipset was somehow killing the battery.) The next step was to replace the motherboard until I filed a Windows battery report. No amount of pleading "hey, you have a report from your own BIOS saying the battery is faulty, it doesn't charge while in the BIOS when no OS is running, and it should charge while off when _nothing_ is running - what possible influence could Windows have on this" worked.
They sent a link me what they said was a "live Windows" USB recovery drive. It was a Windows program. I was able to run it Windows in an kvm instance. It wasn't "live Windows" of course - it was a "wipe SSD with a fresh install of Windows" USB. Not being keen on wiping all my work and risking a backup not working, I bought a new SSD, and let it wipe that. It took about a wasted day (I'm a contractor) to install Windows, let it upgrade itself, and then install the latest Lenovo drivers for everything (which they insisted on), then run the battery report. During the time the laptop started freezing - but it turned out unplugging the 4K USB-C Philips monitor fixed that, adding more evidence to my theory the USB-C interface was on the way out.
The motherboard is scheduled to be replaced today. I'm betting they won't bring a battery and replace the one the USB-C chipset has killed. They go to all this trouble to provide good diagnostics in the firmware, and then don't train their staff to use it. I'm sometimes gob smacked by just how much time these companies cost themselves.
Samsung is to blame for this.
[0]: https://fwupd.org/
In hindsight, I found out that Satechi is now largely reselling cheap Chinese reference designs that can be had from AliExpress from 10-30 Euro/USD, repackaged in different casing [1]. Since they don't actually seem to control the firmware, they are just reshipping half-working Realtek firmware update utilities.
[1] https://overengineer.dev/blog/2021/04/25/usb-c-hub-madness.h...
At least with Toshiba/Dynabook, it's vaguely possible for a techie to pull apart their firmware update process & hack something together that doesn't require Windows any more...
The only way to get FW updates is through Lenovo's proprietary tool which only works on Thinkpads, even though my non-Thinkpad machine has the exact same SSD. What a shit show.
I guess I'll just have to stay on the FW it came out of the factory.
I don't understand how datacenter SSD manufacturer can so blatantly ignore Linux.
Now consider doing this at scale, in a datacenter, or just some office that has a lot of these drives. Ouch.
Now I really have to check if I get firmware updates from Lenovo for all the hardware in my Thinkpad... I always assumed I do!
Those who know of such HDD/SSD vendors (esp. those supporting lvfs) please chime in.
If you mean Sammy, they do provide support for datacenter SSDs:
https://aur.archlinux.org/packages/samsung-ssd-dc-toolkit
I don't think Kingston offers anything in that niche. I know very little about that, though.
I don't want to buy SanDisk SSDs anymore because of this, but I guess every SSD manufacturer is the same when it comes to firmware updates.
One day a long time ago Dell actually sent us the cheapest server they made, R200 or R300 to flash disks in because we had so many to do.
Err no, that's when you boot from USB stick. Or do it from initramfs. Or anything, other than modifying the firmware of a drive you are running from.
Anyone sane implementing the firmware flashing algorithm will either:
1. Just reject all IO requests from the point of the firmware update onwards. The system will see the drive disappear, but since you are using a journaling filesystem, it will be like you just pulled the power, and no data corruption will happen.
2. Complete all IO requests before the update, then update, then continue requests as before.
3. Save the firmware update, but only apply it at next powerup of the drive.
Netapp filers do this; they update disk firmware online by installing it and then just rebooting the drive. The RAID controller/software handles the momentary interruption with zero issues, but even without redundancy an OS can handle a drive disappearing for a few moments.
If the power cycle is controlled, then the kernel can be reasonably sure that the re-appearing device is the same one. File handles wouldn't really be affected as long as the mounted filesystem can resume IO after the device reappears.
Most likely you'd still want to use a RAID scheme and/or data checksumming as an extra layer to ensure data integrity though.
Firmware Updates (0x16): 3 Slots, no Reset required
A quick search did not result in any official changelog, but some forum entries (e.g. [1] and [2]) suggest that random read + write performance under windows seems to have been improved.
[1] https://www.elevenforum.com/t/samsung-980-pro-firmware-updat...
[2] (in german) https://www.deskmodder.de/blog/2022/01/26/samsung-firmwareup...
Upgraded from to 3B2QGXA7 to 5B2QGXA7.
For most applications I can't see why you'd really need to update your SSD firmware, unless the manufacturer messed up their wear-levelling or something like that in the release firmware.
But when you build a Hackintosh with a recent version of macOS this exact TRIM behavior results in boot times of several minutes (compared to a dozen of seconds or so on a SSD with a better TRIM implementation).
I’m not saying Samsung will ever fix their TRIM implementation, but in theory this would be fixable by a firmware update.
And don’t get me even started on the crap that TRIM is, why we shouldn’t need it in the first place, and how convoluted its development was.
Then for something high performance like an NVM.e SSD, you could potentially gain a lot of performance, or lose it, depending on the manufacturer's goals for stability vs speed.
After a couple of decades of updating firmware on things whenever possible (including my car which had me leave the engine running sitting in the car with the window down for 2 hours in the winter), I'm yet to have a real issue.
That said, I also have the tools and skills to dump flash chips and would suggest doing that to anyone before an update of certain devices, or, you know, just don't do it, as you suggest.
I used to be in the same boat, up until a few weeks ago when I updated the BIOS on my new work laptop. It's been a shitshow ever since: it takes around one minute to POST, the webcam isn't detected anymore, it doesn't go to sleep.
And the best part? "Because this update contains security features, it cannot be rolled back".
---
edit to name and shame: HP EliteBook 845 G8
This is a real issue though, there is the risk of failure during the procedure, less so than it once was, but then the risk that the update actually performs worse than the previous version.
The latter I don't know how we resolve.
I was usually able to this, and my other HP machines propose to do a BIOS downgrade. But in this particular case I haven't managed to do a rollback, even with a "BIOS recovery", whatever that is.
Perhaps they can get in touch with your supplier to pass a complaint about the poor update so they might be able to pass it on to their devs and hopefully fix the issue?
I would try have your org reach out to the middleman and see what they can do, suppliers, resellers and partners etc can be quite helpful in encouraging manufacturers to pay attention.
Generally speaking though, outside of printer ink, which I've always considered to be a racket, this sort of thing is perhaps less of an issue.
That's not to say people like Sonos and others haven't used firmware updates to brick otherwise perfectly functional devices in order to force consumers to "upgrade" before now.
In any case, I treat this as a high risk operation; no HDD/SSD firmware flashing without making sure the backup is working...
There was even once a bug which, after X hours of SMART recorded usage, would cause the SSD to lockup every hour thereafter: https://www.reddit.com/r/buildapc/comments/oh18a/attention_c...
SSD firmware issues can be very serious.
But I meant "faulty firmwares due to bad firmware code, needing updates".
Sometimes an update just fixes some incompatibility with some specific hardware. If you are not using that hardware then there probably isn't much benefit from updating. Well, at least if you are either sure that you won't need to use that incompatible hardware later, or are sure that if you ever do need that hardware you will still be able to update.
Sometimes though an update fixes a more pressing defect. For example I updated the firmware on my Samsung 840 EVO drives several years ago because of this issue that caused significant performance degradation [1].
Another example is a firmware update I recently applied to my AKiTio Thunder3 Quad Mini Thunderbolt storage enclosure. The issue there was that when MacOS put drives to sleep occasionally the enclosure would report that the drive ejected. Getting the drive back seemed to require ejecting the other drives in the enclosure and then power cycling it.
This was annoying, but easy to work around by setting the OS to not put drives to sleep. All my drives were SSD so keeping them awake didn't use too much extra power. I ran like this while waiting for them to release a firmware update. After a couple years of waiting I stopped checking.
After a couple years or so of not checking, I remembered and checked and sure enough there was an update specifically for this issue. It was a bit of a pain because they had discontinued the Thunder3 Quad Mini a while back, and some time after that had stopped updating their firmware updater for it. It only ran on MacOS up through 10.15.7, and I was on 12.something.
I ended up using a USB SATA dock and an old spinning disk that was lying around to install MacOS 10.15.something on that and boot from it so I could run AKiTio's updater. Worked like a charm. It would have been a lot more annoying if I hadn't happened to have had the USB dock and old drive sitting around.
[1] https://www.anandtech.com/show/9196/samsung-releases-second-...
I simply couldn't believe that their tool required a PS/2 keyboard, when most modern Mainboards no longer have such a port.
In any case, updating the firmware didn't fix the issue that my 980 Pro NVMes (all 4 of them) only have a write speed of between 400 and 500 MB/s (read is at ~6.7 GB/s), using Ubuntu 20.04.
A WD_BLACK SN850 NVMe manages to write at 3.8 GB/s in the same slot (Ryzen 9 5950X on ASUS Pro WS X570-ACE)
BTW: I faced the same keyboard issue a month ago when I wanted to use another Samsung tool to erase a Evo 870 SSD for RMA with the Secure Erase feature.
In order to see spec sheet read/write speeds you can/should bypass the filesystem and the buffer cache. On Linux you can write directly to the device node (make sure to enable O_DIRECT and make your write sizes big enough).
EDIT: added a quote to clarify context
dd if=/dev/zero of=~/testfile bs=10000M count=1 oflag=direct -> 1.4 GB/s
dd if=/dev/zero of=~/testfile bs=1000M count=10 oflag=direct -> 3.7 GB/s
dd if=/dev/zero of=~/testfile bs=100M count=100 oflag=direct -> 3.7 GB/s
dd if=/dev/zero of=~/testfile bs=10M count=1000 oflag=direct -> 4.0 GB/s
dd if=/dev/zero of=~/testfile bs=1M count=10000 oflag=direct -> 3.6 GB/s
nvme set-feature -f 6 -v 0 /dev/nvme0n1
dd if=/dev/zero of=~/testfile bs=10000M count=1 oflag=direct -> 412 MB/s
dd if=/dev/zero of=~/testfile bs=1000M count=10 oflag=direct -> 556 MB/s
dd if=/dev/zero of=~/testfile bs=100M count=100 oflag=direct -> 553 MB/s
dd if=/dev/zero of=~/testfile bs=10M count=1000 oflag=direct -> 482 MB/s
dd if=/dev/zero of=~/testfile bs=1M count=10000 oflag=direct -> 267 MB/s
The values I got in my first post were obtained through Ubuntu's "Disks" application which has a benchmarking dialog.
If flush works properly, a volatile write cache in the SSD is good for you.
BTW, I followed the wget & cpio route from the how-to and ran the "fumagician" binary just fine on the headless box over ssh, no need to PS/2 anything.
Such a standard could define capabilities (firmware slots, online update, power cycle required, etc). The OS could handle it from that point.
There has to be a better way without the need to hack everything
https://aur.archlinux.org/packages/samsung_magician-consumer...
There's still a better way where they do think so — "enterprise" SSDs:
https://aur.archlinux.org/packages/samsung-ssd-dc-toolkit
Vote with your pocket, support vendors that do care.
At least solaar exists for Logitech stuff. Too bad it's still not something Logitech provides, but the reverse engineering work of some volunteers.
https://wiki.archlinux.org/title/Solid_state_drive/NVMe#Gene...
It worked for me with SK Hynix Gold.
7z e Samsung_SSD_970_EVO_Plus_4B2QEXM7.iso initrd
7z e -so initrd | cpio -idv "\*fumagician\*"
sudo ./root/fumagician/fumagician> No supported SSD detected for Firmware Update
It also doesn't work if you flash the ISO and boot "properly" as Samsung expects you to do.
It's a pain that sysadmins knows all too well when deploying a corporate Windows image through SCCM/MECM, and why they need to often update the embedded drivers in the Preinstallation Environment.
It's also an NVMe drive. I have a pretty vague idea of how PCI-Express drives work, but don't they go directly to the CPU without any storage controllers? It's more of a IDE/SATA/SAS/whatever thing.
Maybe you could turn that off in your BIOS settings if you don't specifically need the feature.
I get why samsung doesn't want to get sucked into this. It does suck though, at least the ISO should work.
The much more likely reason is there is a Windows user in charge of the team. Someone who doesn't know what they don't know.