Raspberry Pi – UASP, Trim, and Boot Performance via USB
jeffgeerling.com
jeffgeerling.com
I used to run that like it was religion, watching the blocks all move together and change color, and knowing the machine would run just a bit faster after it was all done was a bit of magic.
It's funny how, in these late-stage Moore's Law days, a 10-year-old PC is old certainly, but not as hopelessly outmoded now as the 5150 was then.
Samsung cards tend to outperform other brands in that price range.
I like the Arcanite for large files and basic storage needs, or an NVMe + USB 3.0 adapter for the best performance for a reasonable price.
UASP can speed up operations by up to 50%, and many SSDs and high end flash drives do support it. TRIM support is less prevalent, and you often need to manually enable it (there's a guide linked in the post).
Weirdest finding: the Arcanite flash drive firmware indicates it supports trim, but when I tried running fstrim something blew up in the drive, and I could no longer read/mount/initialize it from Mac/Linux/Windows (I had to buy a replacement to finish the testing).
Also, I'm really curious if your only problem with that Arcanite was only that it was a defective unit. If anyone reading has any other ideas for diagnosing that problem then I'd love to know them.
Great guide, Jeff. Really enjoyable (and informative!).
Doesn't really depend on the Pi hardware at all, it depends on the kernel driver (kernel must be new enough) and the USB storage device (does it implement UASP in addition to legacy bulk-only-transport?).
https://www.devever.net/~hl/usbuas has a great explanation of the nitty-gritty details.
Like many - pre Pi4, was wishing for a SATA port, but overall, if they did do some upgrade to the Pi4 in another B+ or 5 flavour - then I'd be more interested in a faster USB flavour, like 3.2 or USB-C. Reason being is that it would be more flexible and those who want SATA or NVMe would be able to tap into that. Certainly more flexible in usage and that does seem to be key.
These devices are usually meant to be used as a carrier that’s only connected to a computer to load/off load data. Long-term operation probably isn’t a priority when developing them.
So indications are good, and the drives in this case was a WD SSD, 3x Toshiba bus powered drives and an ancient 320gb usb externally powered WD affair. So a but of a mix. Though I did have it plugged into a UPS, which like all things tech - kinda helps smooth out the random chaos of electricity and tech.
We have a few hundred Raspberry Pi in operation at customer sites, running 24/7. Back when we started our business we used generic Micro SD (can't remember the brand). They would fail after 2-6 months. This was a big hassle. We have customers all over Germany and would need to ship them a complete RPI as a replacement, as the customers are usually not tech-savvy enough to just replace the SD.
We now use SanDisk's "Industrial" Micro SD and haven't had this problem since.
Also, reducing writes to the SD is certainly a good idea. Ideally - if you can - mount the system as read only.
Slightly off topic but I have also been looking at using net boot instead. Not only would it be just as fast as the Micro SD cards but much easier to perform backups locally and offsite.
I have used a usb stick (SanDisk Ultra Fit) for the Freenas O.S.. That drive died in 6 months (24/7). In general Freenas suggests to use 2 mirrored, quality, name-brand USB drives as they can `wear out or fail unexpectedly`[1]. They not suggest USB drives for data storage and I never tried.
[1] https://www.ixsystems.com/documentation/freenas/11.3-U4/intr...
I have run 24/7 many SSD's attached via USB, with no reliability problems.
Usually buying SATA or NVME SSD's and separate USB adapters for them is cheaper than buying USB SSD's, which actually contain an internal USB adapter.
Be aware (as I found out a couple of days ago) the RPi 4 USB port cannot power an external USB hard disk unless you use a separate powered hub. Should be fine to use a USB SSD / USB NVME though.
You will have problems trying to power a desktop 3.5" drive though.
[0] https://www.amazon.co.uk/Seagate-Expansion-Portable-PlayStat...
And the Corsair GTX, while massive, is as fast as an SSD and supports UASP and TRIM. It's just big, and relatively expensive, for a 'thumb drive'.
For rPi though? Not sure they have an AES accelerator, and if they don’t then it’s going to be a huge cost to performance.
Not only because USB is using a CPU core to operate (because USB uses CPU not dedicated hardware like FireWire) but also because it has to transparently translate and encrypt every access.
It seems like the raspberry actually has zero crypto extensions anyway: https://www.raspberrypi.org/forums/viewtopic.php?t=207888
The ARM chips in the all of the Raspberry pis do not have AES accelerators. Encrypting your pis drive is going to be a serious hit on performance like you said. I don't recommend it.
If you really really really want to encrypt your Rpi then you need to use a disk cipher like Adiantum. It is based off of ChaCha12 and was made specifically for low end CPUs that lack AES accelerators.
That's why it's an interesting thing to measure, how much the competition between boot tasks and AES encryption affect the boot. It may not be much because I suspect boot is IO bound, but I don't know.
That's like having a 'hall of fame' for the fastest marathon runners, but not having anyone keep an eye on if they took a taxi half the route...
These benchmarks should always have a section where a drive has its power pulled repeatedly while under heavy load. If, ever, a single sector becomes overwritten, rolled back, or unreadable, except those being written between a pair of write barriers, then the drive has failed the test, and all other performance results thrown out. That drive took the taxi.
Recently I had the misfortune of having a debian update screw up my docker daemon and I ran over to Best Buy to grab any-old-USB drive to use as a new OS drive. As it happens the SD copy over went fine, but as I performed basic "setting the server up" operations like copying over the old data (50 GB of various sizes) and firing up docker-compose scripts, performance deteriorated rapidly to the point where a docker install of a lighttpd container took at least 30 minutes and the system became effectively unusable.
The poor-performing drive was the PNY Elite-X Fit, USB 3.0, 128GB, purchased for ~$22
I've switched back to a SanDisk SD card for the moment which is perfectly adequate for the next few months but I hope to get back to running off a USB soon when I'm not so busy.
For the Pi 4, a faster USB drive (e.g. SSD or NVMe in adapter) will be a massive improvement for many things (though some operations like launching a browser won't be vastly improved).