UASP makes Raspberry Pi 4 disk IO 50% faster | Hacker News Reader

UASP makes Raspberry Pi 4 disk IO 50% faster

jeffgeerling.com

489 pointsgeerlingguy113 comments

> Without UASP, a drive is mounted as a Mass Storage Device using Bulk Only Transport (or BOT), a protocol that was designed for transferring files way back in the USB 'Full speed' days

Wow, holy shit, TIL.. I was always assuming that USB drives were always SCSI — they show up as 'da' (Direct Attach) on FreeBSD after all.

Looking at umass driver source:

* The driver handles 3 Wire Protocols

* - Command/Bulk/Interrupt (CBI)

* - Command/Bulk/Interrupt with Command Completion Interrupt (CBI with CCI)

* - Mass Storage Bulk-Only (BBB) (BBB refers Bulk/Bulk/Bulk for Command/Data/Status phases)

* Over these wire protocols it handles the following command protocols

* - SCSI

* - UFI (floppy command set)

* - 8070i (ATAPI)

* UFI and 8070i (ATAPI) are transformed versions of the SCSI command set.

Huh, they are pretty much always SCSI, and "bulk only" is (unsurprisingly) only a USB transport level thing that doesn't change the command set.

I guess "UASP" is some sort of marketing term for Command/Bulk/Interrupt then..?

Goals section:

https://en.wikipedia.org/wiki/USB_Attached_SCSI

> The native Intel USB3 UASP solution is only supported under Windows 8. To further complicate matters, not all Z77 motherboards support USB3 UASP. A license is required to implement UASP, and not all motherboard manufacturers are prepared to pass on the extra cost of this license to the end user

Sounds very cursed. I mean, I'm not a hardware vendor, I should not care, but licensing for something that's just another form of SCSI over USB sounds ridiculous.

FreeBSD doesn't support UASP (protocol id 98). AFAIK neither do the other three BSDs.

Kudos for remembering DragonFlyBSD. We’re flattered.

Apparently as of 30 minutes ago FreeBSD does support it on the Pi 4! https://twitter.com/FreeBSDHelp/status/1280062935327313920

This is the PCI express driver for BCM2711. Good to see this but the UAS support would be elsewhere! Either in the umass driver or a new uas driver (just a guess as I am unfamiliar with the USB related kernel code).

Gah, you're right. Totally misread that tweet in my state of non-caffeinated stupor this morning. Still a useful improvement!

It isn't just a matter of speed. There are some drive commands that only work when you attach a device via an adapter that supports UASP. OPAL drive encryption being one.

As I understand it, it is a better USB layer/feature set to run SCSI on top of which has better properties. TFA gets into it a bit.

I think UASP adds command queuing and reduces some software overhead.

I didn't know they even supported scsi, I knew about the old BOT-protocol and the huge limitations for harddrives over USB, and for that reason I've always avoided them whenever possible with the assumption that the basic limitations of the protocol was still true for 3.0 drives.

This does put USB-drive in a better light for me. I might event buy UASP supported USB 3.1g2/3.2 SATA-controller and do some testing myself.

I usually try to reach for a bus-connected controller which usually mean thunderbolt today for external enclosures (expensive), but it would be nice to build a portable DAS-array that can be connected to any device with USB-C.

> I used a Satechi USB-C power tester and measured an 8% peak power savings using UASP. That means you'd get 8% more runtime on a battery if you do a lot of file transfers.

No no, even better! Peak power consumption is lower, but the same work is completed much more quickly due to increased throughput, so the energy required for the same work is decreased dramatically. Between the performance increase and the lower power usage I wouldn't be surprised if this reduces energy use by 50 %.

It depends a lot on the workload, though. That kind of statistic is hard to draw any solid conclusions from, except that “it uses less CPU and less overall power during IO activity” and you’ll save some amount of energy consumption (depending on what you’re doing).

Yeah. This kind of savings comes up in interesting places elsewhere -- if you make your algorithm run twice as fast but using twice the memory, your time-integrated memory use (measured in gigabyte seconds) remains the same. So if you can make good use of that memory while your algorithm isn't running, you're saving CPU and not really using more RAM.

For something like the Pi this is unlikely, but in data centers with big, latency-insensitive distributed workloads "using memory for less time" can be a real win. (The battery example with the Pi really is the perfect example though, because you're usually interested in the time-integrated value more than the peak draw.)

https://en.wikipedia.org/wiki/Amdahl's_law

The Pi 4 is a big revolution, fast enough to serve as a desktop computer for most office people, students, and so on. It's pretty impressive and more attention is being paid to it as a first-class computing platform as time goes on.

Our desktop monoculture has led to serious security problems and massive stagnation at least until recently. Even if you don't use a Pi you should be thankful that they exist.

The CPU is now (IMHO) the major sticking point for many of my own use cases. It would also be nice to have an easier / less cumbersome way to connect higher speed storage. (The mess of cabling you get with an add-on USB 3.0 drive is annoying.)

> The mess of cabling you get with an add-on USB 3.0 drive is annoying

I’m surprised there’s no Raspberry Pi case with a slot for a 2.5” drive and integrated USB 3.0 SATA controller.

there are, can even make your own. One quick example https://www.amazon.co.uk/MakerHawk-Upgrated-Raspberry-2-5-In...

There are a few, and they're not too bad, but it's still something that would be neatly solved with an option of a PCIe slot or even onboard eMMC like on the compute module (assuming the eMMC or built-in NVMe was much faster than the one on CM3+ and below).

I've just bought a Pi4, 7" touchscreen and a case that takes the lot. I have converted the thing to boot and run off a USB3 stick and it runs rather faster than off the SD card. I'm not too bothered about trying to turn it into a NUC.

Nowadays I see a Pi as a kiosk style machine with extras. It is what it is. It is a fantastic jack of all trades. You get loads of GPIO of all sorts, bluetooth, wifi, ethernet, USB2 and 3, a quad core CPU and 2-8GB RAM. Two HDMI ports and a GPU. It's too much for jobs that you would deploy an ESP8266 or ESP32 and it's not enough for a full on desktop replacement.

I can't see a need for more than 16 or 32GB of storage on one of these things. If your use case needs greater storage then it probably needs more "box" or I simply lack imagination.

I have a C-64 under the telly in the sitting room (it has a QSII joystick and a USB interface). I have a ZX81 with 3" of rather stiff Blutac holding the 16Kb RAM expansion steady and a keyboard on a ribbon. Sometimes I go back to school and type in definitions for 8x8 blocks and make them run around the screen. I do not own a Sinclair C5, that would be silly.

Mess of cabling: lol!

    I can't see a need for more than 16 or 32GB of storage 
    on one of these things. If your use case needs 
    greater storage then it probably needs more "box" or
    I simply lack imagination.

Generally yeah. One possible (slightly contrived) exception that comes to mind could be using it as "dumb" volatile storage like Redis/memcached/etc.

I say "dumb" because if you are leaning into any of Redis' more advanced stuff (sorted sets, pub/sub, whatever else they've thrown in lately) I suspect CPU would tend to become a bottleneck rather quickly once you've got > 8GB of data.

It's terrible that 2-8GB RAM is not enough for a full on desktop replacement. You could run Windows 95 in 4 MB. Was that not a desktop? The original Mac was just 128 KB.

Suppose it was all for graphics. Scaling up for modern color depth and screen size, we should need about 400 MB for the Windows 95 level of efficiency, or about 64 MB for Mac level of efficiency. That's a huge overestimate because RAM is not all consumed by graphics.

In other words, modern software can't run a desktop properly without even 10 to 100 times as much RAM as it ought to need.

My desktop with KDE Plasma 5 consumes somewhere in the ballpark of 500 MB RAM initially on startup.

RAM consumption by Konsole, tmux, and Neovim (30+ loaded buffers) combined is negligible to say the least.

RAM consumption by my browser with 10+ windows and 100+ tabs loaded easily exceeds 4+ GB even when exclusively visiting supposedly "lightweight" text based sites (ie documentation).

Going beyond RAM, playing a video on YouTube occupies a small but noticeable fraction of total CPU time. A single script heavy ad laden site sometimes manages to occupy a problematically large fraction of all available cores. Depending on the site, Dark Reader might slow this all to a crawl for 10+ seconds as the bloated page gradually loads (I'm looking at you, Amazon).

The problem isn't modern desktop environments.

A file server is a pretty basic task, and zfs is a pretty good choice of fs for a file server, and zfs wants all the ram you can scrape up.

But still there may not be many such examples so your point is still valid.

I use a 3b+ for a 3d printer and that means, octoprint, klipper, octolapse, live streaming from a webcam while printing, a web server and web ui with various plugins, a slicer (I don't slice on the pi, but it's there and I could) and for all that the 3b+ is hardly scratched while running full out. It's a 4g model that is always using only couple hundred meg if you don't count disk cache. This is exactly consistent with your supposition.

NVME hanging off the USB bus would be nicer and more compact.

At that point you probably want a NUC style computer like the Lenovo M75q. Up to 4 core 8 thread, 32GB+ RAM (two slots), AMD graphics and an M.2 slot.

https://psref.lenovo.com/syspool/Sys/PDF/ThinkCentre/ThinkCe...

Also noteworthy is the upcoming ASUS PN 50 with the new Ryzen Renoir CPUs with up to 8 cores and 64GB RAM.

When this will be available in Europe I’ll get that to replace my MacBook for a developer machine.

https://liliputing.com/2020/06/asus-pn50-is-a-mini-pc-with-u...

The 8 core 16 thread version looks very promising. Wonder what the prices will be?

x64 doesnt excite me as much as Arm does at the moment, for power draw.

The DeskMini series has two M.2 slots AND two 2.5” slots in a tiny enclosure. Here’s mine:

Noctua NH-L9A-AM4, 37mm Premium Low-profile CPU Cooler for AMD AM4 (Brown) £36.17 incl. VAT https://www.amazon.co.uk/Noctua-NH-L9A-AM4-Premium-Low-profi...

ASRock USB 2.0 Header to 2 x USB 2.0 Cable for DeskMini Series Chassis £10.93 Incl. VAT https://www.amazon.co.uk/ASRock-Cable-Deskmini-Mini-Stx-Chas...

AMD Ryzen 5 3400G Processor (4C/8T, 6MB cache, 4.2GHz Max Boost) with Radeon™ RX Vega 11 Graphics £139.99 incl. VAT https://www.amazon.co.uk/dp/B07SXNDKNM/?tag=pcp0f-21

Sabrent 1TB Rocket NVMe PCIe M.2 2280 Internal SSD High Performance Solid State Drive (SB-ROCKET-1TB) £109.99 incl. VAT https://www.amazon.co.uk/dp/B07LGF54XR/?tag=pcp0f-21&th=1

ASRock DeskMini A300 Mini PC Barebone for Socket AM4 £146.94 incl. VAT https://www.amazon.co.uk/dp/B07P9GL1LN/?tag=pcp0f-21

ASRock Wi-Fi Kit for DeskMini Series Chassis, includes 2 x Antennas £29.42 incl. VAT https://www.amazon.co.uk/ASRock-Wi-Fi-DeskMini-Chassis-Anten...

2 x ADATA AD4S2666316G19-S 16GB DDR4 2666 MHz Memory Module - Memory Modules (16GB, 1x 16GB, DDR4, 2666MHz, 260-pin SO-DIMM) 2 x £73.12 incl. VAT https://www.amazon.co.uk/ADATA-AD4S2666316G19-S-16GB-Memory-...

Price a few months ago - £619.68

Note that you don’t need the noctua fan but the provided fan that comes with the case is a bit whiny.

It's also possible to fit the fan included with the 3400G into the deskmini with slight modification (removing the plastic shroud from the top, it's just clipped on).

For a better fit I also had to cut off two plastic fingers that were left after taking the shroud off. Does not affect the functionality in any way and works well to this day.

Ooh, that does sound good.

DeskMini A300's dimensions: 155 x 155 x 80 mm

From the specs: https://www.asrock.com/nettop/AMD/DeskMini%20A300%20Series/i...

being 3-4× the size, weight, price and power consumption, it really doesn't compare. I'm sure pi4 SATA/NVMe enclosures/hats exist

I find the NUC's way overpriced unless you really need a high end CPU. I love my H2s[0]- good mix of cpu, RAM, storage (m2 + sata), and low power usage. Plus, they run standard x86_64 Linux/windows. Tricked out with 16G RAM, and a couple of cheap SSDs mine came in at $250.

[0]https://www.hardkernel.com/shop/odroid-h2plus/

How fast does that setup run in practice? I have an odroid hc2 (which is a wholy different chipset than the h2plus, I know), and that's about as pokey as my pi4.

I haven't benchmarked them - the h2plus is celeron quad core at 2.3Ghz, with on-board Intel graphics.

I find them fine for light interactive use, and running network services for the home network. Also gradually moving my lab machines over to them.

None of this is particularly CPU intensive, but does benefit from being x86 compatible. Also having M2 NVME + 2xSata on-board is nice :-)

That's a sick machine, to bad it's not running Intel Nics otherwise I'd consider it for firewall/NAT-uses.

I really like them - here are some photos of putting it all together: https://news.ycombinator.com/item?id=23744973

Used SFF PC (like Haswell i5) is cheap, powerful for this context, and also nice for power consumption.

> The Pi 4 is a big revolution, fast enough to serve as a desktop computer for most office people, students, and so on.

Even the Pi 3 was already good enough for most of those use cases. The main exception being browsing modern bloated websites.

The Cortex-A53 is really weak. I wouldn't want to run an office suite on that either. Oh, and the Pi 3 was VERY limited by RAM.

"most office people, students, and so on" pretty much always do browse lots of modern websites, at the same time as running an office suite, audio player, and more stuff.

The A72 is really the "minimum viable CPU core" for desktop usage. Anything with less single-core performance is absolute hell and suffering.

The CM 3+ (which is clocked at 1.2 GHz but is same in other ways to the Pi B 3+ which is 1.4 GHz) is less than half as fast as the Pi 4 in most CPU tasks: https://www.jeffgeerling.com/blog/2020/raspberry-pi-cluster-...

Not true. I barely notice any difference between my Orange Pi 3 (4x Cortex-A53 @ 1.8GHz) and my Ivy Bridge workstation, when browsing the web, and things like that.

I don't run a heavy weight desktop though.

The only noticeable thing is some increased latency (mainly due to storage speed limited to 20MiB/s read speeds), but animations and scrolling and basic stuff like that are pretty much smooth. I could easily run the system from USB attached SSD (which can achieve 15-20x the speeds of the SD card), and that would massively improve app startup speeds too.

Also video playback sucks, but that's mainly due to software support, not a hardware issue. It's a chip meant for TV boxes after all.

I wouldn't use it to compile programs, but office work? I can imagine that easily.

> The main exception being browsing modern bloated websites.

So, the most common use case for a desktop computer?

The current version of raspian comes with ublock origin preinstalled in chromium.

Both of these choices surprised me.

Why does the choice of uBlock origin surprise you?

I wasn't expecting an adblocker installed by default.

Makes sense though, for browsing performance.

It should be and is decent, but when we tried we ran into numerous paper-cuts:

- Needed to set up NTP to set the clock, it was always wrong.

- It can't power down through software, so you have to shut down and then hit the power switch like it's 1990.

- Audio was a mess and had to spend hours researching how to write and edit a bunch of config files to make it predictable.

- The weird set up program means you need to do a lot of googling on how to fix issues in a non-standard to linux way. It mostly works however.

- Raspbian feels not quite "finished." Was much happier with Ubuntu Mate, as they fixed 99% of the paper cuts like audio already, but it doesn't support the 4 yet.

In short, it is a bit too cheap. I would have definitely spent another dollar to see these hardware issues fixed. Or dropped the CPU speed a hundred MHZ, whatever it takes.

In the end, a friend gave us a ten-year-old used iMac for free. I installed Ubuntu Mate on it and it had none of the above problems. It's even a touch faster.

A touch faster? The slowest 2010 iMac is the 21" with an i3-540 Dual core CPU, Geekbench 4 is 2211 single/4466 multi. The Pi 4 is around 970/2000, less than half the performance.

The Pi 4 has come a long way in speed since the lowly original Pi, but its still not even close to a 10 year old desktop PC (or iMac).

More people (even in 2010) used laptops than desktops, and the Pi is in line with the numbers from the cheaper MacBook Airs at the time (I still have one that runs... though it’s getting some weird green lines on the display).

But I can say that working on the 2010 MBA is not fun; one of my webcams doesn’t even work with it (choppy at 30p), zoom is... slow at best, and web browsing is a bit choppy even with ad blockers. Even then, it is a smoother overall experience with that hardware and macOS than when I was testing 64-bit Pi OS on the 8 GB Pi 4 a few weeks ago: https://www.jeffgeerling.com/blog/2020/i-replaced-my-macbook...

Don't be so literal. ;-) Looked closer and it is a 7,1 from 2007. Core 2 duo 2.6ghz, decent HD, and Nvidia gfx.

None of those except the second one (and maybe the first one) are hardware issues. Raspbian is not great but nobody is forcing you to use it.

Just run Windows 10 for the Pi and call it a day. Linux for desktop is still a disaster and no surprise Windows 10 for the Pi actually works out of the box.

Linux for desktop has not been a disaster for a few years now. Windows for anything has been a disaster for a few years now.

Nope, Ubuntu Mate is very functional and respects the user. I won't use a disrespecting OS unless forced. Which I do, have a W10 vm for work.

Now Win 2k I might consider, however it is likely very insecure and newer apps don't support it.

I bought an old Lenovo tiny pc for about the same price as a pi4 with all perephials. Performance can hardly be compared.

I tried to use the pi 4 as a nextcloud server and found it to be unbearably slow. Not sure what caused it but I suspect the sd card speed.

I also had a recent install that was unbearably slow. It turns out that the SD card was corrupted. Raspberry Pi OS comes with a SD card checking utility you can use to check. Replacing it with a new class 10 card made it fast again.

You can boot from USB now. No need for SD.

You should also go for at least Application Performance Class 1.

browsing media is still unusably slow on usb3 hdd as well. once it starts generating previews, youre done. pre-generating doesnt help much

been using it for photo sync only, works fine enough as long as youre not using the web interface

Did you try it using a UASP USB3 HDD though?

That was my issue. Images would take 3-10 seconds to show in the UI. Once I moved to a ryzen 5 server I was seeing everything load like it was local.

For my money they've pushed a fraction too hard on the CPU power. One of the best things for me about the 3 is that you don't need to worry about cooling the way you do with the 4. I get why, there's a huge demand for the 4 to be able to handle proper desktop applications, but I'd love to see a version that has the thermal characteristics of the 3 with the IO updates of the 4.

I'd actually like a new version of the Compute Module with the PCI-e lanes exposed (the more the better). That would be a real game changer for NAS systems.

Or, for both CM and the full RPi, a docking header and a corresponding tiny module for a real-time clock module if not outright embed one. It's 2020 ffs.

It's the price of that ancient 28nm manufacturing.

But does it work reliably?

I have a cheap Inateck dock which does support UASP (the FD2005). Two or three times I had a bunch of errors like these:

    Sep 22 17:26:01 nuc kernel: sd 4:0:0:1: [sdb] tag#2 uas_eh_abort_handler 0 uas-tag 3 inflight: CMD OUT
    Sep 22 17:26:01 nuc kernel: sd 4:0:0:1: [sdb] tag#2 CDB: Write(16) 8a 00 00 00 00 01 4d b4 c4 00 00 00 03 b0 00 00
    ...
    Sep 22 17:26:22 nuc kernel: usb 2-3: stat urb: no pending cmd for uas-tag 2
    ...
    Sep 22 17:26:22 nuc kernel: xhci_hcd 0000:00:14.0: ERROR Unknown event condition 34 for slot 3 ep 3 , HC probably busted

followed by filesystem corruption.

I found other folks complaining about similar problems, eg: https://unix.stackexchange.com/questions/239782/connection-p...

Then I disabled UAS. Haven't had any corruption since. It's not absolute proof, but I believe the firmware and/or driver support for UAS with this dock is buggy, and the older protocol works fine. I like speed as much as the next person, but not if it means repeated filesystem corruption...

I'm all for blaming dodgy chipsets, but the Superspeed and Superspeed+ support in Linux definitely has not had all of its rough edges filed off yet either--especially the stuff that uses streams and asynchrony.

However, the article comparing to USB 1.1 is silly.

If you are on USB 2.0 (and I haven't bumped into anything mass storage in the last 10 years that isn't), the bulk endpoints are quite happy to take advantage of the USB 2.0 bulk packet size (512 bytes) and microframes (120us turnarounds) with 13 packets per microframe.

That's going to get you in the 400Mbps+(40Megabytes/sec) range without a lot of problem.

That’s still less than half the speed you get from USB 3.0 even without UASP from a cheap SSD. I drew the 1.1 comparison mostly to illustrate the fact that the general protocol for MSD on USB was created in the late 90s for an interface 400x slower than 3.0.

I had the same issue with a cheap Orico that identifies as a Toshiba device ID 0080:a001.

In UAS mode it would work for a couple of seconds and start to fail, dropping transfer speed to a couple of hundred kB/s

>I found a USB 3.0 SSD was ten times faster than the fastest microSD card I tested

I looked it up because i know enough about hardware that this shouldn't be the case (even with a cheap SD Card). Seems it's a limitation of the Pi's SD Card slot. https://www.jeffgeerling.com/blog/2018/raspberry-pi-microsd-...

Note this is only valid for raspberry pi's made before the 4, The 4 includes a much faster SD card slot.

Note that the testing was done on a Pi 4...

> tested on a Raspberry Pi model 3 B+:

No it wasn't.

This one, however, was: https://www.jeffgeerling.com/blog/2019/raspberry-pi-microsd-...

Even on my Mac (tested with 6 different readers, including one purporting A2 compatibility) the best microSD cards are abysmal for random access. But they do get closer to 90-120 MB/sec for reads, at least.

To get advertised speeds on most of the cards, I am guessing you need to work for the SD association's marketing department :D https://www.jeffgeerling.com/blog/2019/raspberry-pi-microsd-...

Wow great review, thank you.

> To get advertised speeds on most of the cards, I am guessing you need to work for the SD association's marketing department

Note that there's all kinds of special, video camera-oriented things in the SD standard, where you can preallocate/pre-erase a big chunk to sequentially write, stream data in and do big commits. That's where you actually hit the rated speeds, because the flash controller is able to benefit from all the hinting.

Using SD with general purpose filesystem drivers, etc, is really just a very common edge case.

I believed you were writing about analysis linked in parent article (you know, the article we're all discussing), where he compares USB3.0 throughput to SD on the Raspberry Pi 4, and finds speed differences of 3-16x+:

https://www.jeffgeerling.com/blog/2020/im-booting-my-raspber...

Now I see that someone linked another article inbetween, which you beat up for being on 3B+, but the conclusions remain largely the same when tested on the 4. And note that these tests were without UASP on the USB drive, so USB was handicapped.

This is more up-to-date, and includes more models of Pi’s and SD cards. It was linked in your post, but the post is now out-of-date relative to my link below.

https://pidramble.com/wiki/benchmarks/microsd-cards

If you want to bookmark a page, that's the one that I update every year or two (usually after a new Pi model is released). The blog posts fade a little in relevance over time since I don't keep them updated.

Thanks for all the work you’ve put in to this. I hope we see a Pi with NVMe soon.

Not really, even the fast microSD cards have ~1000 IOPS. Regular A1 cards have 500 random address write IOPS. Most USB attached SSDs will top that many times over. It's not really hard to top 2-4 MiB/s write speeds by an order of magnitude.

Excellent reading this; My Pi4/8GB cluster uses a 1000bT PoE hub for a backplane, and / is an nfsroot from a node running a Sandisk UASP USB3 SSD. Like the author, I'm blown away by the fun to be had with Docker and Kubernetes on this cluster. Unlike the author, I'm having a great experience with it as a Mac-replacing daily driver (thanks mostly to never running chromium on the same node that runs gnome-session, to spread things out a bit). Love to read about everyone's Pi cluster adventures.

I think I could make it work, but the hindrance to my media (library of 65,000+ RAW photos from three different camera bodies and about 8 TB of 1080p video projects (and counting)) workflows was what really killed it for me.

Even basic photo import / adjustments were a pain with any of the applications I tested, and that was partly due to the Pi's slower GPU/CPU, the 64-bit ARM paucity of desktop apps, and the lack of good (read: like Adobe/Apple/pro-tier media apps) UI-based applications on Linux.

Most of the other issues (like window management and resolution problems) I ran into with the 'Pi for a Day' project I could probably resolve with a few more days' work.

Friends, I have a question but I don't have a Pi 4 sitting around to test this:

Is it possible to run a Pi 4 using UASP if I set it up to boot from a decent (Sandisk/Samsung/etc) USB 3.1 pen drive?

Also, is this a reliable setup, or will the device be more stable if booted and run from a MicroSD?

I'm going to be testing a few different thumb drives in my next post / video. I have a couple SanDisk models and an Arcanite USB 3.1 drive that I'll be testing... but I don't believe they use UASP (haven't had time to confirm that suspicion yet).

Thanks Jeff, you’re doing a great service to the community here. I’ve tried to hunt around for a good system drive and fell short so far; support for UASP/TRIM in thumb drives is both rare, and seldom specified by manufacturers even when present. Looks like the only way is to buy a bunch and try.

I am not sure about which pen drives support UASP, but one thing of note is they recently updated their firmware (June 15th) to support boot over USB instead of the SD Card. I have mine setup this way now, using an external hard drive enclosure that supports UASP. Works great.

Almost all USB thumb drives are garbage for booting OS because very bad performance for random read/write (Getting worse over time). IIRC SanDisk Extreme Pro is an exception.

I've been using UASP SSD drives in Linux for ~ 5 years, the main issue I face is not performance related but w.r.t monitoring.

S.M.A.R.T doesn't behave well with UASP, so not all Smartmontools's feature work[1]. Especially, LifeTime of the SSD don't and they are crucial. I had to resort to calculating remaining life of the SSD manually from the reported lifecycles and negating it with the manufacture's MTBF if they're reported.

I've faced this with devices from varying price range [USB 3.0/3.1] - MyDigitalSSD(OTG), Samsung T5, various ORICO USB SSD enclosures with Samsung EVO/Crucial SSDs on host systems of different architectures(x86/ARM).

How are you all monitoring the life of your USB SSD on UASP in Linux?

[1]https://www.smartmontools.org/wiki/USB

Update:

smartctl 6.6 on Ubuntu18.04 Jetson Nano (aarch64) gives Total_LBAs_Written for Samsung 840 Evo in ORICO USB 3.0 SSD enclosure (UASP) and Samsung T5 SSD USB 3.0 but failed to read S.M.A.R.T data properly for MyDigitalSSD(OTG) which is likely the fault of the controller in it.

So with Total_LBAs_Written we can find TBW (Total Bytes Written) and compare it with manufacturer's data[1].

[1]https://www.virten.net/2016/12/ssd-total-bytes-written-calcu...

I have a USB 3.0 Orico enclosure with a Pi4 and I had to pass '-d sat' to smartctl for it to work:

smartctl -a -d sat /dev/sda

Newer versions of smartmontools shouldn't need it, only the older one that Raspberry PI OS has.

How does a USB 3.x NVMe drive function on the RPi4? I notice those on Amazon as well.

What possible "controller feature" would be needed from the controller on the hardware side to support just another USB protocol?

I was curious about this as well. The issue appears to be lack of scatter-gather support. See: https://github.com/raspberrypi/linux/issues/875#issuecomment...

which is not required to implement any usb protocol. one can easily use normal DMA and still get a speed gain. since the bottleneck is not CPU, making it work a bit harder putting together those DMA descriptors is worth it.

apparently XHCI Streams

Why can't the RPi have at least 1GB or 2GB of flash storage on its board? My 4 year old, 80 euro android has 8GB.

It's not a secret SD cards quickly fail.

I've not bought the RPi for that specific reason. Even if the RPi is 5 times slower, having reliable storage seems seems like a most important feature.

You have the Compute Module versions of the pi with embedded storage.

The reasoning behind the SD cards on the consumer pis is that they are quick to swap out, and don't have the problems that "bricking" the OS could get you into with booting from onboard storage. There is the possibility to use USB booting in the latest boards (certainly with the 3+ not sure about the status of the 4) which has a flag set in the onboard eeprom so you can use external SSDs/USB drives.

Ah yes, instead you have consumers learn the painful lesson of losing data when the SD cards eventually fail instead...

So you'd prefer your flash, which will eventually fail, to be soldered to the board rather than easily replaceable?

I don't think onboard flash memory fails as often as SD cards do

One of the main reasons SD cards fail in a PI is shutting them down incorrectly. If you pull the power cord you risk corrupting the card. If the PI had a proper power switch system (also omitted due to cost) then you would see less SD cards being corrupted.

It is possible that even on-board flash memory could be corrupted by improper shutdown, and if that happened the entire pi would be useless, not just the SD card.

Because forcing everyone to pay a chunk more money for on-board storage which not everyone needs is unnecessary. Their decision shows that cost is a more important feature and I don't think that should come as a surprise, there's plenty of stuff out there justifying the various decisions and it all comes down to cost.

I'm guessing if the RPi Foundation went that route with the Pi 5, it would be similar to the Compute Module where there would be a cheaper 'Lite' version without eMMC then more expensive ones with eMMC.

Some people use RPi to hack around. You can use one SD card for one project, then swap it out and boot up a different project entirely.

With USB boot it's really nice for me, as I have a few microSD cards preloaded with 'boot up and do X' tasks, as well as a couple SSDs which are set up the same way and I can just remove the microSD and plug in the USB drive.

Just did a "dd if=/dev/zero [etc]" in my Rapberry Pi 4:

4974456832 bytes (5,0 GB, 4,6 GiB) copied, 25,4896 s, 195 MB/s

Using this controller and a Samsung EVO SSD drive, I'm really happy with the results: https://geekworm.com/collections/raspberry-pi/products/raspb...

Well, looks like I need to ditch the SD cards on my Pi4 based robot! Seems like this would improve reliability over SD cards too.

I honestly thought this was common knowledge.

Reply on news.ycombinator.com