> Without UASP, a drive is mounted as a Mass Storage Device using Bulk Only Transport (or BOT), a protocol that was designed for transferring files way back in the USB 'Full speed' days
Wow, holy shit, TIL.. I was always assuming that USB drives were always SCSI — they show up as 'da' (Direct Attach) on FreeBSD after all.
Looking at umass driver source:
* The driver handles 3 Wire Protocols
* - Command/Bulk/Interrupt (CBI)
* - Command/Bulk/Interrupt with Command Completion Interrupt (CBI with CCI)
* - Mass Storage Bulk-Only (BBB) (BBB refers Bulk/Bulk/Bulk for Command/Data/Status phases)
* Over these wire protocols it handles the following command protocols
* - SCSI
* - UFI (floppy command set)
* - 8070i (ATAPI)
* UFI and 8070i (ATAPI) are transformed versions of the SCSI command set.
Huh, they are pretty much always SCSI, and "bulk only" is (unsurprisingly) only a USB transport level thing that doesn't change the command set.
I guess "UASP" is some sort of marketing term for Command/Bulk/Interrupt then..?
This is the PCI express driver for BCM2711. Good to see this but the UAS support would be elsewhere! Either in the umass driver or a new uas driver (just a guess as I am unfamiliar with the USB related kernel code).
It isn't just a matter of speed. There are some drive commands that only work when you attach a device via an adapter that supports UASP. OPAL drive encryption being one.
> The native Intel USB3 UASP solution is only supported under Windows 8. To further complicate matters, not all Z77 motherboards support USB3 UASP. A license is required to implement UASP, and not all motherboard manufacturers are prepared to pass on the extra cost of this license to the end user
Sounds very cursed. I mean, I'm not a hardware vendor, I should not care, but licensing for something that's just another form of SCSI over USB sounds ridiculous.
I didn't know they even supported scsi, I knew about the old BOT-protocol and the huge limitations for harddrives over USB, and for that reason I've always avoided them whenever possible with the assumption that the basic limitations of the protocol was still true for 3.0 drives.
This does put USB-drive in a better light for me. I might event buy UASP supported USB 3.1g2/3.2 SATA-controller and do some testing myself.
I usually try to reach for a bus-connected controller which usually mean thunderbolt today for external enclosures (expensive), but it would be nice to build a portable DAS-array that can be connected to any device with USB-C.
> I used a Satechi USB-C power tester and measured an 8% peak power savings using UASP. That means you'd get 8% more runtime on a battery if you do a lot of file transfers.
No no, even better! Peak power consumption is lower, but the same work is completed much more quickly due to increased throughput, so the energy required for the same work is decreased dramatically. Between the performance increase and the lower power usage I wouldn't be surprised if this reduces energy use by 50 %.
Yeah. This kind of savings comes up in interesting places elsewhere -- if you make your algorithm run twice as fast but using twice the memory, your time-integrated memory use (measured in gigabyte seconds) remains the same. So if you can make good use of that memory while your algorithm isn't running, you're saving CPU and not really using more RAM.
For something like the Pi this is unlikely, but in data centers with big, latency-insensitive distributed workloads "using memory for less time" can be a real win. (The battery example with the Pi really is the perfect example though, because you're usually interested in the time-integrated value more than the peak draw.)
It depends a lot on the workload, though. That kind of statistic is hard to draw any solid conclusions from, except that “it uses less CPU and less overall power during IO activity” and you’ll save some amount of energy consumption (depending on what you’re doing).
The Pi 4 is a big revolution, fast enough to serve as a desktop computer for most office people, students, and so on. It's pretty impressive and more attention is being paid to it as a first-class computing platform as time goes on.
Our desktop monoculture has led to serious security problems and massive stagnation at least until recently. Even if you don't use a Pi you should be thankful that they exist.
It should be and is decent, but when we tried we ran into numerous paper-cuts:
- Needed to set up NTP to set the clock, it was always wrong.
- It can't power down through software, so you have to shut down and then hit the power switch like it's 1990.
- Audio was a mess and had to spend hours researching how to write and edit a bunch of config files to make it predictable.
- The weird set up program means you need to do a lot of googling on how to fix issues in a non-standard to linux way. It mostly works however.
- Raspbian feels not quite "finished." Was much happier with Ubuntu Mate, as they fixed 99% of the paper cuts like audio already, but it doesn't support the 4 yet.
In short, it is a bit too cheap. I would have definitely spent another dollar to see these hardware issues fixed. Or dropped the CPU speed a hundred MHZ, whatever it takes.
In the end, a friend gave us a ten-year-old used iMac for free. I installed Ubuntu Mate on it and it had none of the above problems. It's even a touch faster.
A touch faster? The slowest 2010 iMac is the 21" with an i3-540 Dual core CPU, Geekbench 4 is 2211 single/4466 multi. The Pi 4 is around 970/2000, less than half the performance.
The Pi 4 has come a long way in speed since the lowly original Pi, but its still not even close to a 10 year old desktop PC (or iMac).
More people (even in 2010) used laptops than desktops, and the Pi is in line with the numbers from the cheaper MacBook Airs at the time (I still have one that runs... though it’s getting some weird green lines on the display).
But I can say that working on the 2010 MBA is not fun; one of my webcams doesn’t even work with it (choppy at 30p), zoom is... slow at best, and web browsing is a bit choppy even with ad blockers. Even then, it is a smoother overall experience with that hardware and macOS than when I was testing 64-bit Pi OS on the 8 GB Pi 4 a few weeks ago: https://www.jeffgeerling.com/blog/2020/i-replaced-my-macbook...
Just run Windows 10 for the Pi and call it a day. Linux for desktop is still a disaster and no surprise Windows 10 for the Pi actually works out of the box.
The CPU is now (IMHO) the major sticking point for many of my own use cases. It would also be nice to have an easier / less cumbersome way to connect higher speed storage. (The mess of cabling you get with an add-on USB 3.0 drive is annoying.)
It's also possible to fit the fan included with the 3400G into the deskmini with slight modification (removing the plastic shroud from the top, it's just clipped on).
For a better fit I also had to cut off two plastic fingers that were left after taking the shroud off. Does not affect the functionality in any way and works well to this day.
I find the NUC's way overpriced unless you really need a high end CPU. I love my H2s[0]- good mix of cpu, RAM, storage (m2 + sata), and low power usage. Plus, they run standard x86_64 Linux/windows. Tricked out with 16G RAM, and a couple of cheap SSDs mine came in at $250.
How fast does that setup run in practice? I have an odroid hc2 (which is a wholy different chipset than the h2plus, I know), and that's about as pokey as my pi4.
I've just bought a Pi4, 7" touchscreen and a case that takes the lot. I have converted the thing to boot and run off a USB3 stick and it runs rather faster than off the SD card. I'm not too bothered about trying to turn it into a NUC.
Nowadays I see a Pi as a kiosk style machine with extras. It is what it is. It is a fantastic jack of all trades. You get loads of GPIO of all sorts, bluetooth, wifi, ethernet, USB2 and 3, a quad core CPU and 2-8GB RAM. Two HDMI ports and a GPU. It's too much for jobs that you would deploy an ESP8266 or ESP32 and it's not enough for a full on desktop replacement.
I can't see a need for more than 16 or 32GB of storage on one of these things. If your use case needs greater storage then it probably needs more "box" or I simply lack imagination.
I have a C-64 under the telly in the sitting room (it has a QSII joystick and a USB interface). I have a ZX81 with 3" of rather stiff Blutac holding the 16Kb RAM expansion steady and a keyboard on a ribbon. Sometimes I go back to school and type in definitions for 8x8 blocks and make them run around the screen. I do not own a Sinclair C5, that would be silly.
It's terrible that 2-8GB RAM is not enough for a full on desktop replacement. You could run Windows 95 in 4 MB. Was that not a desktop? The original Mac was just 128 KB.
Suppose it was all for graphics. Scaling up for modern color depth and screen size, we should need about 400 MB for the Windows 95 level of efficiency, or about 64 MB for Mac level of efficiency. That's a huge overestimate because RAM is not all consumed by graphics.
In other words, modern software can't run a desktop properly without even 10 to 100 times as much RAM as it ought to need.
My desktop with KDE Plasma 5 consumes somewhere in the ballpark of 500 MB RAM initially on startup.
RAM consumption by Konsole, tmux, and Neovim (30+ loaded buffers) combined is negligible to say the least.
RAM consumption by my browser with 10+ windows and 100+ tabs loaded easily exceeds 4+ GB even when exclusively visiting supposedly "lightweight" text based sites (ie documentation).
Going beyond RAM, playing a video on YouTube occupies a small but noticeable fraction of total CPU time. A single script heavy ad laden site sometimes manages to occupy a problematically large fraction of all available cores. Depending on the site, Dark Reader might slow this all to a crawl for 10+ seconds as the bloated page gradually loads (I'm looking at you, Amazon).
A file server is a pretty basic task, and zfs is a pretty good choice of fs for a file server, and zfs wants all the ram you can scrape up.
But still there may not be many such examples so your point is still valid.
I use a 3b+ for a 3d printer and that means, octoprint, klipper, octolapse, live streaming from a webcam while printing, a web server and web ui with various plugins, a slicer (I don't slice on the pi, but it's there and I could) and for all that the 3b+ is hardly scratched while running full out. It's a 4g model that is always using only couple hundred meg if you don't count disk cache. This is exactly consistent with your supposition.
I can't see a need for more than 16 or 32GB of storage
on one of these things. If your use case needs
greater storage then it probably needs more "box" or
I simply lack imagination.
Generally yeah. One possible (slightly contrived) exception that comes to mind could be using it as "dumb" volatile storage like Redis/memcached/etc.
I say "dumb" because if you are leaning into any of Redis' more advanced stuff (sorted sets, pub/sub, whatever else they've thrown in lately) I suspect CPU would tend to become a bottleneck rather quickly once you've got > 8GB of data.
There are a few, and they're not too bad, but it's still something that would be neatly solved with an option of a PCIe slot or even onboard eMMC like on the compute module (assuming the eMMC or built-in NVMe was much faster than the one on CM3+ and below).
I also had a recent install that was unbearably slow. It turns out that the SD card was corrupted. Raspberry Pi OS comes with a SD card checking utility you can use to check. Replacing it with a new class 10 card made it fast again.
For my money they've pushed a fraction too hard on the CPU power. One of the best things for me about the 3 is that you don't need to worry about cooling the way you do with the 4. I get why, there's a huge demand for the 4 to be able to handle proper desktop applications, but I'd love to see a version that has the thermal characteristics of the 3 with the IO updates of the 4.
I'd actually like a new version of the Compute Module with the PCI-e lanes exposed (the more the better). That would be a real game changer for NAS systems.
Or, for both CM and the full RPi, a docking header and a corresponding tiny module for a real-time clock module if not outright embed one. It's 2020 ffs.
The Cortex-A53 is really weak. I wouldn't want to run an office suite on that either. Oh, and the Pi 3 was VERY limited by RAM.
"most office people, students, and so on" pretty much always do browse lots of modern websites, at the same time as running an office suite, audio player, and more stuff.
The A72 is really the "minimum viable CPU core" for desktop usage. Anything with less single-core performance is absolute hell and suffering.
Not true. I barely notice any difference between my Orange Pi 3 (4x Cortex-A53 @ 1.8GHz) and my Ivy Bridge workstation, when browsing the web, and things like that.
I don't run a heavy weight desktop though.
The only noticeable thing is some increased latency (mainly due to storage speed limited to 20MiB/s read speeds), but animations and scrolling and basic stuff like that are pretty much smooth. I could easily run the system from USB attached SSD (which can achieve 15-20x the speeds of the SD card), and that would massively improve app startup speeds too.
Also video playback sucks, but that's mainly due to software support, not a hardware issue. It's a chip meant for TV boxes after all.
I wouldn't use it to compile programs, but office work? I can imagine that easily.
Then I disabled UAS. Haven't had any corruption since. It's not absolute proof, but I believe the firmware and/or driver support for UAS with this dock is buggy, and the older protocol works fine. I like speed as much as the next person, but not if it means repeated filesystem corruption...
I'm all for blaming dodgy chipsets, but the Superspeed and Superspeed+ support in Linux definitely has not had all of its rough edges filed off yet either--especially the stuff that uses streams and asynchrony.
However, the article comparing to USB 1.1 is silly.
If you are on USB 2.0 (and I haven't bumped into anything mass storage in the last 10 years that isn't), the bulk endpoints are quite happy to take advantage of the USB 2.0 bulk packet size (512 bytes) and microframes (120us turnarounds) with 13 packets per microframe.
That's going to get you in the 400Mbps+(40Megabytes/sec) range without a lot of problem.
That’s still less than half the speed you get from USB 3.0 even without UASP from a cheap SSD. I drew the 1.1 comparison mostly to illustrate the fact that the general protocol for MSD on USB was created in the late 90s for an interface 400x slower than 3.0.
Not really, even the fast microSD cards have ~1000 IOPS. Regular A1 cards have 500 random address write IOPS. Most USB attached SSDs will top that many times over. It's not really hard to top 2-4 MiB/s write speeds by an order of magnitude.
This is more up-to-date, and includes more models of Pi’s and SD cards. It was linked in your post, but the post is now out-of-date relative to my link below.
If you want to bookmark a page, that's the one that I update every year or two (usually after a new Pi model is released). The blog posts fade a little in relevance over time since I don't keep them updated.
Even on my Mac (tested with 6 different readers, including one purporting A2 compatibility) the best microSD cards are abysmal for random access. But they do get closer to 90-120 MB/sec for reads, at least.
> To get advertised speeds on most of the cards, I am guessing you need to work for the SD association's marketing department
Note that there's all kinds of special, video camera-oriented things in the SD standard, where you can preallocate/pre-erase a big chunk to sequentially write, stream data in and do big commits. That's where you actually hit the rated speeds, because the flash controller is able to benefit from all the hinting.
Using SD with general purpose filesystem drivers, etc, is really just a very common edge case.
I believed you were writing about analysis linked in parent article (you know, the article we're all discussing), where he compares USB3.0 throughput to SD on the Raspberry Pi 4, and finds speed differences of 3-16x+:
Now I see that someone linked another article inbetween, which you beat up for being on 3B+, but the conclusions remain largely the same when tested on the 4. And note that these tests were without UASP on the USB drive, so USB was handicapped.
Excellent reading this; My Pi4/8GB cluster uses a 1000bT PoE hub for a backplane, and / is an nfsroot from a node running a Sandisk UASP USB3 SSD. Like the author, I'm blown away by the fun to be had with Docker and Kubernetes on this cluster. Unlike the author, I'm having a great experience with it as a Mac-replacing daily driver (thanks mostly to never running chromium on the same node that runs gnome-session, to spread things out a bit). Love to read about everyone's Pi cluster adventures.
I think I could make it work, but the hindrance to my media (library of 65,000+ RAW photos from three different camera bodies and about 8 TB of 1080p video projects (and counting)) workflows was what really killed it for me.
Even basic photo import / adjustments were a pain with any of the applications I tested, and that was partly due to the Pi's slower GPU/CPU, the 64-bit ARM paucity of desktop apps, and the lack of good (read: like Adobe/Apple/pro-tier media apps) UI-based applications on Linux.
Most of the other issues (like window management and resolution problems) I ran into with the 'Pi for a Day' project I could probably resolve with a few more days' work.
I'm going to be testing a few different thumb drives in my next post / video. I have a couple SanDisk models and an Arcanite USB 3.1 drive that I'll be testing... but I don't believe they use UASP (haven't had time to confirm that suspicion yet).
Thanks Jeff, you’re doing a great service to the community here. I’ve tried to hunt around for a good system drive and fell short so far; support for UASP/TRIM in thumb drives is both rare, and seldom specified by manufacturers even when present. Looks like the only way is to buy a bunch and try.
I am not sure about which pen drives support UASP, but one thing of note is they recently updated their firmware (June 15th) to support boot over USB instead of the SD Card. I have mine setup this way now, using an external hard drive enclosure that supports UASP. Works great.
Almost all USB thumb drives are garbage for booting OS because very bad performance for random read/write (Getting worse over time). IIRC SanDisk Extreme Pro is an exception.
I've been using UASP SSD drives in Linux for ~ 5 years, the main issue I face is not performance related but w.r.t monitoring.
S.M.A.R.T doesn't behave well with UASP, so not all Smartmontools's feature work[1]. Especially, LifeTime of the SSD don't and they are crucial. I had to resort to calculating remaining life of the SSD manually from the reported lifecycles and negating it with the manufacture's MTBF if they're reported.
I've faced this with devices from varying price range [USB 3.0/3.1] - MyDigitalSSD(OTG), Samsung T5, various ORICO USB SSD enclosures with Samsung EVO/Crucial SSDs on host systems of different architectures(x86/ARM).
How are you all monitoring the life of your USB SSD on UASP in Linux?
smartctl 6.6 on Ubuntu18.04 Jetson Nano (aarch64) gives Total_LBAs_Written for Samsung 840 Evo in ORICO USB 3.0 SSD enclosure (UASP) and Samsung T5 SSD USB 3.0 but failed to read S.M.A.R.T data properly for MyDigitalSSD(OTG) which is likely the fault of the controller in it.
So with Total_LBAs_Written we can find TBW (Total Bytes Written) and compare it with manufacturer's data[1].
which is not required to implement any usb protocol. one can easily use normal DMA and still get a speed gain. since the bottleneck is not CPU, making it work a bit harder putting together those DMA descriptors is worth it.
Why can't the RPi have at least 1GB or 2GB of flash storage on its board? My 4 year old, 80 euro android has 8GB.
It's not a secret SD cards quickly fail.
I've not bought the RPi for that specific reason. Even if the RPi is 5 times slower, having reliable storage seems seems like a most important feature.
Because forcing everyone to pay a chunk more money for on-board storage which not everyone needs is unnecessary. Their decision shows that cost is a more important feature and I don't think that should come as a surprise, there's plenty of stuff out there justifying the various decisions and it all comes down to cost.
I'm guessing if the RPi Foundation went that route with the Pi 5, it would be similar to the Compute Module where there would be a cheaper 'Lite' version without eMMC then more expensive ones with eMMC.
One of the main reasons SD cards fail in a PI is shutting them down incorrectly.
If you pull the power cord you risk corrupting the card.
If the PI had a proper power switch system (also omitted due to cost) then you would see less SD cards being corrupted.
It is possible that even on-board flash memory could be corrupted by improper shutdown, and if that happened the entire pi would be useless, not just the SD card.
You have the Compute Module versions of the pi with embedded storage.
The reasoning behind the SD cards on the consumer pis is that they are quick to swap out, and don't have the problems that "bricking" the OS could get you into with booting from onboard storage. There is the possibility to use USB booting in the latest boards (certainly with the 3+ not sure about the status of the 4) which has a flag set in the onboard eeprom so you can use external SSDs/USB drives.
With USB boot it's really nice for me, as I have a few microSD cards preloaded with 'boot up and do X' tasks, as well as a couple SSDs which are set up the same way and I can just remove the microSD and plug in the USB drive.
Wow, holy shit, TIL.. I was always assuming that USB drives were always SCSI — they show up as 'da' (Direct Attach) on FreeBSD after all.
Looking at umass driver source:
* The driver handles 3 Wire Protocols
* - Command/Bulk/Interrupt (CBI)
* - Command/Bulk/Interrupt with Command Completion Interrupt (CBI with CCI)
* - Mass Storage Bulk-Only (BBB) (BBB refers Bulk/Bulk/Bulk for Command/Data/Status phases)
* Over these wire protocols it handles the following command protocols
* - SCSI
* - UFI (floppy command set)
* - 8070i (ATAPI)
* UFI and 8070i (ATAPI) are transformed versions of the SCSI command set.
Huh, they are pretty much always SCSI, and "bulk only" is (unsurprisingly) only a USB transport level thing that doesn't change the command set.
I guess "UASP" is some sort of marketing term for Command/Bulk/Interrupt then..?