One thing I've always been curious about is hardware for mass-writing of sdcards.
I've setup manufacturing stations for this and I've never been able to find truly robust gang-programmers for sdcards. Luckily my stuff was all low volume so it wasn't a serious problem.
But it seems like there's no "manufacturing grade" gang programmers for sdcards. The stuff that is out there is just infinitely re-branded chinese "white-label" hardware-- just a lot of different "vendors" for a few different models of the basically the same hardware.
My suspicion for there not being good options for gang-programming of sdcards is that savvy consumer manufacturers have realized what a headache it is to support sdcards in their products from the factory and thus opt to make the sdcard the customer's responsibility. In other words, they ship their products without sdcards and just require the customer to purchase the sdcard and program it if needed. I am not sure if my suspicions are justified, can anyone confirm if this is the case?
Decent vendors don't save you at scale, until you're in the millions of units per year kind of scale. Until the all but the 'industrial' cards are crappy. I've had the displeasure of trying to track down their issues before professionally.
But it's not uncommon to have devices boot from sdcards-- even industrial boards. Are you talking about "all" sdcards? Or just consumer stuff whose actual partnumber could change from order to order?
It's more uncommon nowadays. VMware has officially recommended against sdcard boot, which used to be fairly popular. The SD card option is still usually available but generally recommended against when getting blades or pizzaboxes from HPE, Cisco and Dell. For various reasons it's now nearly the same price to toss 2x nvme drives in, instead of the SD card raid controller and SD card mounting assembly. TCO is definitely lower due to the write resilience of your typical nvme flash drive compared to an SD card, physical work swapping parts can be way more expensive than a small capital increase.
If you buy them in volume, you can buy them with the data already on from the manufacturer. I'm pretty sure it's programmed into the flash before the flash is assembled into the sdcard packaging.
You can do a similar thing with microcontrollers - if you're buying sufficient volume the manufacturer will supply them with your program already on, saving you a manufacturing step.
Yes, I've done that before for flash IC's. It works out nicely for "large enough" order quantities. I would expect that doing that for sdcards would require even larger quantities to be economically viable.
Fortunately Darren Shepherd came up with a novel idea of taking way the things he didn't want in Kubernetes and created a "distro" called K3s (3 is less than 8)
It's not that it is less, "3" is what you get if you take the character "8" and cut it in half vertically.
The explanation above was given by one of the Rancher staff members in one tech conference. It may be an alternative explanation that popped up with time.
SDcards are said to die often/easily in Raspberry Pis. I heard someone say they die when there's a write access going while the Raspi is taken from power. Is that true? That would mean running a Raspi from SDcard is fine as long as there is no write access happening, which can be done by enabling readonly (root) filesystem.
I have had a few cards go but I never isolated the cause. Certainly unceremoniously cutting power occurred a few times.
I got tired of the inconvenience of having to rebuild the Pi installation and flashing SD cards, so now I boot my Pi4 directly from a USB attached SSD, no card on-board at all. I haven't looked back. Others seem to also be getting good result like: https://www.jeffgeerling.com/blog/2020/im-booting-my-raspber...
From what happened to me - cards die, especially older ones when there is a lot of data written to them even without power failures (UPS ensured that). And when you have a journaled filesystem there is a lot of writes.
I made hybrid systems where either the card is only used to boot the system and everything (root included) is loaded from a USB drive or where card is mounted R/O, there is small ramdisk for volatile data and only when upgrade/reconfiguration is needed you remount it r/w, do what you need and then remount it back r/o. In both those setups cards didn't fail for the last 3-4 years. With a standard SD card root-fs they lasted about 4-6 months.
> "Where kubeadm created a headache for Raspberry Pi, K3s made things simple with an easy boostrap, low resource requirements and a single binary for distribution and updates."
I'm curious what the headache was. K8S has shipped arm kubeadm and kubelet bins for quite a while.
I also don't understand how k3s, microkube and others claim to be "lightweight". Lightweight compared to what? Certainly not upstream kubeadm which is as lightweight a bootstrap as you can get.
> "Given how much I'd liked the Docker experience of "docker swarm init" and "docker swarm join", I decided to re-create a similar experience for K3s in the k3sup tool"
K3s for example is more lightweight than upstream kubeadm because it compiles all the required components into a single binary. That also leads to considerably less operational complexity.
kubeadm dosen't include ANY components such as kube-apiserver, kube-controller-manager, kube-scheduler, kube-proxy or etcd.
These components are all containerized already from upstream. Pulled and cached when you run "kubeadm init" via static pod manifests.
It seems trying to bundle these inside a non-upstream bootstrapper and running them out-of-band from kubernetes is what would add complexity and bloat.
I think you might be missing the point. You are correct, kubeadm doesn't include any components in that sense and runs them in containers. The issue with this is that there's a lot of overheads for each service that Kubernetes runs, causing Kubernetes as a whole to have a bigger memory footprint. K3s on the other hand combines all the necessary services and runs them in a single binary, which allows for a reduced memory footprint with very little lost in functionality. I don't know what all was involved when this was done but that is the gist of it.
Are RasPi's actually performant for this kind of use case? Like relative to just getting a random desktop or server rack... thingy? I can't help but feel like at one point you're just generating a bunch of garbage/wasting energy compared to more performant offerings that might be more efficient all around.
Is it price?
(Just to be clear, I think "having fun" is a fine excuse to do most stuff, but they're talking about "replacing usage of AWS" which sounds a hell of a lot like a serious business thingy)
VS desktop class hardware, the power draw is a giant plus also leading to no or quiet fans. An old laptop, however, has about the same power/performance consumption while including an USP, keyboard, mouse and screen, albeit it is bigger and has no GPIO if you need those.
That being said, I am running a NAS and PiHole on a RPi4B right now and am happy. I am planning on turning an old laptop with busted hinges into a server with a nice housing though. I'm very interested how it's going to pan out :)
Edit: I had a 1U server with 2x intel CPUs (16 threads all in all I think?) with 32G RAM in 2012 or so and it didn't see much use. It was LOUD (small screaming fans that pushed air through the chasis) so it didn't see much use.
Indeed, if I'm going to build a home or small-office server, I want a 4U case, just so I can use fans that are large enough to be reasonably quiet (let's say, 80mm or bigger). Bonus points if it could somehow run 120 or 140 mm fans, but I haven't seen such a beast. I've seen 1U servers in action at a client, and they're horrifyingly loud.
If you don't really need a rack case, I'd even go as far as to use a normal case with regular Noctua fans. The hardware for PCs is probably more expensive though (plus no ECC, single socket, ...).
The BitScope racks are neat, but the thing that makes me sad about Pi is that there's no way to network from the 40 pin connector that would be reasonable to tie this all together. So you need a separate network infrastructure and a network cable to each pi.
The 40 pins on the Pi are intended for GPIO and low-speed connectivity (SPI, I2C and friends). Unless I'm missing something, it is simply not feasible to use those GPIO ports for any high-speed interfaces for three reasons:
1. The pins are likely connected to a memory-mapped GPIO controller in the Broadcom SoC (you write a value to a special memory location, and the GPIO controller will physically change the voltage on the associated pin). While you can implement certain slow interfaces - UART is the typical example - purely in software using bit-banging, this approach only works as long as your desired interface speed is a lot slower than the CPU clock. For example: To implement a UART controller at 9600 Bd running on a 1 GHz CPU, you have over 100000 cycles for each symbol you send or receive. For 1000BASE-T, you are looking at a symbol rate of 125000000 Bd, so you'd have just 8 cycles for each symbol (and this is completely ignoring the line code used for 1000BASE-T, which the GPIO controller very likely doesn't support on a physical level).
2. If you want to run an interface faster than a few MHz, you have to be more careful on the electrical level to avoid reflections/impedance mismatch and other signal integrity issues. Essentially, you can't just hook up 2.54mm pin headers at gigabit speeds and expect everything to work fine.
3. Related to the two previous points: Almost all high-speed interfaces (including GbE, USB, HDMI etc.) use line codes at the physical level and/or use differential voltage signaling (LVDS/TMDS etc.) for transmission. The Pi's GPIO controller is very likely unable to send/receive data using differential standards. By the way, if you're interested in this topic: Most FPGAs (even entry-level FPGAs) support differential voltage standards in their IO blocks and even include special encoder/decoder blocks (for 8b/10b encoding).
:D It's not my first rodeo doing high speed design.
> 2. If you want to run an interface faster than a few MHz, you have to be more careful on the electrical level to avoid reflections/impedance mismatch and other signal integrity issues. Essentially, you can't just hook up 2.54mm pin headers at gigabit speeds and expect everything to work fine.
Conversely, we got a gigabit out of a few wires, non-differential, on pin headers UltraDMA/ATAPI over distances of tens of inches, albeit with the insulation displacement connector able to put a ground between each wire on the ribbon to lower impedance a bit.
It would be nice to have some more capable interfaces multiplexed onto the pins. That's all I'm saying.
For that matter, an RJ45 doesn't have wonderful electrical characteristics, but it's a small impedance discontinuity at the ends of the circuit. Industrial uses put gigabit ethernet through all kinds of connectors. Gigabit Ethernet works in practice through a few inches of untwist and would work just fine through a pin header. Already not everything on the header is hooked up to a single GPIO controller-- some goes directly to the broadcom and some to an external GPIO controller on earlier PIs, I believe.
Something I'd like to know, if you're moving off AWS and using your own raspberry pi cluster, what would you do to provide a performant MySQL DB for example? I can't imagine that trying to run a high performance mysql instance on a PI makes sense. So would you still want an old-school rack server with RAID and all the rest to be your DB server?
Ask yourself, why do you need a MySQL DB in the first place? If the service can get away with sqlite, use that. Alternatively, there is also rqlite (and I think another similar project) that allows for sqlite to be HA.
That was the interesting detail that I was looking for too. I'm guessing that it's because at the bottom of the article you can see that he wants you to pay $30 for his ebook and video that will tell you how.
> I flashed each of the Raspberry Pis using the same SD card, then I could close the case and take a note of each MAC address and serial number for later on.
But if flashing SD cards, then why do we need network boot? Or, if we have network boot, why do we need to use SD cards?
The easiest way to perform the flash is by booting into Raspbian and performing the process interactively. Without this one-time process, you can't netboot, as the default setting is the SD card, then USB: https://www.raspberrypi.org/documentation/computers/raspberr...
How fast can a Pi boot? Are they suitable for embedded environments where "instant" booting is required? Are there any alternatives which run Linux with graphics which can boot faster?
The Pi has a weird boot sequence because of the Broadcom chip.
SBCs with uncomplicated boot loaders can get you to a shell prompt within one second of applying power; I imagine getting to a lightweight graphical environment in under 2 seconds is not hard. Doubtless you can reduce that a little with tweaking. That's a good deal faster than my car stereo, say.
There is bare metal code that almost immediately starts after powering the Pi on. A Pi3 and below with an absolute minimal Linux based OS could probably boot in 3-4 seconds or so. With the Pi4 that has changed and the EEPROM can take a few seconds on its own before the Pi even gets to the kernel loading phase.
