Gokrazy is easily the most pleasant way I've found to deploy code to a raspberry pi. The worst part about setting up a raspberry pi is having to get out a keyboard and monitor (or serial cable) to manually configure a the rpi before you can start using it. Gokrazy builds fully configured system images, so the rpi comes up properly without having to shell into it first. Gokrazy also lets you push updated images over the network.
The other issue I often had with raspberry pi systems was the high probability that an `apt-get upgrade` would trigger a filesystem error on mediocre sdcards. Gokrazy also helps with this 1) by having a read-only file system by default, 2) by using a/b partitions for updates, and 3) having very small image sizes.
It has honestly rejuvenated my enthusiasm for running code on raspberries pi.
I never had any serious issue with SD cards since the Pi 2B (and I've kept Pis running for years).
Anyway, for those wanting to deploy more generic apps, that is why I initially wrote https://github.com/piku/piku - you still have to flash the OS (and rpi-imager does that with sane defaults these days), but once you're done you have Heroku-like deployments for any language runtime you install on the Pi.
I also have https://github.com/rcarmo/ground-init, a cloud-init like shim that simplifies setting up machines (I'm a big fan of cloud-init, but since Raspbian doesn't support it and Ubuntu on ARM requires some fiddling to make it work I decided it wasn't too hard to roll my own).
(I probably should look into glueing that into rpi-imager, but there is are only so many hours in the day...)
Gokrazy seems pretty great and I loved the idea the moment I saw a description of what it does. (Though, disappointingly, I've never actually tried it.) I've long been interested in trying to write the entire usermode of a Linux system just because it seems interesting what you could do; the Linux kernel offers a whole lot of functionality. I've had the thought that this would be really fun to do on Pinephone, although in that case it does sort-of feel like you're just inventing another Android... Maybe focusing on making Wayland work better on phones would be a more productive use of everyone's time :)
However, I will say one thing about Raspberry Pi's: If you ever find yourself needing to use Raspbian for any reason, you can configure a lot of things by editing the boot files on the FAT32 boot partition. For example, you can throw a wpa_supplicant config to get WiFi working, enable SSH, etc. just using the boot partition. This is basically how I manage Raspberry Pis that I'm just messing around with, although cooking a fresh root image every time would definitely be a nicer solution for "real" work. (My weapon of choice would probably be NixOS in most cases, but I have to try gokrazy first if I ever deploy anything interesting and less temporary to a Raspberry Pi in the near future.)
Have you tried recent versions of the Raspberry Pi Imager utility? It lets you build pre-configured Raspbian images, similar to what you're describing.
Yes, it is. I'm using gokrazy with blocky[0] on a Raspberry Pi as a replacement ever since my PiHole died to sd card failure. It's been running very reliably.
Another thing that should very much be in Xe's wheelhouse: For non-go software you can write a small go wrapper which launches the software from a nix closure. This is a fun way to get a single-purpose RPi up and running.
I have my Twitter alerts set via them. My preferred source (AdaFruit) is still limiting them to one per order. They're still scarce, but not as bad as it used to be, and getting better.
It seems the developer works for Tailscale in an evangelist / developer relations role. I feel that ought to have been disclosed in this article.
That gripe aside - I love this idea and the general principle. Minimal OS's are really interesting to me. I've long dreamed of a basic environment where there are 3 UI surfaces: a Terminal, a Web server / browser, an OpenGL context. Of course, if you have a full capablity web browser you can emulate a Terminal and use WebGL for 3d.
Of course, things like this exist I am sure - but the idea that you could have one that is a dozen files is super compelling. It reminds me of how I was able to get a basic http server in C using civetweb which is a handful of files I can just copy-paste into my source tree. Or the handful of files sized libraries like raylib that provide huge amounts of functionality.
Fun fact: I can't legally call myself an "Engineer" in Canada because I don't have the ring/haven't passed an engineering course in college. My official job title is "Archmage of Infrastructure", but overall I'd call myself an artist/philosopher more than an engineer. I explore what is possible and turn it into what is practical.
I'm curious if this is actually a thing that has ever been an issue for software engineers.
I feel like it's not the government's place to argue semantics, and it's widely understood that the word "engineering" is used to describe a process that one engages in that involves some amount of education, but that the education is not necessarily scholastic in nature.
"Software engineer" specifically is used around the world to refer to the role performed. I'm in Canada, and my job title is "(lead) software engineer". It's on my contract; it's on my job portal. I'm not convinced this is been a problem for anyone, though I have heard of people in Canada getting in trouble for misrepesenting themselves as accredited engineers
The government of Ontario cares and my goal is to get citizenship. Not pissing off the government or professional bodies is a great way to facilitate that goal lol
I've heard an anecdote of someone being denied entry at the border because they described themselves as "software engineer", which as far as the border guard was concerned was a fake job.
> "Software engineer" specifically is used around the world to refer to the role performed.
I don't think it is? I spent 10+ years programming in an English-speaking country and it sounds weird and cringe to me, the same way as "Code Ninja" or "Rockstar" (or indeed the "Archmage" title mentioned elsewhere in the thread).
Typically companies just pay the menial fine.
It's run by a self-regularing body and doesn't impact your citizenship -- it's not the government who enforces it.
I graduated from UW in 2011 in software engineering. Never did my P.Eng.
I drank the juice that software practitioner will need liability in the future....but it never panned out.
> It seems the developer works for Tailscale in an evangelist / developer relations role. I feel that ought to have been disclosed in this article.
Note that the author of gokrazy is Michael Stapelberg (perhaps better known for writing i3), who doesn't seem to be directly affiliated with Tailscale.
Well said. Also hugely grateful to Tuomo Valkonen and everyone involved with ion for gifting the world such a great tiling wm implementation in the first place. I’ve been a happy user of both for 15 years!
What a coincidence! I've just been playing with Gokrazy a couple weeks ago, and just kept thinking "this is so cool". If you're building some sort of an appliance, and want the least amount of reliance on / hassle maintaining the base OS, it definitely is a viable choice.
It can also run programs that are not written in go, by using a little neat hack to build/embed a binary inside a Go package; this is e.g. how Gokrazy sets up persistent storage: https://github.com/gokrazy/mkfs
I don't think it's for everyone; if you're relying on your base OS / package manager for a lot of stuff, or just want to run Docker containers, I think there are simpler/better ways to set things up. But it's absolutely great at what it's made for; doubly so with the Raspberry Pi's finally being back in stock.
18 months ago I spent a few days figuring out how to get WPA working natively on gokrazy, and then as soon as all the bits got merged, got sidetracked and forgot all about it.
One of these days I will get around to picking it up again and trying to use it for the original idea I had.
is it possible to cross-compile that from a mac or window machine and get this rpi working binary ready to ship via Gokrazy? So far cross-compiling has failed for me. Too many x11 deps and weird stuff that makes it so much easier to just run go build on the rpi itself.
Cross-compiling something for gokrazy will likely be a can of worms if it has a lot of C dependencies, which tend to be dynamically linked by default. There's no dynamic library loader on gokrazy, so you either need to add one yourself or get all those C deps (and their deps) to be statically compiled into your Go binary. Neither is a picnic, although maybe someone else has already done the work of figuring out how to set up a dynamic loader.
> I don’t see how a bunch of included GNU utilities gives it naming rights.
On early-ish Linux, gnu projects provided basically the entire userland, and since it was also the compiler and the libc, that was the foundation to the entire system.
The days of GNU significant relevance have passed haven’t they? There’s so many incarnations of Linux that surely GNU is relevant in only a small number of cases?
Because no one counts Android as a Linux distro, and they shouldn't. They couldn't be more oil and water from the rest of the Linux ecosystem. The term would instantly become meaningless. When people talk about alternative to glibc it's meant to be other software that lets you run applications targeting libc.
When the question is "how accurate is it to refer to Linux systems as GNU/Linux", and 3.6 billion devices run Android, and probably as many TVs, home internet devices (modems, wifi, etc), don't ship with the GNU userspace, then the answer very well might be "not accurate at all".
In fact, at this point, I would be surprised to learn that there are more Linux based computers running GNU userspace than not running GNU userspace.
In my house, off the top of my head, I can count 7(!) devices that I know for a fact use Linux and don't have the GNU userspace, three that I'm unsure of, and two that do. And, you probably also have more Linux devices in your household not running GNU userspace, than those that do.
As for the compiler? That's a valid point, but I don't think that earns you naming rights :) We don't call it Windows/Visual Studio 11 or macos/clang.
Interesting point! However, I rarely see people write “GNU/Linux” these days, at least in the parts of Internet where I hang out. (And there’s quite a lot of FOSS/Linux folks in those parts.)
this couldn’t be better timed. i just started using gokrazy + tailscale + a reverse proxy running on a VPS to host https://j3s.sh! my plan is to get a “this van runs linux” bumper sticker, since my gokrazy machine is powered by my van
i’ll make a blog post about this setup soon, it’s all still pretty in flux rn.
one thing worth mentioning is that if you’re operating in a limited bandwidth environment, goks update may be annoying/impossible to tolerate - it uploads the full resulting squashfs to the pi.
in my case, it’s about 40MB over the network per change, even if it’s just a flag adjustment. that’s just the cost of the “appliance model”
Cool site :) I think there is a template error at the moment listing the books you're reading.
books i'm reading currently:template: now.html:9:33: executing "now.html" at <.CurrentBooks.ReadingLogEntries>: can't evaluate field CurrentBooks in type struct { Title string; Data interface {} }
Fortunately Gokrazy's site (https://gokrazy.org/) explains exactly what it is and what it's for up front: "With gokrazy, you can deploy your Go programs as appliances to a Raspberry Pi or PC (→ supported platforms)."
It's a minimal Linux distribution that contains just the Linux kernel and the bare minimum of userspace (basically just init) needed to run Go programs on Linux. Like Alpine Linux, but just for Go.
The article could have replaced the entire section about Linux vs. GNU/Linux (until 1st occurrence of "gokrazy") with 1 line explaining what gokrazy is, and it would be a better article.
Linux vs. GNU/Linux has been discussed ad nauseum.
Reading several pages through an article before it says what it's about, is both annoying & wasting many readers' time.
Not sure if you were being snarky but having the ability to scroll doesn't fix a poorly structured article...
Things should be explained as they come, or have a linked explanation, or say things like "we'll explain what XYZ is later".
If an article is about a tool but I don't know what the tool is and the first paragraph doesn't explain it, am I really expected to skim the entire article in the hope that somewhere is hidden an explanation, and then go back and read from the top? That can't possibly be good story telling?
I don't think it would be too wild to imagine that, maybe, this article isn't meant for you. It does seem perfect for other people that might benefit from a primer on the background.
Good storytelling comes in many shapes and forms, maybe the author was trying to use anticipation as the main element!
I really don't understand the surprise you're apparently filled with when coming across an article that's not presented in the way you expected; an imaginary problem that stops existing once you scroll down or do a keyword search.
Unsolicited recommendation, if I may: do not watch Pulp Fiction.
In all honesty, if you write go programs, provided you don’t need CGo, you can write bare metal services that can bootstrap and run as pid 0. Try doing that with .Net or Java (ok, you technically can, but it’s a challenge). Go + Linux Kernel and maybe musl is really all you need to run a go web service. I wouldn’t recommend for production but it will get you to 5mb image sizes.
> I wouldn’t recommend for production but it will get you to 5mb image sizes.
Former co-worker did this, and we still have tons of docker images in production that lack basic debugging functionality like... a shell. Alpine Linux is only like 8 MB more :(
I prefer to run my service containers without a shell or much of an operating system. YMMV but in this corporate world where every container is scanned for every CVE and some binaries are deep inspected my list of vulnerabilities on a more full fledged container or VM can become a task that spirals out of control. The context being that many (maybe even most) of those CVEs probably don't pertain to how I use the software or package and that it's far easier to patch than file for an exemption.
Thus, I spend the investment up front in getting log streaming working, my logs are concise, I implement application monitoring, and I demand host monitoring from the platforms I use. If I check all of those boxes, I generally don't have anything I need to do with a shell.
I fully agree with you about vuln scans, but as a counterpoint there have been dozens of times when I've saved hours of debugging with a well-applied strace or tcpdump. Logging and monitoring are great and necessary, but they'll only catch things you thought of ahead of time; using them to debug something ongoing is basically printf() debugging where compile time = the full length of your CI/CD pipeline.
Depending on how you run your containers, you should be able to run a debug container in the same namespace as your target container. That way you can keep your images lean and bundle all the debugging tools in a different image, which you run only when you need to.
Yeah, that's definitely a valid take depending on your setup. If I have those kinds of problems with a container then I generally jump into the underlying VM or metal to use those tools, but that also implies a lot of knowledge around how a host system incorporates container networking, which arguably makes hard troubleshooting even harder. On headless systems they usually come with some sort of privileged "admin" container, so the setup is the same.
Second to that is that I have dev stages that are built with containers that do have those tools, and generally if I run into those kinds of problems I see them in dev first.
>tons of docker images in production that lack basic debugging functionality like... a shell
That's a commendable security practice. A whole class of vulnerabilities is mitigated (and others are much harder to exploit) if you don't add unnecessary junk to your images, like a shell.
If you have root access on the host machine you might get away with the host tooling (depending on your issue).
I (as a devops engineer) did that because (rightfully) rhe developers i was working with at the time didn’t include some troubleshooting tools (like tcpdump) and the inages were running as non root anyway.
Look up the manpage for nsenter, it’s all you need really.
Btw, tcpdump in production, hunting the correct network interface on a kubernetes cluster node… fond memories:)
This is the wrong question. The better question is something along the lines of "why do you think you need a shell?", and the only answers you can have there are making attacker's lives easier and theoretical cases where you may need to run shell commands against a service in anger. I don't personally think it's worth it.
You’re debugging in production, I highly doubt running bare metal services is in your wheelhouse. Your services should be logging to another system. If you need a shell to debug, you don’t need bare metal Golang. Go with Alpine (pun).
While novel, you really need to have the engineering excellence in your org to be able to struct log to another system, blue/green deployments, etc.
One place where I worked where did bare metal orchestration with containers, we had tooling around seeing which services were failing, where to go look at logs (and filter them), and we threw away the ssh keys to the AWS ECS hosts to force you into a CD deployment model. You’ll never get Sherlock Holmes access to production. Not even to run SQL queries against your production database.
The other issue I often had with raspberry pi systems was the high probability that an `apt-get upgrade` would trigger a filesystem error on mediocre sdcards. Gokrazy also helps with this 1) by having a read-only file system by default, 2) by using a/b partitions for updates, and 3) having very small image sizes.
It has honestly rejuvenated my enthusiasm for running code on raspberries pi.