Hacker News new | past | comments | ask | show | jobs | submit login
Building static binaries with Go on Linux (thegreenplace.net)
184 points by ingve 6 months ago | hide | past | favorite | 61 comments



The secret with sqlite is to use "-tags sqlite_omit_load_extension", if you don't use any extension. (which is 99% of the users)

This is explained in https://www.arp242.net/static-go.html


In the specific case of SQLite, you can use it through WASM now [1]. It uses the dependency and Cgo-free Wazero runtime.

Performance so far has been better than the modernc transpile and it’s probably sufficient for a lot of use cases.

[1] https://github.com/ncruces/go-sqlite3



The WASM solution doesn’t rely on a custom libc or transpiler to convert C code to Go. The transpile is an amazing feat of engineering, but it’s hard to debug.

I can wrap my head around the small amount of wrapping the go-sqlite3 WASM library does. If I had to I can maintain that should the maintainer lose interest. I can’t say the same for the modernc transpile. You can also apply the WASM trick to other libraries with much less effort.

And as noted, it seems to be performing better. As the wasm runtime improves it should pull further ahead.


Performance is likely part of it. The WASM solution looks faster than modernc: https://github.com/cvilsmeier/go-sqlite-bench


Author here.

Faster was a by product. Maintainability was the goal.

API coverage, ergonomics, extensibility all rank higher in my book than performance.

An example I'd like to cite is sqlite-vec. Alex was able to build a Cgo-free version of it, on his own, which works fine with my bindings. This would be much harder to do with modernc.

https://github.com/asg017/sqlite-vec-go-bindings

I'm also adding support for building off the bedrock branch (begin concurrent, wal2). You just build the branch with wasi-sdk, then embed the resulting blob.


> This would be much harder to do with modernc.

https://go.dev/play/p/IztwH9Jl68i


Careful. We (Tailscale) tried to use static Go binaries a year or two ago built with Zig (zig cc) and the SQLite performance was atrocious. It passed all our tests but it didn't survive deploying to prod. It was a very quick (and uneventful) rollback at least.

(Needless to say, we have better load testing tooling now)

I forget the details, but something about the libc allocator used by SQLite-with-Zig-libc being ... not good.


This sounds like the musl allocator. Using mimalloc or jemalloc would probably fair a lot better.


Is this a problem in distributions that use musl as the system libc (Alpine) ?


I wonder if it's Zig or musl that's to blame; did you end up statically linking with musl using musl-gcc, or did you forego static linking entirely?


It's well-known that musl is in general much, much slower than glibc. People keep rediscovering that for some reason, likely because they hear of stuff like old echoes of Usenet posts ranting against glob "bloat", not being aware that a lot of what people call bloat is specialization of a lot of performance-sensitive algorithms to leverage SIMD, special-casing, math optimisations, etc. When you check the glibc mailing lists, it's obvious performance is a predominant concern.


Interesting...

I have been building & using a statically compiled tailscale (with CGO) for a while but didn't notice any performance hits. Script: https://github.com/Azathothas/Toolpacks/blob/main/.github/sc...


I’ve experienced the same performance issues with cgo + a library compiled with zig cc. IIRC it seemed like an issue with the zig tooling not plumbing the optimization flags through the ancient autotools build system for our required dependency. After a while fiddling, we just rolled it back too.

I haven’t tried this in about a year, so maybe the tooling doesn’t have these issues now.


I've heard you can just swap out allocators pretty easily - did something prevent this? Or perhaps its not as straightforward as I've thought...


Producing static PIE binaries is a bigger challenge still.

  $ go build -ldflags '-linkmode external -s -w -extldflags "--static-pie"' -buildmode=pie -tags 'osusergo,netgo,static_build' main.go
For anyone curious to delve into what is this and why: https://www.leviathansecurity.com/blog/aslr-protection-for-s...


Very interesting and well described! If I were to have one nitpick, it would be the use of "Unix" when it's more specific to Linux. Ex. "The libc typically used on Unix systems is glibc" However I'm sure all of the concepts still apply on BSD, Solaris, etc.


Go is also famous for bypassing libc and issuing syscalls directly on quite a few platforms.


I thought I'd read they backed off of that and started using posix as an abstraction layer.


Go used to do raw syscalls on macOS but changed. Same with some BSDs.

Go always did ~libc on Windows (and Solaris) and still does.

Go still does raw syscalls on Linux, as that's a stable ABI.


libc is a UNIX concept, as the OS API surface, as described by POSIX.

On any other no-UNIX derived OS, it is the C compiler standard library, covering only the ISO C specifies.

All the remaining OS services are exposed by other libraries, which in Windows case, the bare minimum is user, kernel and gdi dlls for the Win32 personality.

Systems like IBM i, IBM z, ClearPath MCP, .... also have similar set of libraries, as do non-POSIX RTOS for embedded.


There’s also Filippo Valsorda linking directly to Rust via .a files. https://words.filippo.io/rustgo/


I feel like there's a lot of potential between Go and Cosmopolitan libc. Go itself does not use libc much, as shown in the blog, but some great libraries like SQLite3 need it (unless you use https://pkg.go.dev/modernc.org/sqlite).

The ability to build a single static binary that works on Linux, Mac and Windows using Go would be life changing for the internal tools I develop at work.


> The ability to build a single static binary that works on Linux, Mac and Windows using Go would be life changing for the internal tools I develop at work.

Just curious, life changing in what way? Obviously, 1 is better than 3, but I'm wondering if there is some other interesting reason.


Hmmm…what about compiling to wasm, and/or then converting the wasm to C?

https://github.com/wasm3/wasm3/releases/tag/v0.4.8

https://github.com/WebAssembly/wabt/tree/main/wasm2c


Has been done and works pretty great: https://github.com/ncruces/go-sqlite3.

Though doesn’t convert the WASM to C, it runs the WASM in Wazero instead.


How does that solve what the person I replied to was asking for?


If I understood you, it doesn't help much, no, but neither does what you suggested.

You're suggesting compiling Go to Wasm (presumably using the wasip1 target?), then converting that to C using wabt, then using Cosmopolitan to create an APE… is that it?

Well, that's not going to work.

First of all, Go's wasip1 target doesn't even support cgo, so if you want SQLite, you're dead right there.

Then, even if you used say TinyGo (which might support cgo, not sure), WASI just isn't a great target to compile SQLite into. WASI is a pretty limited syscall layer. You'd end up with no file locking, no shared memory. Also no threads.

Then, on top of that, you'd layer Cosmopolitan issues. Having written a portable SQLite VFS from scratch, I am not impressed with how they just paper over file locking semantics incompatibilities between OSes, and then confidently ship a forking webserver with SQLite bundled in. It takes a certain temerity, and not running many SQLite torture tests.

Wasm as an intermediate target is great for (single threaded) CPU stuff. WASI is great if you can fit it, but otherwise, it's not, not really.


The same exact binary working isn't going to happen without runtime performance penalties because the syscall numbers are different on different platforms. Also I believe on windows it's not possible to avoid linking some system libraries to use windows.h stuff, there is no stable ABI.


Linux is the exception among modern OSes to have a stable syscall ABI, everyone else offers only the proper OS API as entry point into OS services.

Once upon at time, static linking was the only thing OSes were capable of, all of them moved away from that, and there is no turning back outside embedded bare metal deployments, just become some people are obsessed with static linking.


Though msvcrt.dll has a stable subset of functions available on all Windows versions.


I recall reading about new Macs with ARM chips not supporting static binaries. Is it not true?


That's been Apple's stance on full static linking MacOS in general, many years prior to the move to ARM e.g. https://developer.apple.com/library/archive/qa/qa1118/_index...

You're welcome to ignore it of course, it's just unofficial and a large pain.


>You're welcome to ignore it of course

How do you mean? Like, is it possible to run such binaries on M1? If so I'd really like to know how


You can always disassemble libc and look for the system call numbers used by the syscall assembly instructions. It’s just that these numbers (and associated arguments and return values) are not stable and can and do change upon kernel updates (in which case libc will be updated to keep the libc interface stable). I believe Linux is the only major OS these days to guarantee binary compatibility of the syscall interface.


I know this works on Macs with Intel chips. But the ones with ARM chips just won't execute fully static binaries, and I'm wondering if there's a workaround.


I’m guessing not. According to man ld on macOS, the -static flag, to produce a fully static executable, is only used to build the kernel. I don’t believe fully-static executables were ever officially supported on macOS, although they would work.


For clarity it's not the chip/ARM that causes the limitation, you can recompile the kernel (it's open source) to remove the block and it'll work fine - it's just a ton of work.

Alternatively, Linux :).


It only works if you don't ever upgrade macOS. Even a patch update sometimes can break it.


Nope.


Thanks. That sucks


If you can convince Apple to change this code let me know: https://github.com/apple-oss-distributions/xnu/blob/94d3b452...


All Macs don't support static binaries. That's because the syscall interface on macOS is not stable, only libc is guaranteed to be stable.


What's the best way to include the go runtime itself, as in ability to invoke the "go" program from the program itself . I'm not talking about embedding it or downloading it. I want it included within the program.


How are you supposed to include it within the program without somehow "embedding" it? Or am I missing some vital understanding of what "include" vs "embedding" means here?


By embedding it, I mean using the embed feature to pack the golang binary into the program. What I am going after is similar to kubectl and kustomize. The kustomize source code included with kubectl, it's not a binary packed in and extracted


[flagged]


Since you've continued to start programming language flamewars after we asked you to stop (https://news.ycombinator.com/item?id=40979059), I've banned this account.

We really don't want that type of flamewar here (or any type, really) because it's not compatible with curious conversation, the purpose of the site.

If you don't want to be banned, you're welcome to email hn@ycombinator.com and give us reason to believe that you'll follow the rules in the future. They're here: https://news.ycombinator.com/newsguidelines.html.

We detached this subthread from https://news.ycombinator.com/item?id=41117094.


Man this is disappointing. He was one of the most interesting accounts I would search out to see his comments on C# and dotnet. I think he adds value to the site, and is a contrarian against the general common opinions.

Programming language debates can be interesting and fruitful for people to learn new things in the discussion. Maybe it comes with a little too much hostility sometimes, but banning seems like a pure loss. Hopefully he asks to come back and can constrain himself more.


I don't disagree and hope that they'll commit to following the guidelines in the future so we can unban them.

Accounts with pre-existing flamewar agendas* are definitely not a good thing on HN, and users were complaining.

* or actually any pre-existing agenda. Those are repetitive and repetition is anti-optimal for HN (https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so...).


Disappointing it didn't happen sooner. They've been shitting up Go threads every chance they get for months, if not longer. Pretty always with cheap "zomg Go devs r stoopid"-type potshots that add no value at all. These are not serious discussions anyone learns from, they're hostile toxic puddles of waste that add no value to anyone and only serve to chase off actual people with expertise (include those critical of Go) wanting to have a serious conversation, because why would someone with expertise put up with someone who is not interested in anything but talk shit?

They've also been advocating jail sentences and even wishing death of people they disagree with: https://news.ycombinator.com/item?id=39285838 - https://news.ycombinator.com/item?id=39285851 – that alone should be a near-bannable offence IMO, and underscores that this isn't some person who occasionally breaks the rules on one topic. I kind of regret not emailing Dan about that last one – I probably should have.

I actually have a long list of their comment for a blog post I plan to write about how social dynamics on the internet, and they're not there as an example of friendly kind of people on the internet. Their highly toxic behaviour stands out that much.

That they may also post some good comments on C# or .NET is besides the point. Bad behaviour is bad behaviour, and not cancelled out by some good comments elsewhere.


They? lol

Look, I don't 100% agree with that either, and I don't really want to "defend" someone else's comments from 2 years ago. I just think it's strange to ban people just for having an naïve or off-colour opinion, esp. if they're otherwise not causing any problems.


It's five months ago, not two years. And it's not an "off-colour opinion", it's just assholery. My point was that this isn't a one-off "everyone has a bad day"-type thing, but a consistent pattern that's been going on for months and that it's frequent and severe enough for people to take notice.

If you want to have vaguely serious discussions then you need to treat people with a modicum of respect, otherwise all the people wanting to have a serious discussion will just leave and what you end up with is just the assholes swimming in a sea of cynicism, negativity, and general assholery. So yes, they're absolutely causing problems.


It's a fine line, but passionate high energy discussion makes this place more interesting than if people with strong opinions were banned. It would become boring, "safe", and conventional. Everyone would be saying the same things.

A wise man once said:

"Conflict is essential to human life, whether between different aspects of oneself, between oneself and the environment, between different individuals or between different groups. It follows that the aim of healthy living is not the direct elimination of conflict, which is possible only by forcible suppression of one or other of its antagonistic components, but the toleration of it—the capacity to bear the tensions of doubt and of unsatisfied need and the willingness to hold judgement in suspense until finer and finer solutions can be discovered which integrate more and more the claims of both sides. It is the psychologist's job to make possible the acceptance of such an idea so that the richness of the varieties of experience, whether within the unit of the single personality or in the wider unit of the group, can come to expression."

Maybe if people could be muted or something would be compromise.


There is no discussion. What are you even supposed to "discuss" on the comment that started this thread? This isn't someone engaging in a bit of an aggressive way, it's merely an expression of negativity and nothing else.

You don't see me crapping up every TypeScript or PHP thread with that, and when I do it's 1) on-topic, and 2) a criticism that has more substance than "bruh-huh, TypeScript bad". Example: https://news.ycombinator.com/item?id=37765713 – this is something you can agree or disagree with and have a serious conversation about. "Go sux, just use .NET" ... not so much. Especially when that same discussion recurs on every damn thread because this user has posted dozens of those types of comments. It gets very tedious very quickly.

And muting people isn't really a solution; any new user who hasn't built up a "block list" will just see a community of assholes, so it's not really a substitute. Even with blocklist features, there needs to be a baseline of acceptable behaviour.


and yet a ton of fundamentally important projects are written in Go like k8s, Docker, Prometheus, Terraform ...


And barely any big open source is written in .NET, the only one I can recall without googling is Ryujinx


Jellyfin is one of the bigger ones for sure.


Yup, and Bitwarden is another one. Many larger game publishers run their infrastructure exclusively on .NET like Roblox or Ubisoft.

At the end of the day, all the tools written in Go that are listed above are not exactly paragons of performance.

Ryujinx is probably one of the best showcases of the kind of task that is impossible to solve in Go effectively, and yet can be solved very well with .NET. Or you could look at Garnet which is a pure C# Redis implementation which beats vanilla Redis, KeyDB and Dragonfly.


Many large companies including Google, Uber and so on run a ton of Go services :)


This doesn't make Go's FFI not suck. It's one of the worst across all general purpose languages that are in use today.


Quite true, but it shows how much work Microsoft is yet to do in UNIX shops, and bosting about performance improvements on twitter isn't going to change the culture and attitute towards .NET.

Wouldn't it great if even Azure would contribute to CNCF projects using .NET, instead of Go and Rust, as they do currently?




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: