Hacker News new | past | comments | ask | show | jobs | submit login
OpenBSD: Removing syscall(2) from libc and kernel (marc.info)
144 points by ecliptik on Oct 27, 2023 | hide | past | favorite | 118 comments



OpenBSD developers are making a serious effort to kill off indirect syscalls, the base system is completely clean, take a look at the work Andrew Fresh did to adapt Perl. He wrote a complete syscall "dispatcher" or emulator for the Perl syscall function so that it calls the libc stubs.

https://github.com/openbsd/src/commit/312e26c80be876012ae979...

The ports tree is being cleansed of syscall(2) usage, until they're all gone.

msyscall, pinsyscall, recent mandatory IBT/BTI, xonly. OpenBSD is making some waves, but people aren't really seeing them yet.


It looks like golang is going to have to deal with — again — OpenBSD treating libc as the interface with the kernel instead of syscalls being the interface with the kernel.

I wonder if there could be some way to sign a dynamic library to allow it to create direct system calls and then pass that as a kernel command line argument at boot?


OpenBSD is certainly taking it quite far. They want to disallow system calls from all non-libc code segments so that only whitelisted code can interface with it.

It is not the only operating system in the "unstable kernel interface" group though. Linux is actually the only one with a stable system call interface.

I've written somewhat at length about this:

https://www.matheusmoreira.com/articles/linux-system-calls


> It is not the only operating system in the "unstable kernel interface" group though. Linux is actually the only one with a stable system call interface.

It's wrong to say that Linux stands alone in having a stable syscall interface. FreeBSD [0] and NetBSD [1] both retain syscall compatibility for old binaries (the former apparently with some exceptions permitted); DragonFly BSD also appears to keep old syscalls in place. In fact, I only know of Windows, OpenBSD, and presumably macOS as mainstream desktop OSes without mostly-stable syscalls.

[0] https://wiki.freebsd.org/AddingSyscalls#Backward_compatibily

[1] https://www.netbsd.org/docs/internals/en/chap-processes.html...


When I wrote the article I searched the web for the kernel-userspace ABI situation in the BSDs. I found posts that contradict that.

https://forums.freebsd.org/threads/what-is-meant-by-abi.8622...

> I keep reading that the ABI of FreeBSD is constantly changing.

> Someone's system has random crashes and someone else says it's that pesky ABI changing again.

It seems FreeBSD has a reputation of changing binary interfaces. Even when they clarified that they meant the kernel-userspace interface, it was pointed out they're only stable for current releases. There seems to be some emulation but I can't figure out how hard of a guarantee that is.

https://docs.freebsd.org/en/books/handbook/kernelconfig/#ker...

> If the kernel version differs from the one that the system utilities have been built with

> for example, a kernel built from -CURRENT sources is installed on a -RELEASE system

> many system status commands like ps(1) and vmstat(8) will not work.

> To fix this, recompile and install a world built with the same version of the source tree as the kernel.

> It is never a good idea to use a different version of the kernel than the rest of the operating system.

How could the ABI be stable if you have to recompile user space when you update the kernel? Linux is said to be able to run binaries from the 90s.

https://old.reddit.com/r/BSD/comments/8vysxg/a_question_abou...

> FreeBSD breaks syscall ABI compatibility with some regularity.

> Like Windows and unlike Linux, the compatibility layer is considered to be libc, rather than the syscall interface.


FWIW your website is unusable on desktop: the font is unreadably huge. And while Firefox's reader mode does seem to mitigate the issue correctly, Safari's cuts off at the second paragraph.


Thank you. I'll try to fix it. I gotta get better at this.


I reduced the font size to 24px. Does it read better despite the margins?

I appreciate the feedback. My intention was to maximize the use of space with big fonts even on the desktop but there's no point if the resulting text is unreadable.


Improved the look of the site as well while I was at it.


We need more fat kernels and less libcs, libc ties us forcibly to the C legacy. Time for more type safe and richer interfaces with the kernel.


What do you mean when you say "fat kernel"?

Also, FYI, OpenBSD is never going to stop being written in C, and is never going to introduce a language like Rust into the kernel [1], so there's little point in wishing for this.

If you wish to rid yourself of legacy of C (and therefore that of Unix), then OpenBSD, which is Unix (or derived from it) and will always be written in C, is not a good operating system for you.

Edit: Changed to be less rude; it wasn't my intention to be rude.

[1]: https://marc.info/?t=151233221700001&r=1&w=2


I'm not the person you're replying to, but I don't really mind the kernel (or anything else) being written in C as such, but interacting with C from $other_language can be rather painful for various reasons. In Go the reason they do what they do is because integrating it with how the Go scheduler works is tricky, and getting rid of libc makes cross-compilation a lot easier.

So in short, it's a pragmatic thing more than anything else.

(Aside: that thread is a little bit outdated by the way, as I do believe there's a reasonably complete Rust coreutils implementation now. Not that I think Rust is a good fit for OpenBSD though – I wish people would stop conflating "safety" with "Rust".)


I understand. I don't actually like it, and I wouldn't want Linux to adopt the same practice; exposing raw syscalls in assembly rather than libc functions allows for a more flexible runtime, since C is a lot more limited (although that in itself invites greater security risk). I was saying that because, if you're asking for more type-safe kernel interfaces and frown upon uses of C, then the OpenBSD team seems like the last people you'd want to talk to. They're true Unix believers, love writing C and assembly, and dislike modern programming languages.

> Aside: that thread is a little bit outdated by the way, ...

Yeah, that thread was more to show that they really dislike the idea of replacing C with memory safe language, and instead say that people who can't program safely in C shouldn't be programming.

bytevolcano:

> I've always subscribed to the idea that too much safety results in too may idiots, and the same is true for all these "safe" programming languages. "Oh I don't have to write any form of bounds-checking, because the language will do it for me."

Nick Holland:

> Idiots who shouldn't be coding, coding. "safe" languages being trusted to be safe when in the hands of idiots. Like you said.

> The more I see of "safe" languages, the more I love assembly. Most people who call themselves programmers...shouldn't.

bytevolcano again (in the context of memory safe languages):

> A good programmer won't even need these languages in the first place. Case in point, the entire OpenBSD dev team. :)

They'd probably be fine if Go dropped support for OpenBSD altogether; they don't seem to have a high opinion of managed languages or their users.


Right, I didn't read all messages in that thread before (why don't these archives just present the entire thread in one page?)

In the end it's just random people posting stuff; you see that on HN as well, but also "it should literally be forbidden to use non-safe languages" on the other extreme end. bytevolcano seems to have just a few posts to the mailing list. Nick Holland is more involved (nick@), but mostly in documentation and that kind of stuff. There are no actual code commits beyond "add fstab example" and that kind of thing.

I happen to know tedu@ (actual OpenBSD developer) does a lot of Go stuff. I don't want to put words in his mouth, but I'd be surprised if he would come close to agreeing with that kind of stuff.

Unfortunately OpenBSD has always attracted its share of "ima very smart for using openbsd unlike poor dumb linux penguins!11" type of people.

In general I don't think OpenBSD people in general would be against safer languages. I mean, they did a lot of work to make a "safer libc" and it includes Perl in base.


I see; I didn't realize they weren't actually OpenBSD developers. That completely changes things.

I had assumed you needed permission or else you'd be blocked for posting on a mailing thread, or something like that, so I perceived everyone there except the asker of the question as an OpenBSD developers.

I'm sorry; I didn't realize I was spreading borderline libel.


I recommend against attempting to divine opinions of “the OpenBSD team” from mailing list randos. Theo is the only OpenBSD developer in that thread.

> They … dislike modern programming languages.

You based this statement on the words of random commenters. And yet OpenBSD maintains packages for these modern programming languages, and OpenBSD developers spend their free time actively sending patches and bug reports to these language projects to keep them running on OpenBSD. Why not look up the opinions of those people before writing sentences like that?


I didn't know they weren't OpenBSD developers, or that they made patches like that. I'm sorry. I would delete my comment, but it's too late to do that.


Wow, that discussion does not warm me at all to OpenBSD. A lot of those responses seemed way more abrasive than they needed to be.

It's interesting, though, that at least initially, the only examples of memory-safe languages were ones like Java or Python, which are both very high-level and come with a lot of runtime tradeoffs that would practically make them unsuitable to tools like ls or cat that need to be run quickly and a lot. (Also interesting: this never seems to be given as a reason not to use memory-safe languages.) Later on Rust and Ada are discussed, but it feels much more cursory, except in the final comment.

The big reason to use C seems to be that it provides a signal that the developer is competent - you need to be at least good enough to learn C in order to contribute to OpenBSD. And more specifically, if you would rather write code in a non-C, memory-safe language, you are probably a bad developer who should not contribute to OpenBSD. To me, this intuitively feels like quite a weak argument (are there so many people clamouring to contribute to OpenBSD that they need this kind of gatekeeping mechanism?) but it's pretty much the main one that gets repeated. The more convincing argument at the end is that, practically speaking, most of the benefits of memory-safe, low-level languages like Rust or Ada can be provided by static C linters/checkers, and so they don't add much value overall. This seems very much the opposite of what some other companies are saying, e.g. Google and Microsoft, which is that even with static analysis of C/C++, Rust is still dramatically reducing the number of memory safety issues they're running into.

I find it interesting that they don't bring up what seems to be the more obvious reasons not to bring in memory-safe languages, which are mainly the classic "rewrites are hard, here be dragons" arguments - the existing tools have been hardened through years of use, and rewriting them is more likely to add new bugs than remove old ones; new languages/tools means new knowledge which means having people on hand familiar with the new languages to understand what's going on; reimplantation tends to be slow initially, which eats resources and prevents ongoing maintenance and development; etc.

My main takeaways from that discussion are (a) I really hope I never have to work with the OpenBSD maintainers, and (b) their main arguments seem more about maintaining the exclusivity of their club than about the technical details of any transition.


> Wow, that discussion does not warm me at all to OpenBSD. A lot of those responses seemed way more abrasive than they needed to be.

It's pretty much on-brand for them. The OpenBSD project started when the founder was kicked off of the NetBSD core team, allegedly for being rude and abrasive to users. At one point, both the FreeBSD[0] and NetBSD[1] mailing lists blocked traffic from this individual because they claimed he threatened to aggressively spam their mailing lists.

0 - https://mail-archive.freebsd.org/cgi/getmsg.cgi?fetch=56044+... 1 - http://mail-index.netbsd.org/current-users/1996/10/20/0004.h...


Some folks are abrasive when young but grow out of it. I haven't really kept up with Theo's antics of late but have you checked into it? Not saying that kind of behavior is ok but we all "grow up" at some point. Saying as a major open source project co-founder.


From three years ago, when Theo would have been in his early fifties: https://marc.info/?l=openbsd-misc&m=157819616927844&w=2


A kernel is not a programming languages. Type safety is something enforced by a compiler or interpreter, while accessing a system call interface is always something that can be expressed in terms of a sequence of assembly language instructions. Doesn't matter whether you're accessing a C API or a syscall mechanism.

Forcing usage of libc is actually an effort towards type safety. `syscall` just takes a syscall number and an unspecified amount of arguments of unconstrained type. It's the opposite of being type safe.


It does matter. You don't have to link with the kernel, or support the C ABI. You're still tied to C with libc. It's especially bad on Linux because glibc is so awful.

Forcing libc has absolutely nothing to do with type safety. How do you think they invoke syscalls in libc? There's no more type checking of syscalls there than there is in Go's standard library. From a user's point of view it's identical, except you don't have to worry about endless glibc version errors with Go.


Glibc and the situation on Linux is not involved at all here; we're talking about OpenBSD that takes invoking system calls via libc very seriously.

The libc can enforce some minimal sanity checks on the syscall arguments. For example the number of arguments and some early validation to avoid entering the kernel if failure can be foreseen. But the main utility is to make it more difficult to use Return Oriented Programming to build exploits.


> It looks like golang is going to have to deal with — again — OpenBSD treating libc as the interface with the kernel instead of syscalls being the interface with the kernel.

Something they wouldn't have to do if they'd heeded the warnings they got since they first started going raw syscalls on non linux systems.

But as usual, go is uniquely american, only doing the right thing after it has tried everything else.


You could also describe it as doing the sane thing (avoiding hardcoding against the libc ABI through FFI) until the only sane option is removed. Did you know many libc APIs are preprocessor macros?


> You could also describe it as doing the sane thing (avoiding hardcoding against the libc ABI through FFI)

No, you could not.

On pretty much every system but linux the libc (or equivalent) is the officially supported API to the kernel. Bypassing it is not supported and thus definitionally can not be sane, that's like declaring that doing the sane thing is avoiding hardcoding against the front door and entering your home through the roof instead. The front door is what you're supposed to use, do hardcode against it. Same with libc.

And Go has been going "ouch" then putting its fingers straight back in the socket from the start, as it keeps trying to work around libcs on all the platform where it is not supported.


It's not about hardcoding, it's about the fact that libc isn't even an ABI, really. It's a C source API, which compiles down to a secret ABI that you can't know (you just have to expand all the macros and see what you get). That's sort of OK if every compiled program in the world is written in C, but as soon as you want to start writing in another language it kind of falls apart.


On linux the stable API and ABI that you are supposed to use is also just libc. But due to various reasons people think that the libc ABI is less stable than how stable it really is.

Edit: NT with its split OS API dll and separate libc is its own can of worms. You can have multiple instances of libc in the same address space, which among other things implies that you cannot safely malloc() in one module and free() in another, and don't even think about passing FILE* around.


Linux doesn’t have an official libc. Raw syscalls are its only official API.


My point is that libc as the only system API is not a sane choice, and appeals to tradition don't make it less insane.


> My point is that libc as the only system API is not a sane choice

That might be your new point after having yeeted the goalposts out, but it's definitely not your original statement.

> and appeals to tradition don't make it less insane.

This is not an appeal to tradition, it's how things work. If the authors and maintainers of the system tell you something is the supported API, wilfully bypassing that supported API is not "the sane thing", especially when you complain that your bypass blows your face off down the line, then you're an asshole too.


> go is uniquely american, doing the right thing after it has tried everything else.

That doesn't seem like anything common to an American way of doing things. What an odd statement.


It's a play on a quote often attributed to Winston Churchill: https://quoteinvestigator.com/2012/11/11/exhaust-alternative...


Bizarre nationalistic "they're doing it because they're Americans" rantings are still not okay even if they're a "play on a quote". I can "play" with all sorts of quotes to say all sorts of idiotic things.


I guess. I'm proudly American and I thought it was funny, but everyone has different tolerances.


I don't use Go, but I had the exact same reaction; C is the source of most of the problems and this is just codifies the use of C. The whole thing has a security by obscurity smell.


Your comment seems to conflate a language with an ABI. To what end? What is inherently insecure about the C ABI? Why is using a syscall interface any more secure?


I'm not conflating anything. Why do you need to burden a language with a C ABI when you can just invoke the language independent mechanism directly?


The C ABI is influenced by C's type model, but so is OpenBSD itself. There is no truly language independent mechanism. If a programming language implementation can't interface with a C ABI, how can it interface with the language independent mechanism?


> go is uniquely american, only doing the right thing after it has tried everything else.

Perhaps this is why most innovations are American. We don't automatically fall for the bully pulpit of the gnostic class.

> Something they wouldn't have to do

Yea, but they'll get it done anyways, and the language will continue to be excellent. I'm sure Google can absorb the engineering challenge without subtracting anything from us or other languages.


> OpenBSD treating libc as the interface with the kernel instead of syscalls being the interface with the kernel

Which is a reasonable thing, given that the libc interface is defined by a widely used IEEE standard while the kernel interface is not.


libC is an extremely leaky abstraction. Programming to it assumes you have a runtime that supports constructor functions, POSIX errno and locale, a global heap allocator singleton, and more. The design space of _Hello World_ is massively constrained by libC.


OpenBSD’s position is far from unique. It’s shared with MacOS (and iOS).


And windows.

And that's just the platforms which technically enforce it. Linux is essentially the only platform which actually supports raw syscalls, in the sense that it's considered a normal system API.


And Windows, in a sense: not that libc is the interface, but that the assembly-level API is not the interface.


And almost every other OS still in use today, what Linux does is reminiscent from OSes like CP/M and MS-DOS.


And seL4.


> I wonder if there could be some way to sign a dynamic library to allow it to create direct system calls and then pass that as a kernel command line argument at boot?

That sounds like an additional knob to tweak. The OpenBSD project is famously opposed to such knobs.


I am sure Cosmopolitan is going to love this change.


I would assume they already have ways to bounce through a dynamically linked system library since cosmopolitan works on macos, and even more so windows.


That's true, but it's a point of pride in Cosmo's marketing that it produces statically linked binaries acroscreate, least several operating systems.

It can dynamically link the system32 DLLs in Windows and probably could OpenBSD's crt, but dynamic linking in general doesn't work. `dlopen` is no-op, and this has made Cosmo graphics extremely difficult so far.


Is that a self-imposed limitation? It sounds eminently possible to add some kind of work-around for dynamic linking. Maybe not for a general case, but for creating a GUI for instance.


Yes, there actually is a to-be merged fork of Cosmopolitan that makes Dear ImGui work, but no dynamic linking is generally a big limitation for GPU acceleration.


What about OSes that are not built around monolithic kernels and the syscall() paradigm?


Follow-up mail from Theo showing what the full removal looks like.

https://marc.info/?l=openbsd-tech&m=169842095809570&w=2


I don't know a lot about this corner of the OS. Why are syscalls in libc, instead of something like a libsyscall? I could see why a language might want not to depend on what's at least notionally the C runtime. Is the fact that the kernel interface is in libc an accident of Unix being written in C, or is there something more fundamental there?


Libc is really a conflation of at least three different notional libraries. The first library is what its name suggests it is, a standard library for C. Another part of the library is in providing the userspace portion of system services--things like handling static initializers, dynamic loading, or the userspace side of things like creating new threads (not to mention, the actual raw functions you call to get the kernel to do something). The final part of the library is a collection of userspace services which are language agnostic, you might choose different implementations, but you'll always assume are somehow present--libm and malloc are the goto examples here.

As for why the userspace system service library is part of libc instead of being a separate libsyscall or libkernel, that probably is due to Unix being a C operating system--written at a time when most operating systems also came with their own system language. It's definitely not the case for all OS's that the C runtime library is the same as the libsyscall/libkernel--most notably, on Windows, the former is MSVCRT*.dll and the latter is kernel32.dll (or ntdll.dll if you're looking very specifically at syscalls themselves, but ntdll.dll is largely an unstable interface).


That's the best explanation I have ever read! I even knew these things beforehand, nodding along, but now I can explain it also to someone else in a coherent way instead of rambling up implementation details like a madman.

Thanks!


It's that libc is considered the stable interface for userland in general on openbsd.


In pretty much every OS of the unix tradition. libc is the API, with a stable ABI. Although Windows has a something pretty much identical in — I believe — ntdll. The name and API differ, but the intent is the same.


No, Windows has stable API and ABI named Win32 API. This comes from times of 16bit Windows and works also in x64 / ARM / ARM64, previously it worked in Alpha / MIPS / IA64. This API is implemented in kernel.dll user.dll gdi.dll advapi.dll and similar, it does some stuff, but mostly forwards to the NT API. Beware, kernel.dll is user space component despite its name (historical reasons). NT API is undocumented and not meant to be used by user programs, it is not stable, it lives in ntdll.dll, it does syscalls to the kernel: ntoskrnl.exe. Windows doesn't have a libc (for user programs, it has private one for its own programs), Visual Studio has a libc. Each version of Visual Studio (roughly) has its own libc named msvcrt.dll msvrt100.dll msvcrt140.dll and similar, it hosts the C and C++ libc, it could be linked statically for various benefits and drawbacks.


Windows does actually have a universal C runtime library now, but it's pretty recent; only Windows 10+ https://learn.microsoft.com/en-us/cpp/porting/upgrade-your-c...

A big motivation for it was security posture; it means that Microsoft can now ship security updates to UCRT that everyone can rely on rather than a ton of extra surface area through various multiple versions of runtime libraries.


It had (has) an unsupported, crippled, unversioned msvcrt.dll which if you used it very carefully with a subset of functions, you could write programs which worked fine on Windows NT and up.


FWIW, large swaths of ntdll are documented and supported these days, for performing work that can't be expressed via the win32 API like raw disk manipulation.


Except notably Linux where the kernel ABI is the thing that’s stable.


glibc actually goes through quite a bit of effort to remain backwards compatible. There have been times when mistakes were made here, but the extent to which glibc does not have a stable ABI is overstated. The only annoying thing about glibc and ABI stability is that the way it achieves that is via symbol versioning which makes it annoying to target an old glibc on a system with a new er glibc.


I didn’t mean to imply that glibc’s API isn’t stable. Just that the stable API boundary for the kernel on Linux is the syscall ABI, not libc.


Another stable ABI boundary is anything produced by glibc that can be conceivably placed into memory shared by different unrelated processes (ie. Posix IPC primitives). The requirement for backwards compatibility and architecture variant compatibility (32b/64b…) is the largest reason why things like pthread_mutex have somewhat large overhead and why it is worthwhile to invent various iterations of “futex in userspace”.


Ah sorry, I misunderstood


No worries. My wording was muddled.


Is that different from Linux/Mac/etc?


It is not different on mac (although there libc is just a subset of the wider libSystem).

It is different, uniquely so, on linux: on most unices the kernel and libc are developed as two sides of an entire system, both being updated in lockstep when the system is updated. As such there is no real concern about keeping the syscalls stable, if you need to change it on the kernel side you update it to match on the libc side and you're done, everybody is supposed to use the libc.

Not so on linux, the kernel and the libc (most commonly glibc) are developed by entirely different groups which don't necessarily like or communicate with one another. As a result, on linux syscalls are a stable API, and direct syscalls are an officially supported method of interaction. In fact its sometimes necessary as the libc might decide not to expose a syscall.


> It is different, uniquely so, on linux: on most unices the kernel and libc are developed as two sides of an entire system, both being updated in lockstep when the system is updated.

Do FreeBSD [0] and NetBSD [1] not fall under "most unices"? They similarly retain backward compatibility in their syscall interface, so that old binaries with old libcs can run on newer kernels. Linux does not stand alone here.

[0] https://wiki.freebsd.org/AddingSyscalls#Backward_compatibily

[1] https://www.netbsd.org/docs/internals/en/chap-processes.html...


Thanks for that. So although the name is "libc", it's more like "lib-how-to-use-the-kernel", too?


Yes, for historical reasons it bunches syscalls (section 2) and C library functions (section 3) into a single binary.


I'm pretty sure if you just repackage the syscalls into a basic module of your language, you will get more stability than by linking to the libc.

I guess people don't do it because the difference is minimal and they use a lot of other features from the C runtime too.

EDIT: Ops. Not on BSD! The entire thread is about BSD and here I am mindlessly talking about Linux.


Only on systems where the syscall ABI is stable, like on Linux. Others, like macOS and Windows[0], can and will change theirs between releases. OpenBSD even goes one step further and actively prevents code other than libc from performing syscalls[1]

[0] I seem to remember that this changed in a recent Windows version, but I couldn't immediately find a source.

[1] msyscall(2) or, if you don't have an OpenBSD system at hand, https://man.openbsd.org/msyscall


Why not remove syscall instructions altogether? When libc wants to do something, it traps on an undefined instruction and then the kernel looks at the program counter to see what it should do. Seems like this would be the ultimate application of this line of thought…


That is basically what we used to do, int 0x80 and such. I mean, I guess you do not read the PC since that is much more brittle than just having the caller say what they want to do, but it is structurally the same.

Turns out, having a dedicated syscall instruction and trap pathway is just better design. Unlike a regular exception, this is a deliberate change of control to the kernel, so you can enforce a much stronger ABI requirement. In particular, you can define it to use a standard function call ABI with respect to preserved and non-preserved registers making it literally look like a standard function call.

For similar reasons, having a dedicated hardware pathway like on x86-64 is also just better design. System calls are a synchronous, voluntary transfer of control that is expected to return in contrast to (1) interrupts which are a asynchronous involuntary transfer of control and (2) instruction stream exceptions which are a synchronous involuntary transfer of control with no guarantee of return. This fundamental distinction can be leveraged for more efficient and simpler implementations.


I don't think that helps much. OpenBSD already only allows syscalls originating out of the libc .text section, so whether the trap itself comes from a syscall instruction or some other trap mechanism doesn't really improve security AFAICT.


Yeah but it sounds super cool doesn’t it!


Every time I've seen a dev team go down that road, it's come with rather unfortunate unintended side effects.

https://devblogs.microsoft.com/oldnewthing/20041215-00/?p=37...


Yeah but think of the attacks I thought of in the shower that it mitigates


My understanding is this was done by one system, which then made the architecture's life hard because now they lost half their opcode encoding space due to it being used as pseudo-syscalls by their largest customer.


Just one encoding, or maybe we’re thinking of different systems?


Are there any performance implications on such traps on x64 or ARM?

IIRC, PDP11 and VAX used traps as the way to call the supervisor, and it was pretty cheap.


You're asking the wrong questions of OpenBSD :)


Fair! :)

But interesting nevertheless.


I think ARM (or at least ARM64) does the trap thing too.


For language runtimes that don't normally have to deal with C baggage having to drag libc into your address space and going through libc code for syscalls makes this a less secure platform.


+1.. Why not create a standard/stable syscall interface instead of pushing an anachronistic libc interface.


How I would design it would be to have a system call which registers the addresses, in user space, where system calls are allowed to come from. After that, any system call not coming from an address in that list will fail.

The list would be editable, but there would be an operation which seals it from further editing.

With this feature, we could load a new plugin into a C program with dlopen, and it would not be able to make syscalls of its own, other than through the existing C library, no matter how prim and proper its call sequences look like.

Possibly, ranges could be used instead: syscalls can originate from several registered address ranges, and that's it. Programs that create dynamic code on the fly, which can make system calls, could allocate trampolines in a registered area, for that purpose.


Can someone explain the significance.


OpenBSD has been putting in a lot of work lately to harden the syscall ABI; a large component of that work has been constricting how a syscall is invoked from user space as a defense in depth technique to make shell code style exploits more difficult. That's previously taken the form of techniques like only allowing syscalls to be invoked from the libc .text section.

This work is removing a very indirect morph of syscall where the arguments/sysnum are in a struct in memory, making it harder for exploits to invoke weird versions of syscalls on their own terms.


But aren't shellcode style exploits already fairly rare with W^X, so most end up using return-to-libc style attacks? Wouldn't CFI be a much better solution?


It's defense in depth. Removing a function like syscall(2) that's a "whatever syscall you want" function that is ostensibly white listed works very well when combined with CFI, and strengthens security of systems without hardware support for CFI.


OpenBSD users mostly don’t use systems with strong hardware CFI, so they make do with stuff like this.


Why aren't these changes made in kernel to keep the syscall ABI standardized and safe instead of requiring the use of an unsafe language wrapper? We should be discouraging more use of unsafe languages, not forcing it.


Then don't run a OS with a kernel written in C. That would rule out Windows, Mac, Linux and *BSD.


Using syscalls means you can replace the kernel. Using libc means you are locked into and FFI/C calling convention for anything other than C. Apples/Oranges.


There’s nothing wrong with the C calling convention, at least from a security perspective.


The C ABI doesn't necessarily force you to use C. Maybe to turn some pesky macros into proper functions. Bury the bindings deep in the language's stdlib. Done.

Under the hood, it's all assembly language instructions invoking an operating system written in C. Operating systems care very much about types, but can't offer type safety. Programming languages can.


A+

This is awesome.

Also, it’s rarely used in my experience. I wonder what kind of code would even notice?


Exploits, I guess is the concern.


Code that uses syscalls that aren’t in libc but doesn’t want to have to drop to inline assembly?


Right, so exploits.


Exploits are more than happy to drop to inline assembly.


Theo already prevented you from making the syscall by using the trap instruction directly because of in kernel checking that you got to the kernel via the right point in libc.


It is security theatre. The instruction pattern by which a system call is invoked is not a security attribute. You can't trust something just because it looks like a libc call.


It's an indicator whether they instruction sequence was concerned together by an exploit or whether it's normal code execution. Attackers will have to use more complicated and brittle mechanisms to still achieve their goal. It's an arms race, but so is security in general.


Just remove the kernel from the base install already?


> Piece by piece, I've been trying to remove the easiest of the terminal-actions that exploit code uses (ie. getting to execve, or performing other system calls, etc).

> I recognize we can never completely remove all mechanisms they use. However, I hope I am forcing attack coders into using increasingly more complicated methods

It's honestly so ridiculous an OS that claims to have security a a focus refuses to add even some sort of basic MAC/RBAC implementation. Even both OSX and Windows have had something for ages now.


These things are kind of orthogonal. OpenBSD maybe gets there eventually.

OpenBSD is like a very hardened safe, made of steel and huge bolts and locks. Very polished, very smooth and hard surface.

MAC/RBAC is like having security officers, interviews, checking of IDs, filling in forms and getting an OK from ones boss before performing work someplace in the building and so on.

Both these things can be good. But OpenBSD was always about making a small system as hardened as possible. Evidently, they aren't completely done yet with making the core as hardened as possible.

Windows has great architecture, but lacks instead severerly this hardness which OpenBSD possesses. What use is MAC/RBAC if someone can gain kernel access with a 0 day exploit?


> OpenBSD maybe gets there eventually.

Nah they won't. The devs have an irrational resistance to the very idea.

I disagree with your analogies. OpenBSD has a focus on auditing to remove all bugs, which is great, but they provide very little to help prevent what can be done if a bug is exploited, and they've certainly had no shortage of serious bugs.

> What use is MAC/RBAC if someone can gain kernel access with a 0 day exploit?

Kernel exploits are pretty rare. Most exploits are in userland.


I actually still think my analogy is apt.

Their safe is very hard, but once you are in, you are in. And I think I agree with your assessment, they aren’t likely to start creating MAC/RBAC solutions.


I think pledge(2) and unveil(2) would help to prevent "what can be done if a bug is exploited", yes?


Ever so slightly, but still a long ways off from proper mac or rbac support.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: