A more robust raw OpenBSD syscall demo | Hacker News Reader

A more robust raw OpenBSD syscall demo

nullprogram.com

74 pointssigna1147 comments

Why involve C at all? This is much cleaner in assembly

            .global _start
            .data
    what:   .string "hello\n"
            .set    len, .-what - 1
            .text
    _start:
            mov     w0, 1
            adr     x1, what
            mov     w2, len
            mov     w8, 4
    99:     svc     0
            dsb     nsh
            isb
            mov     w8, 1
    98:     svc     0
            .section        .openbsd.syscalls, ""
            .long   99b, 4
            .long   98b, 1
            .section        .note.openbsd.ident, "a"
            .long   8, 4, 1
            .string "OpenBSD"
            .long   0

especially when confronted with a soup of C flags and declarations, and still not being entirely sure what the C compiler is going to emit

How is that an issue? You inspect the generated assembler. Godbolt is the industry standard tool for that, but of course just use objdump on the command line if a browser UI isn't your fancy.

Mixing C is helpful because most code doesn't need to be written in raw assembly.

keeping assembly in asm files, c in c files, compiled by their respective compilers, then linked, has fewer footguns than inline asm

Yeah but then you have to maintain function interfaces between them in order to link them. The case in this article is for inserting one single asm instruction in an otherwise C codebase.

platform ABI is usually better and more clearly defined than whatever GCC & Clang feel like doing to inline ASM across versions and optimization levels

and inspecting the output is obviously easy. the hard part is figuring out which flags, pragmas & directives are required to get gcc to emit what you wanted it to emit when it doesn’t.

You also need to re-inspect it at every toolchain change/upgrade. In my experience, most programmers don't, and then we end up spending two day chasing some impossible-to-happen bug which has happened nonetheless because new Clang version had a regression in some obscure and rarely exercised part of its codegen.

No. You have a unit test to inspect it, if your other unit tests cannot cover this codegen bug. Asking programmers to re-inspect by hand is like asking programmers to run all tests manually after each commit. Of course most programmers don't.

Probably due to the audience.

I can grasp the example, but back in my days, if we wanting something faster than interpreted BASIC, Assembly was the way.

Folks nowadays start with Python and JavaScript.

Isn't "dsb nsh" or "isb" redundant for simple syscalls like write and exit?

Without them it'll segfault on QEMU, IIRC OpenBSD libc uses them invariably as well. I don't know how it fares on real hardware

Yeah, probably (although I am not sure) because QEMU doesn't fully emulate hardware behavior, but on real hardware some CPUs may internally handle these state transitions better. I suppose its inclusion is recommended to ensure correctness across all CPUs.

It may happen because without "dsb nsh" (Data Synchronization Barrier) and "isb" (Instruction Synchronization Barrier), QEMU may continue execution before the syscall fully completes, which causes a segfault or UB, but I am not entirely sure.

"dsb nsh" and "isb" ensures correct execution order which prevents speculative execution issues.

That doesn't sound right -- QEMU always works one instruction at a time, so we never start executing a following insn until the previous one is completed.

Yeah the segfault doesn't happen on other OSes, I believe it's an OpenBSD thing

So what causes QEMU to segfault without the synchronization? I have not tested it myself, however, so I cannot guarantee that it indeed does segfault.

The "dsb nsh; isb" sequence after "svc 0" is part of OpenBSD's mitigations for Spectre.

https://github.com/openbsd/src/commit/bbeaada4689520859307d5...

https://github.com/openbsd/src/commit/0c401ffc2a2550c32105ce...

https://github.com/openbsd/src/commit/5ecc9681133f1894e81c38...

If I'm reading the commits correctly, the OpenBSD kernel will skip two instructions after a "svc 0" when returning to userspace, on the assumption that any syscall comes from libc and therefore has "dsb nsh; isb" after it.

The inline assembly is not idiomatic. Today you should be using register asm. Here is a RISC-V example:

    register long a0 asm("a0") = arg0;
    register long syscall_id asm("a7") = n;

    asm volatile ("ecall" : "+r"(a0) : "r"(syscall_id));

    return a0;

This is an example where a0 is an in/out integer. For memory, change long a0 to a pointer to some struct and add a "m" input or "+m" in/out. It's even easier in other languages like Rust and Zig.

you can also out register preferences directly into the asm volatile line.

asm ( assembler template : output operands (optional) : input operands (optional) : clobbered registers list (optional) );

clobbered register list will give same hints to compiler... register keyword is also a hint not a fixed thing so i cant imagine its really handled differently?

might be wrong, but it seem earier to me to use all the features of the asm call itself rather than outside of it pre-specifying these preferences.

Actually nowadays ideally we would already caught up with ESPOL/NEWP from 1961 and use only intrisics.

Apparently that is a VC++ only thing.

MCP and the rest of the Burroughs Large Systems are an interesting road not taken.

> The inline assembly is not idiomatic. Today you should be using register asm.

Who's writing these idioms? I've always seen register variables as an alternative, not the default option, except for those registers which have no constraint code.

Probably the current state that GCC for Aarch64 and RISC-V simply don't offer individual register constraints like x86.

Register variables are idiomatic when you're locking down every single register anyway, which is always the case for system calls. If you're writing assembly where you want the register allocators help, by all means use regular inline assembly.

Doesn't GCC have long-standing bugs where it a) doesn't support assigning some select registers, and b) when you try to use them anyway, it instead completely ignores your annotations with no diagnostics, and c) when it gets reported on GCC's bug tracker, a maintainer would reply with "well, it's always been like that and everyone knows you can't use this register like that anyway" and close it as WONTFIX? I distinctly remember reading about something like that...

Perhaps, but I can't imagine its buggy for general registers like those used in system calls. I haven't seen that bug, but I don't doubt it. There's many obscure registers for various architectures that could bug out.

I’d really prefer using inline assembly tbh

This is inline assembly.

Normal inline assembly not the register pinning you’re doing

Idiomatic to you perhaps, to me this looks ridiculous and unnecessary, bordering on masochistic.

Seems like an interesting if maybe not practical protection to implement in eBPF for programs that never make a naked syscall.

Step one would be to ensure that every syscall has a wrapper. Place a uprobe at the start of that wrapper which, when hit, sets a per-thread permission bit and a per-thread-per-syscall permission bit in an eBPF map. Place a corresponding uretprobe that clears the per-thread-per-syscall bit. For each syscall place a kprobe which checks the per-thread table to make sure the thread is one which has enabled the feature, and which then checks to make sure the per-thread-per-syscall bit is set for that syscall. If not, sigkill.

Performance would probably suck but it seems like it would protect the syscall entrypoints enough to do some potentially interesting attack surface reduction. The question is really why you would do that there instead of by attaching to, say, the LSM hooks where you have stronger guarantees vis a vis userspace.

What’s the threat model this protects against?

"Attacker has ROP and shouldn't be able to make arbitrary syscalls".

Seems mildly useful if you have a really flexible syscall you can't forbid (ioctl, say) but which you only use for a specific narrow purpose.

If they can ROP they can jump to a syscall instruction with controlled arguments

The point of pinsyscall is that they have to jump to the single entry point for that syscall, rather than any of the ~200+ syscall instructions littering the address space. ALSR makes finding an entry point difficult, but that's easier if you only need to find any syscall instruction, rather than the specific one for the syscall you're invoking. The rationale is explained here: https://undeadly.org/cgi?action=article;sid=20230222064027

I’ve already expressed my opinion on this particular mitigation elsewhere and I’m pretty sure you’ve read it so I won’t go into it again here. I was more interested in the scheme that the other comment was constructing.

The point of what I spelled out above is that they can jump to the instruction but the kernel will kill the program if they don't go through the function up to that point. That allows you to restrict the arguments to the syscall at the point of call.

Ah, so it’s like a poor man’s BTI

I like how Go provides the "syscall" package in the standard library. It's OS/ARCH specific, and takes every precaution to call the raw thing safely - notably syscall.ForkExec on UNIX platforms (digging thru the source made me really appreciate the scope of their work).

It "feels" very low-level in an otherwise very high-level language, but provides enough power to implement an entire userland from initrd up, libc- and asm-free (check out Gokrazy and u-root).

OpenBSD and Go have always been at odds here. Go really wants to produce static executables whenever possible, and do syscalls directly; OpenBSD really wants to prevent user programs from accidentally creating gadgets. I guess they've settled on dynamically linking ld.so?

And the MacOS API is similar.

It's a bit more complicated. Ideally you'd only need libdyld.dylib and libSystem.B.dylib or so, but...

    $ echo 'package main; func main() {}' > nop.go
    $ go build ./nop.go
    $ DYLD_PRINT_LIBRARIES=1 ./nop 2>&1 | wc -l
          44

OpenBSD & macOS philosophies are often surprisingly aligned in certain ways, but OpenBSD is simple, macOS is comprehensive (and - TBF quite bloated).

That's a very fair assessment of Apple – IIRC a lot of early idiosyncratic hardening work in XNU world was done for the launch of the iPhone OS App Store, but it's only been (relatively) recently that they've started long-term initiatives systematically harden macOS on top.

I think it's also because a desktop is broadly considered a "power tool", you absolutely need it for systems development, and that is forever at odds with "iPhone-level" lockdown/security. Even if you'd start from scratch, you'd need a solid theoretical foundation (e.g. capabilities), and probably build up new metaphors on top.

I like what OpenBSD did with pledge&unveil. It gets "the first 80%" of the work done, it's easy for the developer, and invisible to the user.

maybe to make it more concise binary you can do something like nostdinc nostdlib or freestanding and add own linker file to be more explicit about how binary is build and what to /discard/. also max page size and other linker flags can impact. its sometimes a bit trial and error to get it to spit out a small binary with only the code u really wrote.

own linker file and manual linking step imo is key. (i use gcc/ld). if u let gcc spit linker file for modern platform, u can see its full of clutter... most of it unneeded. u can also strip that one down, but i am sure u know what elf sections to put and omit, and you found all the bsd specific ones.

in the linker step u can also add symbols / calculate offsets.

in gcc u can also in the c code use attribute section('bla'). not sure if its handy in this case but maybe it'll ease somewhat these things or bring it back more into C :).

cool example :) remebering struggle tirleesly tryin to find out how to run a raw syscall on openbsd. a lot of man pages, readelfs and headaches i was so happy to get my exit code hahah

I'm sorry but I got stuck on the first sentence "Ted Unangst published dude, where are your syscalls? on flak yesterday" and as a long time fediverse operator I got insanely curious about "flak".

So I ended up on the flak tag of this blog[1], but I still can't figure out what it is. I can find no links to any source code, or any service description. Even though the blogger mentions flak being their "signature service".

I'm guessing it's a blogging platform, with ActivityPub support, but I can't find any info about how it's used.

1. https://flak.tedunangst.com/t/flak

The post you're looking for is about how they also NIH'd a source code browser: https://flak.tedunangst.com/post/humungus

Flak is a blogging platform... I think. He also has Honk, which is a ActivityPub server?

Reply on news.ycombinator.com