Basic proxy implementation using io_uring | Hacker News Reader

Basic proxy implementation using io_uring

git.kernel.dk

132 pointsmfiguiere53 comments

I really need to get my head across some of these things. I've been meaning to also look into https://www.kernel.org/doc/html/latest/networking/tproxy.htm... (in regards to udp(quic)/ztunnel-esque behaviour). This will serve as a great reference, thanks.

Come on HN. Do better. Silly conversations about the number of lines? How about any C experts and networking pros weigh in on this? Has anyone built it and tested it?

So, is io_uring used anywhere yet?

Anything using glommio [1].

[1] https://crates.io/crates/glommio/reverse_dependencies

I used it for some high datarate record applications at work.

Does the job nicely without even half the boilerplate of aio if you use liburing.

I think there's still a lot of concerns related to security https://www.phoronix.com/news/Google-Restricting-IO_uring

Were any of this vulnerabilities disclosed? I saw that Google blocked it in some places, but without fully understanding what are these vulnerabilities it's hard to make a good judgement on whether it is usable or not in some application.

There are practical backward compatibility and security issues currently that prevent widespread deployment. I’ve been leaning into io_uring going forward because it is clearly the future and brings a lot to the table. It still can’t be used in many environments but I expect it to be mainstream in 2-3 years. Niche, but not for much longer.

Most software doesn’t properly use the old io_submit facility and that has been around for decades, despite the large performance improvements possible. Even when io_uring is widely deployable, most software won’t use it.

What are the open security issues?

This only applies to Linux distro packages though right? Servers, embedded, mobile, etc only target specific kernels and so can avoid the backwards compatibility issues and in many cases have all of thr latest security patches.

Yes.

But sometimes it means that what you have is software that can't run on recent enough kernel to have io_uring (^_^;;)

The old aio API is very inefficient and has awful quirks, such as not working on regular files opened without O_DIRECT. Not surprised no one used it

Am I wrong? Just noticed the downvote. That is my understanding of the old AIO kernel API anyway. The original io_uring paper describes some of the deficiencies of AIO. And the fact that the API is broken on non-O_DIRECT regular file descriptions is well known.

You're not wrong.

I’m using it for all I/O (file, socket, SPI and UART) in an embedded application written in Swift. Wrappers here:

https://github.com/PADL/IORingSwift

What type of app can afford to run on Linux (not bare metal) but still needs such low level performance optimizations?

Lowering CPU usage can save battery. Just because you are running Linux doesn't mean you don't have a power budget.

Among many other projects I touched, we had a database engine which planned to move to io_uring as soon as it could be made available (it wasn't distributed outside company, so it was a question of internal infrastructure)

PREEMPT_RT also includes a lot of possible low-latency work.

io_uring also provides ridiculous performance benefits for normal server code, too.

Context switches are expensive. Often more expensive than operating on the data. io_uring providews a major reduction, or even near elimination of context swithes for io. It provides big win in a lot of server contexts in terms of packets/iops/throughput.

Initial motivation was lack of non-blocking SPI I/O in Linux user space. Then it made sense to use it for everything, it’s a straightforward mapping to Swift’s structured concurrency, so why not?

My team uses it as part of a high performance storage system at work (but only for disk IO).

libblkio, used by qemu for some block storage, uses it. Independently, qemu also uses io_uring for some Linux file operations.

My experience with io_uring is that it requires a fairly fundamental change in the way you architect software, which makes it hard to retrofit it to existing code (eg. code using poll). If you have a greenfield project that requires very high performance and will only/mainly run on Linux then you should strongly consider it. I'd actually love to hear peoples' thoughts on whether there is a good way to retrofit code.

io_uring is wonderful.

io_uring has produced many security problems. Google has found it necessary to either completely forego io_uring or severely limit its use to trusted code[1]. This policy includes Chrome OS, Android and Google Cloud. Docker has also blocked io_uring syscalls (by default) from being called by containers for the same reason and, although they've flip flopped on this at least once, I believe io_uring is currently blocked.

This is highly disappointing to me, but also not surprising given the nature of io_uring. Solving it will require more design rigor, probably some performance compromises and possibly some degree of formal validation.

[1] https://www.phoronix.com/news/Google-Restricting-IO_uring

“Faster than possible” is a phrase I picked up an awfully long time ago. I hope someone finds a reasonable middle ground that’s still several times faster but not Swiss cheese.

The security problems for io_uring or for any other system calls exist only when an adversary has access to those system calls.

Whenever you write an application for internal use, where there is no possibility for an adversary to interfere and control in any way the arguments of the system calls, there is no reason to avoid io_uring or other risky system calls.

Defense in depth through sanboxing is there to make sure that even an attacker that get some amount of undue privilege will not be able to do harm. At this point we've learned that all existing systems will have vulnerabilities of some sort, so defense in depth is the only credible security posture.

You’ve written the same message, almost verbatim multiple times in this thread.

Google also writes applications in golang, maintains their own version of the Linux kernel and runs a browser monopoly. What works/doesn’t work for them shouldn’t be treated as the only standard worth considering. FWIW, fb makes plentiful use of it as I understand.

Docker doesn’t block IO_URING syscalls, you just have to be able to lock memory.

Clearly you have an axe to grind about io_uring, which is fine, I guess? But you’re really going out of your way to make it seem like it’s the worst thing ever, which is a extra.

> You’ve written the same message, almost verbatim multiple times in this thread.

Until now I had written exactly one message in this thread. If others have made similar statements or cited the same link it's down to the fact that some of us care a great deal and are enthusiastic about io_uring, so this relatively recent information is top of mind.

> Docker doesn’t block IO_URING syscalls

"Update RuntimeDefault seccomp profile to disallow io_uring related syscalls" from containerd-2.0.0 beta2 release[1], 3 weeks ago... (bear in mind there is considerable history here.)

What a bizarre reply.

[1] https://github.com/containerd/containerd/releases/tag/v2.0.0...

Does Facebook have the same threat model? They don't run a public cloud nor host enterprise data.

You’ve written the same message, almost verbatim multiple times in this thread

Comment history says otherwise. If you’re going to make ad hominems, at least make ‘em accurate.

I just checked one of our GCP VMs, and it seems it's supported there:

    # grep -i uring /boot/config-$(uname -r)
    CONFIG_IO_URING=y

(message #3 attributable to me, since someone appears to be counting...)

"Google production servers" is what I should have written. io_uring on your GPC VMs has no security implications for Google itself.

would be interesting to see some performance benchmarks - I do care on TCP streams mostly for now, cases I do cover by HAproxy/Nginx in the systems.

> basic

1300 lines...

also, the line numbers dont actually align with the lines:

https://git.kernel.dk/cgit/liburing/tree/examples/proxy.c#n1...

FWIW it's about 800 lines excluding comments.

and yeah, this is why you don't abuse side-by-side divs ... actually, that's a table ... and instead just make your number unselectable (there are several ways to do this).

That's pretty small for a complete C application...

It aligns perfectly for me. Must've been mostly tested on Firefox.

I am on Firefox...

Doesn't work on Chrome too.

Needs .linenumbers { line-height: 12pt } but I think vertical line spacing within text node cannot be precisely controlled and is browser dependent.

There's something super weird going on with those line numbers. They don't align for me either (Firefox on Windows 11). In the web inspector, the line numbers are rendering with Courier New as the font, whereas the code itself is `monospace`.

The weird thing is both are controlled by a single CSS rule,

    div#cgit pre {
      font-family: "Source Code Pro", "Courier New", monospace;
    }

(Changing to just `monospace` fixes it.)

I'm not sure how two elements could have different fonts with that. I'm possibly missing something obscure, but it really feels like a browser bug.

Great sleuthing, the missing piece of the puzzle is that the file contents are inside a <code> element while the line numbers are not, and <code> elements have a default font so they don't inherit the font from their parent element. Changing the selector to the following fixes the issue:

  div#cgit pre, div#cgit code { ... }

(The buggy CSS is not present in the the official cgit repository, so I assume the owner of kernel.dk is running a patched version of cgit.)

Reply on news.ycombinator.com