touisteur

There is limited chaining capability in io_uring that can be an actual gamechanger if your chain of ops can be fully in-kernel.

An intern of mine wrote a whole tcp-checkpoint-(pause-ingress-and-egress-and-the-socket-save-all-state-unpause-everything)-and-send-full-state-on-other-socket (which was a dozen or so ops - netlink writes, ioctls, set/getsockopt, read/write calls...) in a chain - all in one command-queue-write IIRC.

Performance was as good as an ad-hoc kernel module, without any ebpf. We just had one kernel patch to handle some unhandled syscall (getsockopt ? Setsockopt ? Ioctl?) (that we sadly didn't upstream... 2 years ago) and we were done. Really a great system for batching syscalls.

It made me wish for a form of DAG for error-handling or for parallel operations in chains...

Good fun. Now I wish RT cores would be programmable with some form of PTX, but for now it's Optix or die. Managed to do fun stuff with it but it's like pulling teeth.

I discovered his rant(s) about this recently and indeed thought it was interesting in the light of io_uring. If there's a similar compendium of Linus rants against io_uring I'm interested.

Yes Linus has been ranting for decades against O_DIRECT saying similar things (aka better hints on pages and cache usage).

The notorious archive of Linus rants on [0] starts with "The thing that has always disturbed me about O_DIRECT is that the whole interface is just stupid, and was probably designed by a deranged monkey on some serious mind-controlling substances". It gets better afterwards, though I'm not clear whether his articulated vision is implemented yet.

[0] https://yarchive.net/comp/linux/o_direct.html

I ... kind of agree with the difficulty. I don't get it - DPDK is at its core really not a complex API ! Allocate a pool of buffers, and in an infinite loop, ask your NIC to fill these buffers. There. After that, yes you have to decap every packet (ethernet then IP - don't forget reassembly - then whatever you have over - UDP is absolutely no effort, TCP... not so). It's wholly manageable to anyone knowing a bit of light C++ (more C-like) and lower layers (and can parse the sometimes very dry and cryptic doc, for all the utility fonctions. Interaction with the actual consumer of the data can be done with DPDK-provided primitives or simple shared memory... it's really not hard for a mid-level systems programmer. But I still find myself unable to hire people who can work at that level of the stack, a bit baffling. I can't see how they'd be better with io_uring or AF_XDP and all their inherent complexity. Anything harder than a socket and epoll and you're a wizard now...

One other big plus of DPDK for me is the low-level access to hardware offload. GPUDirect (when you can get it to work), StorageDirect or most of the available DMA engines in some (not so) high-end hardware. The flow API on mellanox hardware is the basis of many of my multi-accelerator applications (I wish they supported P4 for packet format instead, or just open-source whatever low-level ISA the controller is running, but I don't buy enough gear to have a voice). Perusing the DPDK documentation can give ideas.

So, yes, very low-level with some batteries included. Good and stable for niche uses. But far smaller hiring pool (is the io_uring-100Gb pool bigger ? I don't know).

Recently did 400gb/s on a single core / 4x100gb nics (or just the one 400g nic, too) with dpdk. Mind you it's with jumbo frames and constant packet size for hundreds of mostly synchronized streams... You won't process each packet individually, mostly put them in queues for later batch-process by other cores. Amazing for data acquisition applications using UDP streams.

I keep watching and trying io_uring and still can't make it work as fast with simple code as consistently for those use cases. AF_XDP gets me partly there but then you're writing ebpf... might as well go full-dpdk.

Maybe it's a skill issue on my part, though. Or just a well-fitting niche.

L40/L40S didn't have it either, which was announced late and felt like a gut-punch for the rare actual non-AI use-case at the time.

A40 nvlink was limited though, one-to-one (bridge), never saw it daisy-chained or nvswitched (might have just not seen them myself, they may have existed).

I'm guessing the temperature could increase quite fast (milliseconds or less) in heavy duty areas, especially when going scalar-to-dense-vector operations.

My best understanding of the avx-512 'power license' debacle on Intel CPUs was that the processor was actually watching the instruction stream and computing heuristics to lower core frequency before reaching avx512 or dense-avx2 instructions. I guessed they knew or worried that even a short large-vector stint would fry stuff...

Apparently voltage and thermal sensor have vastly improved and looking at the crazy swings on NVIDIA GPU's clocks seem to agree with this :-)

I wish I could have been paid to work on SPARK specification around io_uring so that one could have built on it. Or to work on SPARK-to-eBPF (there's already a llvm backend for gnat) and have some form of guarantees at the seams... alas.

Intel Processor Trace was already pretty great. Built a MC-DC coverage tool with it. Used it for fine profiling, live program monitoring...

perf record -e "syscalls:,sched:" is my go-to there - or just apply it on whole-system, aasone does.

Moreover the advantage of Mode S radar (ground-initiated transactions) is that you're using a self-coherent system: the radar knows what it sent and what it is supposed to receive, and when, in a narrow beam, etc.

It makes lots of spoofing attacks unpractical. Where as relying on GNSS exposes to a whole bunch of 'constellation replay' or fake-adsb-telegrams attacks. Mode S lacks authentication, but there's no reason it couldn't be tacked on one day (through e.g. the already available 'advanced' data-link capabilities or the many available 'gicb' registers...). Even light session-authentication would break most of the remaining spoofing attacks (and most swap problems) on ground-initiated Mode S radar.

If you're lucky to be in an area where a Mode S radar is requesting BDS4,4 you can gather more (sparse) live meteorological data: wind speed/direction, temperature, air pressure, humidity... haven't seen BDS 4,4 requested in a while but maybe someone will enable it in your area...

Another fun thing to do is to get mode c (barometric altitude) from aircraft transponders (either SSR or Mode-S downlink 5 or 21 - same physical layer as ADS-B just different bitfield header) and compare it to its GNSS altitude (from ADS-B most of the time) from the same aircraft (the Mode S address used as key) and you can build a map of atmospheric pressures.

Qwant also has an AI overview. Pretty bad too.

The most interesting part of SPARK for me is how all the runtime checks are implicitly, automatically generated and need to be proved.

A whole bunch of assumptions about the language are hidden and the verification conditions are generated automatically for the underlying automatic or semi-automatic proof tools (why3-intermediated) - the second best part of SPARK.

You have to trust that the SPARK FrontEnd makers got it right - or you can inspect/review all the discharged VC if you want - and you still have to actually prove or help prove them all, but I'm not losing sleep over a forgotten 'normal' check.

Now wondering whether the author might be able to force a core dump. With recent snapshot abilities, on Modern Intel processors one can get a Processor Trace that can be helpful even without getting an actual interactive debugging session (haven't done one of those in a while as snapshots seem enough for my needs these days).

Ideally you don't make the cuda cores aware but rather the ray-tracing circuitry. RT cores are designed to perform ray-triangle intersections in a BVH. You get the teraflops and memory bandwidth (or more of it) if you fit the RT-core computing model.

And in most cases it's ok to spend time on one CPU function (creating and loading the BVH) against the hundred thousands of frames you'll be drawing on GPU.

Think more of it as 'tasking by hand' where you have one kernel driving the 18k+ cores and you manually (or using device libraries) fine-grained synchronize them, handle memory traffic asynchoronously and pipeline as much as you can.

You might have a look at cooperative groups, also things cuda::pipeline in libcudacxx to handle asynchronous and pipelined memory traffic, and also most of block/warp CUB primitives, and move on up to cuFFTDx, cuBLASDx and now cuSolverDx as the starting toolbox for your fused kernel journey.

It's not so much kernel overhead than memory traffic between global memory and L2 cache / shared memory that you (or at least I) target with fused kernel approach. Kernel launch overhead can be drastically reduced with cuda-graph indeed.

I'm not sure it applies so well in LLMs though (should read the paper...).

But then you get complaints/problems of revolving doors and how's she supposed to reform an industry that she will have to go back working for again after her tenure as minister or secretary - assuming she wants to eat and feed her family using her hard-earned industry experience.

As much as I wish the world to be different, I've heard so much whining about the Ada language's not-C-likeness that I tend to agree with you.

Wondering how you'd classify Gaudi, tenstorrent-stuff, groq, or lightmatter's photonic thing.

Calling something a GPU tends to make people ask for (good, performant) support for opengl, Vulkan, direct3d... which seem like a huge waste of effort if you want to be an "AI-coprocessor".

A mix plantuml for constrained no-frills diffable standard diagrams and yEd for actual manual placement and boxes-and-arrows drawing - local editing, both can be scripted to export to an image format - is a good local optimum. I haven't found a much better combination for years.

You might be increasing the resources affected to the background thread or its priority. I too try to run as many background loops all the time, to be prompted by insights at the weirdest fun times and banging my head against problems seem to increase the number and insighfulness of these prompts. Frustration, more context, spite... Who knows.

With the level of regression I get from 'security patches' :-) I won't blame the conservative mindset there.

The Air France Rio-Paris crash is a good example of sudden full mistrust of automation and sensors by the crew after a sensor failure appeared and then recovered. Very, very sad transcript and analysis... I'm arguing against myself here, singe it was also a huge case of crew management failure and it might not have gone to crash with only one person in the cockpit.

For those who can stomach it, reading aviation accident reports, listening to actual recorded voice footage, you very often read about the cognitive load of a two-person team trying to get through a shitty moment.

Richard de Crespigny, who flew the Quantas A380 that blew one of its engines after a departure from Changi, explains very clearly and in a gripping way the amount of stuff happening while trying to save an aircraft.

Lots of accidents happen today already at the seams of automation, I don't think we're collectively ready for a world with much more automation, especially in the name of more shareholder value of a 4 dollars discount.

You (and I) prefer to keep the pilots there, but still, there's a push to need only one person and not two in that plane/cockpit. I have little to no doubt we'll have to relearn some hard lessons after we've AI'd up pilots.

In a world telling her to get over her psychosomatic issues which are 'in her head' and that she should be dismissed as incapable of deciding for herself...

Yes this is one of my favorite Ada features (combined with named paramètres at call-site) it always felt clearer reading or writing Ada. The intent ('what') is more important (to me, reading heaps and heaps of code) than the 'how' that I can infer quickly and is less useful in data-flow reading mode.