Hacker News Reader

jared_hulberton io_uring is faster than mmap

Just ran a version with 6 prefetching threads. I get 5.81GB/s. Same as the io_uring with 2 drives, but still a lot slower than the in memory solution.

jared_hulberton io_uring is faster than mmap

Someone else suggested this, results are even worse by 2.5s.

jared_hulberton io_uring is faster than mmap

When I run the 50GB in-mem setup I still have 40GB+ of free memory, I drop the page cache before I run "sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'" there wouldn't really be anything to evict from page cache and swap isn't changing.

I think I'm crossing the numa boundary which means some percentage of the accesses are higher latency.

jared_hulberton io_uring is faster than mmap

The io_uring solution avoids this whole effort of mapping. It doesn't have to map the already-in-RAM pages at all. It reuses a small set of buffers. So there is a lot of random cache-miss prone work that mmap() has to do that the io_uring solution avoids. If mmap() does this in the background it would cache up with io_uring. I'd then have to get a couple more drives to get io_uring to catch up. With enough drives I'd bet they'd be closer than you think. I still think I could get the io_uring to be faster than the mmap() even if the count never faulted, mostly because the io_uring has a smaller TLB footprint and can fit in L3 cache. But it'd be tough.

jared_hulberton io_uring is faster than mmap

I get that. But I do actually show a scenario where accessing data from memory using a very standard mechanism IS slower than a newer but equally standard way of accessing data from an NVMe drive.

"Accessing memory is slower in some circumstances than direct disk access"

jared_hulberton io_uring is faster than mmap

Adding MAP_HUGE_1GB and not MAP_HUGETLB does compile and run for me. Not convinced that its' actually doing anything. Performance is the same.

jared_hulberton io_uring is faster than mmap

Not by a ton but if you add up the DDR5 channel bandwidth and the PCIe lanes most systems the PCIe bandwidth is higher. Yes. HBM and L3 cache will be higher than the PCIe.

jared_hulberton io_uring is faster than mmap

Hmm. tmpfs was slower. hugetlbfs wasn't working for me.

jared_hulberton io_uring is faster than mmap

spdk and I go way back. I'm confident it'd be about the same, possibly ~200-300MB/s more, I was pretty close to the rated throughput of the drives. Io_uring has really closed the gap that used to exist between the in kernel and userspace solutions.

With the Intel connection they might have explicit support for DDIO. Good idea.

jared_hulberton io_uring is faster than mmap

YES! gcc and clang don't like to optimize this. But they do if you hardcode the size_bytes to an aligned value. It kind of makes sense, what if a user passes size_bytes as 3? With enough effort the compilers could handle this, but it's a lot to ask.

I just ran MAP_POPULATE the results are interesting.

It speeds up the counting loop. Same speed or higher as the my read() to a malloced buffer tests.

HOWEVER... It takes a longer time overall to do the population of the buffer. The end result is it's 2.5 seconds slower to run the full test when compared to the original. I did not guess that one correctly.

time ./count_10_unrolled ./mnt/datafile.bin 53687091200 unrolled loop found 167802249 10s processed at 5.39 GB/s ./count_10_unrolled ./mnt/datafile.bin 53687091200 5.58s user 6.39s system 99% cpu 11.972 total time ./count_10_populate ./mnt/datafile.bin 53687091200 unrolled loop found 167802249 10s processed at 8.99 GB/s ./count_10_populate ./mnt/datafile.bin 53687091200 5.56s user 8.99s system 99% cpu 14.551 total

jared_hulberton io_uring is faster than mmap

I worked on SSDs for years. Too many people are suffering from insufficiently solid values of "disk" IMHO.

jared_hulberton io_uring is faster than mmap

int fd = open(filename, O_RDONLY); void* buffer = mmap(NULL, size_bytes, PROT_READ, (MAP_HUGETLB | MAP_HUGE_1GB), fd, 0);

This doesn't work with a file on my ext4 volume. What am I missing?

jared_hulberton io_uring is faster than mmap

The original blog post title is intentionally clickbaity. You know, to bait people into clicking. Also I do want to challenge people to really think here.

Seeing if the cached file data can be accessed quickly is the point of the experiment. I can't get mmap() to open a file with huge pages.

void* buffer = mmap(NULL, size_bytes, PROT_READ, (MAP_HUGETLB | MAP_HUGE_1GB), fd, 0); doesn't work.

You can can see my code here https://github.com/bitflux-ai/blog_notes. Any ideas?

jared_hulberton io_uring is faster than mmap

https://www.intel.com/content/www/us/en/io/data-direct-i-o-t...

AMD has something similar.

The PCIe bus and memory bus both originate from the processor or IO die of the "CPU" when you use an NVMe drive you are really just sending it a bunch of structured DMA requests. Normally you are telling the drive to DMA to an address that maps to the memory, so you can direct it cache and bypass sending it out on the DRAM bus.

In theory... the specifics of what is supported exactly? I can't vouch for that.

jared_hulberton io_uring is faster than mmap

I tried with the log scale before. They failed to express the exponential hockey stick growth unless you really spend the time with the charts and know what log scale is. I'll work on incorporating log scale due to popular demand. They do show the progress has been nice and exponential over time.

When I put the lines on the same chart it made the y axis impossible to understand. The units are so different. Maybe I'll revisit that.

Yeah around 2000-2010 the doubling is noticeable. Interestingly it's also when alot of factors started to stagnate.

jared_hulberton io_uring is faster than mmap

Would this actually create huge page page cache entries?

jared_hulberton io_uring is faster than mmap

Lol. Thanks.

jared_hulberton io_uring is faster than mmap

Oh man... I'd have look into that. Off the top of my head I don't know how you'd make that happen. Way back when I'd have said no. Now with all the folio updates to the Linux kernel memory handling I'm not sure. I think you'd have to take care to make sure the data gets into to page cache as huge pages. If not then when you tried to madvise() or whatever the buffer to use huge pages it would likely just ignore you. In theory it could aggregate the small pages into huge pages but that would be more latency bound work and it's not clear how that impacts the page cache.

But the arm64 systems with 16K or 64K native pages would have fewer faults.

jared_hulberton io_uring is faster than mmap

Cool. Original author here. AMA.

jared_hulberton Memory is slow, Disk is fast – Part 1

In this context CXL is kinda storage and kinda higher latency RAM. It's latency is worse than standard DDR5 DIMMs but the bandwidth is on the same trajectory as storage/networking.

I'm inclined to think of it like storage in this context. It's scaling, but it will require new thinking to take full advantage of.

jared_hulberton Memory is slow, Disk is fast – Part 1

We are seeing more SRAM in the form of CPU caches, this is one of the things that is actually scaling still.

Are you asking why not use SRAM in something like a DIMM? You could do this. Here's why I wouldn't advocate for this. Assume you had zero latency SRAM in your DIMM. It still takes ~40ns to get out of the processor by the time you go through all the memory controller and phy. So you'd have an incredibly expensive but small DIMM taking up limited pins on the processor package/die. Even then you'd only cut the memory latency in half, and we'd still be stuck at a new lower flatline.

Incorporating the SRAM in die is different story, you get to scale the latency and the bandwidth closer to the other capabilities of the cores.

jared_hulberton Memory is slow, Disk is fast – Part 1

Original author AMA

jared_hulberton Memory is slow, Disk is fast – Part 1

https://www.bitflux.ai/blog/memory-is-slow-part2/

jared_hulberton Show HN: Agents for DevOps? Demo

I found https://aokumo.ai/

Who else?

jared_hulberton Rest in Peace, Optane

memverge.com does some cool work around making that happen.

jared_hulberton Rest in Peace, Optane

Silly hack of a product. x2 PCIe for Optane, 2x for NAND. Meant compromised bandwidth for all! The fact that it actually had value in some cases highlights how cool it could have been if it was a proper hybrid design.