Hacker News Reader

Comment by cratermoon | Hacker News Reader

I knew before I got into the meat of the article that it was going to be i/o contention. The first two sections talked about memory and cpu limits on the containers, but nothing about i/o rates. This was a Known Problem back in the 90s when a variety of filesystems (DEC's AdvFS being one) that were efforts to address the issues around dentries and inode. See also http://www.starcomsoftware.com/proj/usenet/doc/c-news.pdf

View on HN

I think they're actually talking about blowing out the dentry cache, but sure IO contention is anoyher shared resource in containers. Depending on what you're doing you might run into issues in various networking limits (somaxconns comes to mind, as does what's left of the route cache), blowing out the page cache, or something eating up your memory bandwidth (maybe via some unlucky NUMA).

All for containers but they don't solve the hard problems folks often ascribe to them, really just shows in most cases you don't need to solve the hard problems. Most of the time what containers are buying you is an easy deployment method that leverages some nice features in the OS to make believe you're on separate machines.

>"Depending on what you're doing you might run into issues in various networking limits (somaxconns comes to mind, as does what's left of the route cache)"

I'm curious what issue(s) you might be referring to here with the route cache? Could you elaborate?

Sure - The route cache was largely removed in (IIRC) 3.8, but there's still entries that get stored[0]. There's a limit to how many entries Linux will store, and like any LRU-esk data structure rapidly cycling entries through it isn't going to do anything wonderful for your performance, never mind if you actually expected to use any of the cached data for a business performance 'feature'.

25g NIC is an awful lot of 60byte packets. I'm not saying this is going to be a common concern, just that like any other shared kernel resource cgroups and namespaces aren't going to help.

0: https://www.systutorials.com/docs/linux/man/8-ip-tcp_metrics...

Sure, that makes sense and thank you for the link. I was curious about your comment:

>"25g NIC is an awful lot of 60byte packets."

Where are you getting that 60 number from? A minimum IPv4 header is 20 bytes and a minimum TCP header is 20 bytes. Also how would a tiny TCP packet relate to the route cache? Tiny TCP packet are certainly a problem with PPS that a NIC is capable I understand that. Cheers.

But it's not, strictly speaking, about i/o contention.

OP here

That's correct, I should have included a chart explicitly measuring the I/O activity done by the two containers, but I can assure you there was literally no I/O activity, a dozen open files per second is a very negligible throughput. The bottleneck was solely in the cache.

Yep, the old version of this was 'why is tar/find/rsync/etc hosing my server even when I've [io]niced it to hell'. Except now (as everything else) with containers!

Reply on news.ycombinator.com