> Wangyang and Arjan reported a bottleneck in the networking code related to struct dst_entry::__refcnt. Performance tanks massively when concurrency on a dst_entry increases.
> This happens when there are a large amount of connections to or from the same IP address. The memtier benchmark when run on the same host as memcached amplifies this massively. But even over real network connections this issue can be observed at an obviously smaller scale (due to the network bandwith limitations in my setup, i.e. 1Gb).
Basically, processes that produce a lot of threads (i.e.: 100+) suffer performance degradation. This impacted network performance by up to 5% on 1 Gb/s connections and showed up to 3.2x performance degradation in synthetic benchmarks.
Independent testing verifies these improvements in the Memcached benchmark. Simulating a more realistic workload using Cockroach DB shows a more modest performance improvement of ~5% in terms of ops/s.