What is the theoretical maximum number of open TCP connections? (2010) | Hacker News Reader

What is the theoretical maximum number of open TCP connections? (2010)

stackoverflow.com

46 pointsgeorgecmu20 comments

Open articleView on HN

"The Secret To 10 Million Concurrent Connections -The Kernel Is The Problem, Not The Solution":

http://highscalability.com/blog/2013/5/13/the-secret-to-10-m...

https://news.ycombinator.com/item?id=5711232

> http://highscalability.com/blog/2013/5/13/the-secret-to-10-m...

Really interesting read. Some of the notions espoused in this link really seem to make a case for microkernels.

This guy has some nice posts of scaling a single node.js server up to 1m connections:

http://blog.caustik.com/2012/08/19/node-js-w1m-concurrent-co...

http://blog.caustik.com/2012/04/10/node-js-w250k-concurrent-...

http://blog.caustik.com/2012/04/08/scaling-node-js-to-100k-c...

http://blog.caustik.com/2012/04/06/node-js-scalability-testi...

That's odd - a question I asked in 2010 has now made it to the front page.

The Internet is forever.

But you already got the "Famous Question" badge for it a year ago: http://stackoverflow.com/help/badges/28/famous-question?user...

The URL indicates that the user posted it to get a stack overflow badge just like you did.

I don't think you can fault it for him if you did the same:)

On the ipv4 Internet? 2^96. (32 bit src and dest addrs, 16 bit src and dest ports)

Can't you have multiple connections to the same host?

Yes, but only to separate ports. A connection in TCP is uniquely identified by a tuple of {source IP, source port, destination IP, destination port}. Usually the source port is automatically assigned by the source system, and it picks an unused one.

That would be encoded in the multiple source ports, and multiple source IP addresses if your box has them.

Yeah, that's why I included port numbers.

The actual question title limits it to a "modern Linux box", so they aren't really asking for the "theoretical maximum".

http://www.kegel.com/c10k.html

Also relevant: "1 million is so 2011" from WhatsApp's blog

http://blog.whatsapp.com/index.php/2012/01/1-million-is-so-2...

Just be sure to turn off iptables because that doesn't support more than ~64K connections.

Could you post a link to that info? I hadn't come across that limitation before.

I'm looking for a link, but in the meantime, I was working on a product where we had to map every connection between 2 routers to a new connection between 2 boxes (imagine having 2 boxes MITMing 2 routers and remapping every TCP session internally so you could change the traffic transparently to the recipient).

We needed to prove we could handle 100k TCP sessions and it would fail at ~64k sessions when iptables was running on either box (even with an "allow all" rule). Otherwise our hardware would fall over around the default File Descriptor limit.

You should have a look to see how much memory the connection tracking table is actually taking up. IPTables stores a lot of its information in kernel space, but modifications are copied to user space, updated, then written back. As an example, for large project X with >100,000 users connecting through a linux-based gateway device, using a single firewall rule to allow access for each device grew larger than RAM available to the kernel. You can also tune the size of the connection tracking table (and pretty much everything else related), but 64k sessions was never a breaking point for us.

You probably had the maximum number of connections in ip_conntrack_max set to ~64K

Reply on news.ycombinator.com