Hacker News Reader

Comment by zinekeller | Hacker News Reader

Sidenote: a dispute (https://news.ycombinator.com/item?id=29305298) has been raised on the validity of UPnP even properly using the port. All the details here not contradicting are still correct to the best of my knowledge and research.

Update: "It would appear commplex was a part of seeplex, some visualization tool for old Sun workstations in the early 90ies https://lib.dr.iastate.edu/cgi/viewcontent.cgi?article=11617..."

"The Commplex is a communications package for communication with the NCUBE from Sun workstations."

> * Those ports are the defaults for AirPlay.

No, those are the default port for UPnP discovery, and have been used also by Microsoft in the XP days. There's a reason Apple picked them: it's aligned with their use case, and already assigned by what, 16 years or so?

Probably also the context that you're missing, since that this post is actually somewhat a dupe.

View on HN

UPnP does not own this port. In the past that also hasn't been much of a problem as the port 5000 was opened by UPnP on the Router, not on localhost where Flask or other web servers are normally listening.

Software should be written under the assumption that any given port can already be in use.

I'm not aware of any software that is not written that way.

Well, apparently Flask or other web dev environments fail to start if their default port is already in use...

That's a misunderstanding of sockets and ports. A port alone is not sufficient for anything, to send traffic across an IP network you need a socket, a socket is tuple of a port and an IP address. When a SERVER "binds" a port, what it's doing is opening a socket for listening against the address specified for binding (generally 0.0.0.0 known as INADDR_ANY).

Sockets cannot collide, therefore you cannot bind more than one service running on a host to the same socket, which means if both are using INADDR_ANY, they cannot use the same port. Nothing prevents you from having multiple IPs on a host, each bound by IP to the same port (which would be separate sockets).

For CLIENT applications, they aren't just utilizing sockets, they're utilizing connections, and a connection is a tuple of two sockets. You can have many many many connections utilizing the same port, as long as they don't collide exactly with the socket on the other side in their signature. There are various ways to handle this, but a typical one is that servers do a hand-off during connection initialization to a temporary port assigned to that connection.

So, Flask (a server), is saying you can't bind it to listen on a socket which is previously bound because it would cause a collision and violate the no-collision rule for sockets. But it doesn't in any way mean that port can't be overloaded by different sockets being bound to the same port.

That’s not the point at all.

Thank you for explaining to me how Berkley sockets work (though you forgot about the protocol identifier, you can absolutely have two sockets listening on 127.0.0.1:8000 as long as one is using TCP port 8000 and the other UDP port 8000).

> There are various ways to handle this, but a typical one is that servers do a hand-off during connection initialization to a temporary port assigned to that connection.

I don't think this is right. Servers don't normally use additional ports. Instead, a process can actually create multiple TCP (streaming) sockets with the same local address:local port combination, but different remote address:remote port combinations (each TCP socket is uniquely identified by a four-tuple).

The point though was that once Flask realizes that it can't bind the address:TCP socket, it shouldn't just give up without even telling users about other options.

> I don't think this is right. Servers don't normally use additional ports. Instead, a process can actually create multiple TCP (streaming) sockets with the same local address:local port combination, but different remote address:remote port combinations (each TCP socket is uniquely identified by a four-tuple).

It's not the only way to address this issue, but it is a common pattern for servers to do a TCP handoff to an ephemeral port during connection initialization. One of the reasons that this is done is because it allows you to have /multiple/ connections from the same client to the same server, which is useful for a number of reasons. You're correct that each TCP connection (which is two sockets, a socket is IP + port) four-tuple with a different remote IP or remote port is considered a separate identifiable entity, so in some cases that means the client is responsible for providing an ephemeral port that should be connected back on. It depends on a variety of factors. If neither side does this, then you are limited to a single connection for that protocol for a single client.

If you'd like to investigate some common servers that use connection handoff and ephemeral ports look at nginx and haproxy, both do this and can also suffer from ephemeral port exhaustion on heavily loaded servers because of it (there are mitigation strategies as well).

I fail to see what other behaviour they could have.

Picking a random port when you have been told to pick a specific port (even implicitly) makes no sense. What if I start flask telling it to use 8000 and it’s already taken by an other program I’m running (something I do semi-regularly as I run tests concurrently), just pick a random port unasked and notify no one about it?

Given how much confusion the current behavior seems to have caused, the best course of action seems to have been a simple:

"Listening on port $PORT. Warning: failed to bind localhost:5000. Start process with --port=$X to specify a different port"

That would be much more helpful behavior and messaging than simply failing, especially since it seems people don't know that they can control the port, and start looking to disabling OS services rather than changing a port...

It might be something to consider, but there are definitely some negative consequences to silently starting on another port, particularly when using automation and tests. It's not fun to have flask silently start on another port and then an integration test hits a completely unrelated service that happens to occupy that port.

Totally agree, but if you're running tests, presumably you are looking for errors in the first place, right? So it doesn't have to fail silently, does it? The testing tools failing for whatever reason seems important to know as well.

> notify no one about it

But why this part? Python -m http.server tells me which port it is using or gives me "OSError: [Errno 48] Address already in use".

In fact, I just tried

python -m http.server $((10000 + RANDOM % 10000))

and it told me what port it started using.

If the suggestion is that Flask should use a random other available port: that could be done, but generally people usually prefer to manually pick a different port themselves. Having a dev server come up on different ports all the time is not a great user experience.

Are you being intentionally obtuse for some reason?

We wouldn't be having this conversation if the web tools were written that way. If the web tool is already running, Airplay will move to another port. The reverse is apparently not true.

Most software has zero need for this.

My company's product contains a web server. We don't bother to check to see if some other program is using port 80 or 443, we just use it. The reason is that we own the machine, we install the software, and we know there is nothing running on these ports; and more, if something is on these ports, we have no way to recover, so terminating abnormally is the right answer.

Neither does Flask and similar, and they are the ones who arbitrarily picked an already-registered port and didn’t bother registering their own.

The port registration system effectively broke down somewhere in the 90ies. There are just too many software projects that need ports in a very limited space. The software we're discussing here (AirPlay apparently) also did not register this port.

Reply on news.ycombinator.com