Comment by superbatfish | Hacker News Reader

Comment by superbatfish | Hacker News Reader

Parent Full thread

I'd like to emphasize the point even further for CPython users. Here's the author's closing sentence:

>Your setup script is not going to run out of file descriptors because you wrote open("README.md").read().

In CPython, files are closed (if necessary) in __del__, so the descriptor is released immediately after that line executes, thanks to CPython's reference-counting mechanism. That idiom wouldn't cause you to run out of descriptors even in a long program.

(The author knows this. Their point is that even if you're NOT using CPython, you STILL shouldn't worry about it too much, and I think I agree.)

I know, I know, CPython's reference-counting behavior is considered an "implementation detail", but for RAM-intensive workloads which are common in Scientific Computing (especially biological imaging), it's often a necessity. So if I were to force myself to use context managers in cases like this, I'd be losing the benefits of terse code without actually preserving portability. (I didn't have portability in the first place.)

In other words, if you're dependent on ref-counting anyway, you may as well admit it and enjoy the benefits that come with it!

It won’t be released immediately, that’s the point. It will be “released at some point in the future and definitely not during interpreter shutdown”.

A “with” statement ensures the release happens deterministically, which it turns out is a desirable trait to have.

The Python standard makes no guarantee, but in CPython, the example above is deterministic. The cycle-detecting garbage collector is not involved in this case. Just refcounting.

If open() throws an exception, perhaps after the FD is already open (Python does non-trivial IO to a file immediately after open), or read() throws an exception, the file object is at the mercy of any try/except higher in the stack, any bugs/"features" contained in those, and any sys.excepthook installed like Sentry

The example above is deterministic only on current versions of CPython and only if it is guaranteed to succeed and if it is the only line of code in the program now and in perpetuity.

> The author knows this. Their point is that even if you're NOT using CPython, you STILL shouldn't worry about it too much, and I think I agree.

The author is completely wrong and so are you.

Used to be you could not install some libraries in pypy because setuptools leaked fds handles, and if your installation was not simple enough it’d run out.

You could not install libraries because some folks couldn’t be arsed to properly close their fds.

Reply on news.ycombinator.com