Hacker News Reader

Comment by 0cf8612b2e1e | Hacker News Reader

That does happen, but not frequently in the common libraries that are going to be in public training data.

Is there a top 100 package that does something funny on import?

I'd be surprised. That kind of thing was en vogue for a little while in the early 2000s before cooler heads prevailed, but now people will understandably shout at you for changing behavior in someone else's code.

My guess is that nearly all packages that did this sort of thing were left behind in the 2-to-3 migration, which a lot of us used as the excuse for a clean break.

Not sure if that counts, but if you import both matplotlib and OpenCV at once, there is a good chance of a crash due to conflicting PyQt binaries: https://github.com/matplotlib/matplotlib/issues/29139

But I agree that observable side effects are generally pretty rare. And apparently, both libraries are not even in the top 100 packages, depending on how you count. It looks like those spots are all taken by libraries used in uncached, wasteful CI workflows: https://hugovk.github.io/top-pypi-packages/

Oof. That doesn't count in my opinion. The conflict is unfortunate, but it's not because either package is trying to modify other code. That is, the error is a side effect of how loading multiple libs into the interpret works. In theory, at least, you could fix those bugs without modifying the packages' behavior at all.

But still a bummer, to be sure. It's easy enough for me to say it doesn't count when I haven't been affected by it.

Well “import torch” for example will resolve certain dynamically linked symbols, which must be done first before importing your own .so code that uses libtorch and pybind11. If not you will get a super fun to debug segfault, leaving you staring at gdb backtrace output while you ponder your career choice.

This is buried deep in the PyTorch docs and I don’t have the willpower to go find it right now, sorry.

Heh. A ML library was my sneaking suspicion of where there might be something unexpected. Anything goes for performance and/or to get Nvidia to cooperate.

Dunno if it counts as funny, but the following code only works if you keep the matplotlib import:

  import matplotlib  
  import PIL  
  im = PIL.Image.open("foo.png")

What? There’s no way that’s correct. I use PIL exactly like that and don’t have matplotlib in my codebase.

Just try it. IIRC, to do the PIL import correctly you have to

  import PIL.Image

Turns out that matplotlib (and probably lots of other stuff) does that, and then it gets resolved correctly.

Oh right, gotcha. I always do from PIL import Image

Reply on news.ycombinator.com