Comment by miohtama | Hacker News Reader

Comment by miohtama | Hacker News Reader

Parent Full thread

I was working on the mobile web server project. The project was heavily subcontracted to a company in Oulu, Finland, where I worked by the time before starting a career in open source and Python consultancy. Mobile web server was a Series 60 web server, based on Apache, some glue and reverse proxy gateway (written in Java). You could deploy your own “apps” that could be accessed by anyone, running on your mobile phone.

Unfortunely, the project did not have a real business use case. People believed that peer-to-peer is a thing (XMPP, BitTorrent, IRC) and this would drive mobile-to-mobile interactions. Unfortunately, it turned out not to be true.

Later me and my friend ported Python to Series 60 with our own rewrite, as Nokia’s official port had many issues. Unfortunately by this time Series 60 was already losing market share for iPhone and Android. Furthermore we found that because of module import delay, Python will never be a good run-time for mobile applications, unless you do some sort of a binary dump/prewarmed up interpreter.

Binary dump / prewarmed up Python interpreter would be amazing even today. Could enable a lot use cases with Python.

youtube-dl/yt-dlp (Python) is a full second slower than youtubedr (Go) or youtube.com (JS) to download YouTube videos, and I think a lot of that is due to interpreter warmup (though I don't know exactly why). Sadly mpv only supports youtube-dl/yt-dlp for watching videos, and youtubedr doesn't have the vast plugin ecosystem of yt-dlp for non-YouTube websites.

The above mentioned Python’s import delay. This is a notorious behavior in high performance situation (including your cli startup time example). Essentially the import process is IO intensive. This comes up in massive parallel applications as at launch each process is importing the same stuffs. It is briefly mentioned for example in https://docs.nersc.gov/development/languages/python/

If you really care about the startup time, you could pre-compile it. Nuitka and Cython could do this with different limitations.

I've run into the import issue as well, it's especially painful if something imported is doing work using a slow library (e.g. a config module loading a HOCON config using pyhocon a.k.a slowasmolasseshocon)

That will be another slow down happening at import stage. But even if ideally everything you import only does import with not much work, the import overhead mentioned above still exists. But frankly I’m not sure how significant it is on a typical personal computer with SSD, especially when the OS may cache recently used files already.

The problem you mention can be significant because some design patterns in Python encourages this. Eg function arg with a default that’s expensive. Unfortunately this whole speeds up repeatedly calling the func, the cost is paid at first import.

Unless your 1) execution environment (CPU, dlls) or 2) Python source code files change, you can use “unexec” strategies like Emacs

https://news.ycombinator.com/item?id=21394916

Reply on news.ycombinator.com