Comment by deaps | Hacker News Reader

Comment by deaps | Hacker News Reader

I totally understand that milliseconds matter in the use case described in the article.

For me, personally, I use python to automate tasks - or to quickly parse through loads and loads of data. To me, startup speed is somewhat irrelevant.

I built a micro-framework that is completely unorthodox in nature, but very effective for what I needed - that being a suite of tools available from an 'internet' server, available to me (and my coworkers) over port 80 or 443.

My internet server, which runs python on the backend (and uses apache to actually serve the GET / POST) literally spits out pages in 0.012 seconds. Some of the 'tools' run processes on the system, reach out to other resources, and spit the results out in under 0.03 seconds (much of that being network / internet RTT). To me, that's good enough - adding 30 or even 300 milliseconds to any of that just wouldn't matter.

I totally get that if Python wants to be a big (read bigger?) player then startup time matters more...but for my personal use cases, I'm not concerned with the current startup time one bit.

As expected, language start up time only matters to some people. Often in my case, Python is used to build command line tools (similar to the case of Mercurial).

In such an event, the start-up time of the program might dominate the total run time of the application. And on my laptop or desktop with a fast SSD with good caching and a reasonably fast CPU... that still ends up being 'okay'.

But once I put that on an ARM chip with a mediocre hard drive - some python scripts spend so long initializing that they are practically unusable. Whereas the comparable Perl/BASH script runs almost instantaneously.

Often to make Python even practically usable for such systems I have to implement my own lazily loaded module system. Having some language which allowed me to say...

    import(eager) some_module
    import(lazy) another_module

Which could trigger the import process only when that module becomes necessary (if ever).

Have you tried moving import statements into the functions where they are invoked? My understanding is this is effectively the same as lazy loading the module[1].

[1] https://stackoverflow.com/questions/3095071/in-python-what-h...

I have, and it actually works well (performance wise). The maintenance burden is a little higher.

A little Python preprocessor that lets you annotate your lazy modules sounds like a fun little toy project, actually. Not something I'd use for real, but it would be fun to build.

3.7 makes it easier to use dynamic imports. https://snarky.ca/lazy-importing-in-python-3-7/

I've also written a little asynchronous module loading system. Why not load a module we know we're going to need in the background?

Python is moving to have a lazy loader as part of the standard library. I mean, it's there already, at https://docs.python.org/3/library/importlib.html#importlib.u... , but not clearly easy to use, and with a big warning label against using it.

The issue at https://bugs.python.org/issue32192 says the plan is to start with an easier to use system as a PyPI package.

I think you're telling us about how you're not affected by a problem that does affect other people. I feel like this doesn't add any substantial, interesting points to this discussion.

I have similar use cases. Startup time starts to matter once you either want to build test cases or put scripts in loops. If I have a script that parses one big data file, and I decide to parse 1000, it's often helpful if I can run that script a thousand times rather than refactor it to handle file lists. Or if you want to optimize some parameter.

> To me, startup speed is somewhat irrelevant.

But isn’t that the author’s point? It doesn’t seem like much time but because you’re paying it so often in so many little places it really does add up.

Reply on news.ycombinator.com