Hacker News Reader

Comment by fny | Hacker News Reader

How is this different from multiprocessing? The examples looks like a complete nightmare...

    interp = interpreters.create()
    interp.run(tw.dedent("""
        import some_lib
        import an_expensive_module
        some_lib.set_up()
        """))
    wait_for_request()
    interp.run(tw.dedent("""
        some_lib.handle_request()
        """))

I'm actually shocked this is even being contemplated. We've regressed to evaling?

View on HN

There's a massive difference to multiprocessing: the different sub-interpreters can use a C(++)/Rust extension module to talk to shared state. In the current multi-processing world, the whole C++/Rust state needs to be duplicated for each process (in the case of our app, this means 5 GB memory usage per core); with subinterpreters, we can share the same C++/Rust state.

The `interpreters` API is just the starting point. Compare it with `subprocess`, not with `multiprocessing`. Once subinterpreters are useful, people will build higher-level APIs for them.

It's just a poc to show indecently running codes, the exec() is not how you will use it, but it simulates importing python code and running it (because an exec() is what import does :)).

It's the minimum needed to get something going, and any really sane use of it would be evalling something like

    import worker
    worker.run()

The PEP explicitly mentions this, and that something like subinterpreter.run(func, ...) could be considered in the future: https://peps.python.org/pep-0554/#interpreter-call

Reply on news.ycombinator.com