Hacker News Reader

Comment by ryan-duve | Hacker News Reader

When you say "multiplex a terminal" with tmux, do you mean splitting screens so the same terminal window has multiple shells prompts in it? I'm trying to understand how that would be used to address the problem in the demo at the bottom of the post.

View on HN

I will try to be more concrete than I was (though I did link to some Nim/shell that "does it all" the 2nd way I mentioned).

j.py:

    import multiprocessing as mp, contextlib as cl, time
    def f(x):
        name = mp.current_process()._name
        with open("o." + name, 'a+') as f:
            with cl.redirect_stdout(f):
                print(x * x)
                time.sleep(2)
    mp.Pool().map(f, range(36))

Then

    shell$ python3 j.py & sleep 1; multitail o.*

On Unix that should create $(nproc)-panes (i.e. screen areas) of output in your terminal, each pane logging any output each worker process did. (EDIT: For fancier progress, multitail -cT may help; See the man page.)

Some Py MP expert could perhaps show how to do this with only one `open` or a nicer `name`. As a bonus - the log/status outputs are not erased but left in "o.Stuff". The cost of that is having log files to clean up.

Separate files means no contention at all and any stray prints from any libraries go into the log file. It's a separate exercise for the reader to extend this to stderr capture (either in e.* or also in o.*).

And, really, as popular as Python and MP are, all of this is probably written up somewhere.

Someone else should outline imitating `multitail o.*` in tmux, I think, as it is more involved, but a pointer to get started is here: https://github.com/tmux/tmux/wiki/Advanced-Use (That's all I have time for today. Sorry.)

Reply on news.ycombinator.com