Hacker News Reader

Comment by andy_ppp | Hacker News Reader

There is a huge problem with this sadly, the startup time of the BEAM is too much! I was thinking about having a running VM process and injecting code directly, but I think this could be quite insecure. Love the idea though.

View on HN

Totally understandable concern, in my quick experiments I haven't been bothered too much by the bootup time. But there's definitely better performance aware solutions for scripting

What is the startup time compared to bash?

I haven't test, but IIRC, Jose Valim in an interview commented that the BEAM startup time is about 300 to 500 milliseconds

Powershell suffers from this same problem, and I find it very annoying.

Really recommend you speed this up if it's possible.

You could spawn independent actors for each invocation. That provide a degree of separation and be fine for secuirty, if the beam process was run as your user pid. Handling multiple stdin/stdouts would take some work, but doable with standard erlang libraries.

ok on my pentium G3470 running FreeBSD 12.2:

time /bin/sh -c "echo hi" hi

    real    0m0.003s
    user    0m0.000s
    sys     0m0.003s

time /usr/local/bin/bash -c "echo hi" hi

    real    0m0.006s
    user    0m0.000s
    sys     0m0.006s

time /usr/local/lib/erlang23/bin/erl -eval 'io:format("hi~n"), exit(0)'

Erlang/OTP 23 [erts-11.1] [source] [64-bit] [smp:2:2] [ds:2:2:10] [async-threads:1] [dtrace]

    hi
    {"init terminating in do_boot",0}
    init terminating in do_boot (0)

    Crash dump is being written to: erl_crash.dump...done

    real    0m0.304s
    user    0m0.117s
    sys     0m0.010s

Ok, so 300 ms is clearly too much, but that's running a bunch of stuff.

Turning a bunch of stuff off, I get

time /usr/local/lib/erlang23/bin/erl -eval 'io:format("hi~n"), halt(0, [{flush, false}])' -noshell -start_epmd false -boot no_dot_erlang hi

    real    0m0.077s
    user    0m0.094s
    sys     0m0.008s

(I tried tuning down the number of schedulers, but couldn't get the real time lower)

This is still a lot worse than bash, but about 75 ms is sort of okish. Maybe a non-stock boot file could help. I usually plan for my BEAMs to run for months or years, so I don't worry too much about startup time :)

Nice that you’ve managed to speed it up quite a lot! Still it’s fairly slow, that’s why I was thinking about streaming stuff to a running BEAM from a Golang executable, say. Presumably then this problem goes away (compilation may still take a while). I guess if the #! just streams the file content over we are still protected by the execution bit, but what about malicious programs streaming their own content?

You could do something like that, but it adds a lot of complexity. I suspect if I can get the time down to 75 ms with a frw minutes of fiddling, someone who actually cares to run this could spend a couple hours or days on it and get it down to 30ms, which should be tollerable. Reduce as much runtime startup as possible (probably allocator tuning), and then reduce or postpone as much OTP startup as possible.

Otherwise you have to deal with things like how do you kill one script without killing the others, version fun including what happens if you are running scripts and update the beam, how to keep scripts separated for reals, etc.

But would that be more insecure than running arbitrary scripts on your machine? I imagine you could build something akin to Ruby's spring for elixir, might be interesting.

Reply on news.ycombinator.com