Hacker News Reader

Comment by bmitc | Hacker News Reader

It's also important to point out ports, because as you mention, NIFs are a way to integrate external code. But as someone else points out, NIFs can crash the entire BEAM VM. Ports are a safer way to integrate external code because they are just another BEAM process that talks to an external program. If that program crashes, then the port process crashes just like any other BEAM process but it won't crash the entire BEAM VM.

View on HN

And then there are port drivers which are the worst of both worlds! Can crash the BEAM and need much more ceremony than NIF to set up but they’re pretty nice to do in Zig[1] as well

[1]: https://github.com/borgoat/er_zig_driver

That's true. Haha!

There's another option and that's setting up an Erlang node in the other language. The Erlang term format is relatively straightforward. But I'm honestly not sure of the benefit of a node versus just using a port.

Node:

- can "easily" send beam terms back and forth

- if you want it to be os-supervised separately (systemd, kubernetes, e.g.)

- pain in the ass

Port:

- easy

- usually the only choice if you're not the software author

- really only communicates via stdio bytestreams

- risk of zombies if... Iirc the stdout is not closed properly?

- kind of crazy how it works, Erlang VM spawns a separate process as a middleman

The Erlang term format is straightforward, but if you want to set up another node in another language you need to correctly implement/emulate process linking, binaries, and some other stuff too, it's not just a matter of writing a socket to accept and emit Erlang terms.

It's not impossibly large but it's not something one does on a lark either; if there isn't support in your language already it's hard to justify this over any of the many, many message busses supported by both Erlang and other languages that don't have so many requirements.

Why would anyone use a NIF instead of a Port then?

IPC/shared memory overhead?

Yeap, this is a big one. In Nx we have some facilities for doing zero-copy stuff that only really work if you have, say, Evision and EXLA running on the same OS process.

We do have IPC handles that could enable this over, say, ports, but then there's a whole other discussion on pointers vs ipc handles

NIFs are great for things that really feel like a relatively quick function call.

If you've got some mathematical/crypto function, chances are you don't want that to go through a command queue to an external port, because that's too much overhead. If it's a many round crypto function like bcrypt or something, you do need to be a bit careful doing it as a NIF because of runtime. But you wouldn't want to put a sha256 through an external program and have to pass all that data to it, etc.

Something that you might actually want queueing for and is likely to have potential for memory unsafety like say transcoding with ffmpeg, would be a good fit as an external Port rather than a NIF or a linked in Port driver.

Ports are generally great, but you are running multiple apps and communicating between them using STDIN/STDOUT etc. There are certain corner cases where they might not be suitable. I had been using an OPCUA library where the logging had to be turned off because otherwise it was sending the logs back to our Elixir app and we were expecting Elixir terms. Also the shutdown of the remote end of a port can stop the data getting back to Elixir. There are ways around all of this but it's slightly annoying. In general though, ports work 80% of the time and are really convenient.

Reply on news.ycombinator.com