Digging into the source of its dependencies a bit, it looks like it uses NIFs (inlined functions) to bind into your choice of a couple of different potential C backends, the big ones being Torch and Google's XLA compiler.
So performance shouldn't be an issue at all, when compared to similar "high-level language calling low-level language for heavy tasks" situations, like Tensorflow or PyTorch. It does, however, come at the cost of sacrificing the extreme stability BEAM applications typically enjoy: misbehavior by the backend can cause hangs or even crashes of the BEAM node they're running in.