OTP is a programming library/paradigm that doesn't work without the ability to create extremely lightweight threads, which is what the BEAM provides.
I agree that Elixir without OTP becomes much less useful, but there could be some changes to the language to enable in-process state changes so that you wouldn't be limited to a single process and the immutability restrictions that would make that very difficult to do anything useful with. I'm sure they thought of this problem and have a solution in some form.
Lots of people have written alternatives to BEAM. The only problem they run into is that BEAM is very good, and would be tough to beat. I was an admirer of Erlang on Xen: https://github.com/cloudozer/ling