I also tend to believe Docker makes little sense, where OTP releases are the alternative.
What's the opinion here? This question only pertains to OTP development.
You can use Docker to provide a common/consistent base on which you'd build the nodes running your OTP applications. If you have zero non-OTP dependencies, then this is overkill (you'd be better off deploying directly to some server, be it a VM or bare-metal, and managing configuration/deployment on that level), but most real-world applications do have other Dockerizable dependencies (databases come to mind, though there's ongoing debate on whether or not putting Postgres in a Docker container is actually smart). Also, using Docker to abstract the runtime environment for BEAM does help considerably with deployment to various PaaS platforms (e.g. Heroku, Bluemix, Elastic Beanstalk, etc.).
Personally, I lean toward a couple big servers dedicated to BEAM instances (running all the OTP apps), and a couple big servers dedicated to some SQL database (usually Postgres). Whether you maintain those servers as Docker containers or EC2 instances or DO droplets or physical servers is an implementation detail that doesn't really matter that much as long as it works and it's easy for you (or your DevOps / sysadmin teams) to maintain.
On that note: I don't think it makes sense to have a bunch of little containerized microservices each running a separate BEAM and a separate set of OTP applications. OTP already takes care of that sort of orchestration, and OTP applications already are (or can be) microservices. Just throw 'em all into a big server and let BEAM's scheduler and OTP's supervision model do their jobs.
That said, they still do not do similar things. OTP does not take care of orchestration if you must run more than 1 node.
You are still benefiting from OTP's supervision and scheduling when using docker or k8s, the difference is in what other features and the size you need.
No one is saying always deploy to k8s, but they are not overlapping.
Nor does using BEAM make it uniquely fitted to running on physical machines compared to other languages like java, go, C++, Rust, ... If your application is suited to running on a few droplets the language you are using is not likely the deciding factor for that.
> If you don't and you work in even a medium size company you likely work with what they have, which is increasingly kubernetes.
And that's totally fine, per my second and third paragraphs.
> OTP does not take care of orchestration if you must run more than 1 node.
No, but it (with BEAM) gets you most of the way there. The only thing not explicitly handled is spinning up the machine running that node (though you can certainly have the app itself drive that, e.g. by using the AWS API to spin up another EC2 instance with an AMI preconfigured to boot up and start ERTS). Once the machine is running, the application can use that node as a spawn() target and start throwing processes at it.
> Nor does using BEAM make it uniquely fitted to running on physical machines compared to other languages like java, go, C++, Rust
BEAM (or at least some sort of Erlang VM) is (in conjunction with an application framework that can capitalize on it, like OTP) is exactly what pushes the needle toward running on a couple big machines instead of a bunch of little machines (and again, whether those big machines are containers or VMs or physical servers is an implementation detail). Java and Go and C++ and Rust don't have Erlang's/BEAM's process model; they're dependent on the OS to provide concurrency and isolation (though there's no particular reason why a JVM implementation couldn't support an Erlang-like process model, on that note; similar deal with a .NET CLR implementation, or any other VM).
The compelling reason to go with a bunch of small machines instead of a couple big machines is resource isolation (e.g. enforcing memory/disk/CPU quotas for specific applications). It's definitely possible to do this on an OS process level (i.e. by applying those quotas to the BEAM processes), but if you're already using something like Kubernetes, no reason to not use that particular hammer to smack that particular nail.
Jose Valim, creator of Elixir, also wrote about this recently http://blog.plataformatec.com.br/2019/10/kubernetes-and-the-...?
https://www.scylladb.com/2018/08/09/cost-containerization-sc...
Erlang does have the option to bind schedulers to processors but I don't know of any real world usage of it:
Binding Schedulers: http://erlang.org/doc/man/erl.html#+sbt
Custom CPU Topologies: http://erlang.org/doc/man/erl.html#+sct
You can run BEAM apps in a container the same way you’d run any other app. It’s possible. But BEAM is less like a PHP/Python/Ruby etc interpreter and more like an abstraction for the OS (some might even consider it to be one). When you’re in Erlang you think in Erlang and the runtime abstracts things like processes, threads, and network partitions away... so you tend to want a big beast BEAM instance (or a few big beasts) rather than a lot of tiny ones (as one might scale up a horizontally scaled app like one on Heroku dynos).
It can be done. It’s just kinda going against the grain.
They are great to have when you need them but should not be chosen lightly.
Part of the point of this booksite is to show integrating Erlang into a modern environment and not being the oddball deploy at the company. It is the lack of libraries for services used for deployment/monitoring and lack of documentation that is the reason for this.
Also, there is no abstracting away network partitions.
Docker makes a lot of sense, actually, because it allows you to run integration tests easily and ship Erlang releases in the same way as everything else. There are exceptions, of course. As for OTP releases, they are a good way intermediate step before packaging code to RPMs, but upgrades should be done the usual way: via blue-green deployment. Don't even argue with me :-) I happened to support a huge system that uses OTP upgrades, and the amount of complexity and tricky bugs, that live code loading introduces, is immense. No one is smart enough to do it :-)