Hacker News Reader

Comment by douglasisshiny | Hacker News Reader

I'm more interested in a language like gleam, but the BEAM VM does sound appealing. But one thing I have been wondering is what is the hosting story like for BEAM languages?

Is there easy tooling for deploying to something like AWS ECS and having nodes communicate with each other?

View on HN

You can host Elixir apps on PaaS like Gigalixir[0] or Fly.io[1] (no associated with these).

I personally use mix release[2] that assembles a tarball of itself (together with BEAM), then rsync that to Hetzner, restart the remote process and viola. I like simplicity.

Re cluster of nodes, the easiest would be to use a library[3] to automate the formation of said cluster.

[0] https://www.gigalixir.com/

[1] https://fly.io/docs/elixir/getting-started/

[2] https://elixir-lang.org/getting-started/mix-otp/config-and-r...

[3] https://github.com/bitwalker/libcluster

A caveat with libcluster + k8s though - it doesn't play nicely with sudden burst of traffic.

To be fair, there are problems with other languages/runtimes and bursty traffic as well - autoscaling isn't great when it takes 20 minutes to provision a new server to run your .Net app.

Could you expand on that or link to something that does? I was considering using libcluster on k8s, so curious about the limitations you're referring to.

https://news.ycombinator.com/item?id=37488627 :)

Can you talk about this more? :) Why is that the case?

I'll give you a bit of background to explain it better.

Our service is primarily a graphql API. The mobile apps talk to our service, and then it internally talks to other services. We use Absinthe for graphql, and a lot of it is subscription based.

Since these nodes (pods in k8s terms) are clustered, each pod is aware of the other pod. Sometimes we get a huge spike in traffic and the clusters can't start fast enough (we're increasing the scale up time in k8s) and a pod can get OOM killed due to number of graphql subscription and then just go in crashloop. This leads to libcluster thinking the node is still good but when infact it's in crashloop, and thus not allowing any new pods coming up to start up.

We're experimenting with few things, but yeah clustering is not without it's pain

thanks for such a concise answer!

how do you do deploy / bundle your configs?

I compile the tarball using Dockerfile, as the target OS must be the same as source OS to compile it (I deploy Debians on remote, and use Fedora locally). Then there's bunch of mix tasks to bundle the js/css assets - it's all really there ready to be used. The other day I spent half a day implementing same behaviour for Django app.

> Is there easy tooling for deploying to something like AWS ECS and having nodes communicate with each other?

libcluster[0] has a bunch of strategies to form clusters. It seems that ECS supports service discovery through DNS, so the DNSPoll[1] strategy should work.

[0] https://github.com/bitwalker/libcluster [1] https://hexdocs.pm/libcluster/Cluster.Strategy.DNSPoll.html

The answer for any language is just to stick it in a container. Elixir fits inside a container, and the Phoenix generator creates a Dockerfile for you out of the box.

As an aside, Elixir is doing some interesting, novel stuff exploring its own type system. José Valim's keynote at ElixirConf last week went into detail on the topic, so I'd keep an eye out for it on YouTube in the coming weeks if that sort of thing interests you.

I couldn’t imagine working on a large scale project without strong types.

What happens when you have to refactor a large elixir feature on a code base which many teams touch? In such cases you can’t test manually all code paths for runtime errors, so are you just relying (hoping for) perfect test coverage to catch any type related errors you may have caused?

Unless you've worked with Clojure or another immutable, dynamically-typed language with pattern matching, you probably don't have a good mental model for what this would look like. It's a completely different story from a language like Python or JavaScript.

Immutability and pattern-matching get you at least the 80-20 of what a rigid type system does for larger code bases, and your team won't have to be as large when you're using a language like Elixir or Clojure as it would with older, more entrenched languages.

I feel like a common refactor in a large project might be a complete redefinition of a model layer exposed via a bunch of structs.

How does immutability or patter matching save me from runtime errors caused by changing the fields or types inside a struct? Wouldn’t the pattern matching just fail and go down the wrong code path since it was matching for the old pattern?

I feel like immutability is great, pattern matching is great, but static typing is also great and there is no reason why it can’t happily live along side the other two.

In this case, the question is moot.

Elixir structs have compile-time checks: https://elixir-lang.org/getting-started/structs.html

It's not that hard, also Elixir is basically gradually typed with dialyzer. It's not like working on a dynamic and weakly typed lang like vanilla JavaScript or Perl at all.

I do want to clear something up. Elixir is strongly typed, but its not statically typed.

More here: https://www.educative.io/answers/statically-v-dynamically-v-...

Even for langs that are statically/ and strongly typed on the BEAM (like Gleam, whose type system is similar to that of Standard ML or OCaml) still subscribe to the "Let is Crash" philosophy, especially when it comes to messages sent and received between processes. The only thing that is guaranteed is your system will fail at some point, how should the system protect its self form that?

https://www.educative.io/courses/concurrent-data-processing-...

In short, its not like working with Ruby or JavaScript or Python, etc..

The “let it crash” philosophy is great, but at the same time a proper static type system protects you from a whole class of crashes before they ever reach deployment.

If you change the shape of a model object and break its type, it’s great that BEAM will gracefully handle that mistake, but no matter how many times that service restarts, it will still be broken at runtime.

I think type systems have gotten so good in modern languages (Kotlin for example) that it’s a disservice to your org to not use a statically typed language.

I haven’t looked at Dialyzer lately, but is it as robust and easy to use as a first class type system? Having experienced partial typing with annotations with python I must say it’s not nearly as smooth as a proper statically typed language.

I do wish something like Gleam was the front runner language on BEAM - I bet it would get taken a lot more readily.

Dialyzer is good enough for my use-cases, especially in large web apps. In fact, Elixir's dynamic nature, when combined with Dialyzer and the BEAM's fault-tolerance features, offers a reasonable middle ground that has proven robust in large-scale systems.

Comparing it to Python's gradual typing isn't entirely fair for a couple of reasons:

1) Elixir is functional, meaning a value's type never changes. This makes it easier to reason about the code and allows tools like Dialyzer to be more effective.

2) The combination of value/pattern matching and the "let it crash" philosophy is unique to the BEAM ecosystem and has proven invaluable for building complex applications. this out side of Dialyzer gets you 80% the way there. (I don't even use Dialyzer in my applications because I don't think it buys you that much. I lean on testing, typically to cover that last 20%)

I'm intrigued, however by the new type system that's being explored for Elixir by Jose and this team. It promises the best of both worlds: dynamic typing with the assurances of a static one.

If we were to impose a Java/Kotlin-like type system onto Erlang/Elixir, there would be several concerns:

    It could encourage harmful coding practices that hinder code evolution.
    It might conflict with the "let it crash" philosophy, leading to more "nanny coding."
    The effort required to satisfy the type system could outweigh its benefits.
    It could reduce productivity and readability.
    If not integrated into the BEAM core, it might not offer optimization benefits and could create compatibility issues.
    It could divide the community and distract from other important work.

Elixir's existing features like testing, value matching, and the "let it crash" philosophy already provide a robust safety net that a static type system might not significantly improve upon.

A common pattern that I use is to pass a behaviour module as a function argument to support dynamic behavior or dependency injection. Last I looked, I couldn't use dialyzer to describe the behaviour that a function accepts.

Maybe something like fly.io?

I know they support Elixir/Phoenix.

https://fly.io/docs/elixir/the-basics/clustering/

In my experience, deploying BEAM has been just as easy as anything else I can containerize.

WRT getting them to communicate? I ended up giving up on that altogether…

I’m usually good at figuring that stuff out but this one ended up not being worth it (in my case).

What ended up being the problem, maybe is could make this a little clearer (I used https://www.erlang.org/doc/apps/ssl/ssl_distribution as a guide).

Was your requirement to have ssl as the distribution communication mechanism ?

I appreciate it, but it's been a couple of years now, so I can't remember specifically.

We deploy Erlang VMs to EC2 and they communicate just fine. Deploying via a Dockerfile and `mix release` has been a breeze.

As a bonus point, no real need for container orchestration as the BEAM just figures it out.

You already got good answers, I'm only going to add that you can also integrate the self-contained `mix release` final build production artifact with systemd just fine. I've done it.

Reply on news.ycombinator.com