How to sell Elixir again
gist.github.com
gist.github.com
I feel for anyone using a Node.js backend. Y'all really don't know what you're missing.
Also dealing with bugs is infinitely harder. Try catch is infinitely more wordy and messy. And you lose out on so so many valuable CI steps the Elixir community has from sobelow to credo.
And then there's writing tests. In the Node.js community it is an actual joke. It is so hard to write tests and I have yet to see any framework create a nice sandboxed test environment that just works.
All of this stuff in Elixir is free.
And like someone else said, the best ORM is Ecto and it's hard to disagree with that.
Making a reliable and organized Node.js application is actually so difficult.
If my pet project was in Node.js I would not even be able to accomplish it. And that's a fact. Elixir makes me so much more productive and the code I write is so much less buggy and less complex.
I love that HN loves Elixir. If only it was more common in the workplace.
And there's no repl if you're using transpiled code. I have tried using `ts-node` but that didn't work. In Elixir this stuff just works.
Maybe there's some tooling I am missing out on with Node.js. I am not familiar with Deno.
Refactoring code without static type checking is a nightmare. If you're using a strict statically typed ORM like Prisma [1], you can change your data model then run tsc --noEmit --watch in your terminal and the compiler notifies you of every error with their associated file names and line numbers. In VSCode you can even click the line number and it'll open the specific file where the error occurred. Before adopting Typescript, I basically never attempted large refactors, and when I did it consisted of a long process of grepping for specific symbols in the project, and manually writing lots of tedious and verbose tests to do a fraction of what the compiler does automatically.
I remember developers from the Elm community years ago constantly advocating for static typing, and thinking "this has to be hyperbolic". Richard Feldman gave a talk about how he wrote code on an airplane for hours without an internet connection, and without actually running the code, and how the compiler ensured that the code just worked [2]. Typescript is not as sound as a Hindler-Milney type system, but the benefits are similar (especially if you're running Typescript in strict mode).
In Elixir you have the option of calling functions that raise vs functions that return an error tuple so there's no contention there (the difference between function() (this does error tuple) and function!() (this raises).
Elixir also has this idea of a typespec and I have never felt I needed more, personally.
Also even with Elixir's typespecs, static typing is only fully useful if the entire ecosystem around a language embraces it. If some of the libraries you use don't ship with type definitions, it's going to be difficult for any static analysis tool to do a decent job. Almost every major JS library either ships with its own types, or has quality third party types available via Definitely Typed. It doesn't seem like the Elixir community has really embraced static typing to the same extent that TypeScript developers have. The creator of Phoenix for example hates dialyzer [2].
[1] https://elixir-lang.org/blog/2022/10/05/my-future-with-elixi....
[2] https://elixirforum.com/t/do-you-use-dialyzer-in-your-projec...
Is this coming from experience? Because it surely doesn’t match mine
From my experience with TypeScript, you really want your type checker to be eager about complaining about type issues, which is why so many people turn on strict mode. In fact, occasionally I'll have to suppress a false positive error with a @ts-expect-error comment.
But even assuming Dialyzer is as good as TypeScript, have you found that the libraries you use actually have quality type definitions? The few times I've used a JS library without type definitions it ended up breaking type inference, and infecting the codebase with the any type, which renders the type checker basically useless. How has your experience been in regards to that with Elixir?
[1] https://elixir-lang.org/blog/2022/10/05/my-future-with-elixi....
I have some experience with Typescript and it was way better than vanilla JavaScript. Typescript mappings we’re available for most libraries, though for a couple they were not up to date. Similarly for Elixir, the coverage for nearly all APIs is pretty good but also has some holes. In my experience it hasn’t been a big deal either. Any serious library developer puts types alongside their function signatures/implementation.
> The few times I've used a JS library without type definitions it ended up breaking type inference, and infecting the codebase with the any type, which renders the type checker basically useless. How has your experience been in regards to that with Elixir?
Being a functional language, Dialyzer can easily make inferences on untyped code based on ‘success typing’ (what Jose talks about), though it can make for some very obtuse and verbose warning messages. This inference is in addition to safety provided by typespecs which every Elixir library developer typically intermingles with their API surface.
If I would level criticisms at dialyzer it would be its sometimes difficult to read warnings, it’s speed (despite being multithreaded) and the race conditions in the VS Code plugin (which is looking for extra maintainers – if I had time I would help). For the weird warnings I find I just need to add extra typing to my code to narrow it down, which I should be doing anyway
One of the advantages of TypeScript is that VSCode is written in TypeScript, and both VSCode and TypeScript are developed by the same company, so there's a really nice synergy there. I imagine Kotlin users feel the same way using Jetbrains products, and Swift users feel the same way about XCode.
Dialyzer looks interesting, but I can't imagine giving up on the expressiveness of TypeScript. Some of the things you can do with generics, mapped types, intersection types, template literal types, conditional types, and utility types are almost mind boggling. It's difficult to reap all of the benefits of static analysis without some of these advanced type operators. The type manipulation section of the TS manual is really underrated.
Someone for example wrote an SQL parser in TypeScript that requires no runtime code [1]. It can infer the types of an SQL query's result based on an SQL string without any runtime code execution. There was a similar project where someone built a JSON parser entirely using the type system [2]. There's also an ongoing discussion on Github about the the fact that TypeScript's type system appears to be a Turing-complete language with some other cool examples [3]. My point is that the type system is incredibly expressive. You rarely run into an idiom that can't be typed effectively.
[1] https://github.com/codemix/ts-sql
[2] https://twitter.com/buildsghost/status/1301976526603206657
All languages have their own flavor, and their own pros and cons, and if the advanced types of Typescript work for you, then great! In my own experience, I have found Elixir and it’s typing capabilities to work well for me
The SQL example I linked to isn't something you'd use to interact with a database in production, for that you'd probably reach for an ORM like Prisma. I was just trying to demonstrate the level of type inference you can achieve with TS. Going from a pure string of SQL or JSON to a concrete type without actually executing any runtime code is pretty crazy.
> Elixir typing can handle a subset of intersection type which is rather niche
I personally use intersection types quite a bit. If union types are like the logical or operator, then intersection types are like the logical and operator. Being able to define a type that combines one type and another is not a niche workflow for me.
> In my own experience, I have found Elixir and it’s typing capabilities to work well for me
Can't argue with that! Everyone has their preferences.
Elixir/Erlang might already be able to do something like this with metaprogramming. It’s certainly possible to generate and run Elixir at compile time, and map types are already a good superset for JSON objects, so a compile time JSON to map could then provide an inferrable type. I think someone more savvy with Elixir would know more. I’d certainly not something that I’ve needed.
Going from a JSON string to a type for it is actually one of the easier examples of inference I can imagine. JSON is a data description format in which all the base types are syntactically distinguishable, it has no variables, no arrows (i.e. functions), no generics. In the topic of type inference, you can't have a much easier example.
SQL is more complex, indeed, but still doesn't seem too crazy if you have access to table schemas. It's also a matter of whether triggers and stored procedures are taken into account, but I assume they're not.
There's a lot of prior art described in literature as well as practical programming implementations with much crazier, yet successfully working type inference.
The example I linked to is taking a serialized JSON string, and parsing the literal characters in the string (characters like double quotes, commas, whitespace, etc) into a type, purely using type annotations. And the structure of that JSON can be of arbitrary nested depth.
All of this is accomplished using template literal types which allow you to make assertions about the contents of a string. In TypeScript you can assert more than just "this value should be a string". You can make detailed assertions about the structure of strings, and that's what allows these parser demos to exist.
When you combine these template literal types with recursive types, conditional types, and TypeScript's infer keyword you can do some pretty interesting type level programming.
Just to further demonstrate the point, there's an interpreter for the BF programming language, written entirely using TypeScript type annotations [1].
> There's a lot of prior art described in literature as well as practical programming implementations with much crazier, yet successfully working type inference.
Has any of this been demonstrated in Elixir?
Metaprogramming is also why generics aren’t really needed in Elixir or Erlang. All specializations boil down to their reified types using macros
It's not the compile time text processing that's interesting, it's the fact that this compile time code can yield incredibly detailed types, and the fact that it's all done purely using type annotations. Almost every language that's used to build websites has some sort of server side templating language where you can sprinkle some code into your html templates.
> That it hasn’t been done yet is probably because no one has deemed to worthwhile to implement.
Maybe, or maybe it's less feasible without an advanced type system. The final version of the JSON example we're talking about ended up being 61 lines of code. Without some concrete Elixir code to inspect, this conversation is becoming very abstract.
Honestly even if some of this were possible with metaprogramming, my intuition is that it would be much more verbose and complex. I feel like if I tried to implement all of the features of TypeScript by mucking around with an abstract syntax tree using metaprogramming, I would end up with a difficult to maintain crude approximation of the real thing. I don't think we're giving compiler developers enough credit by saying everything they've worked on can be replaced with a quick macro. José Valim's blog post on static typing ended with an announcement that they've sponsored a PhD student to work on these problems.
After two days of discussion, I think we're reaching a bit of an impasse. Honestly just use what works for you!
You've talked right past me. Metaprogramming is not compile time text processing.
> Honestly even if some of this were possible with metaprogramming, my intuition is that it would be much more verbose and complex.
No. It's probably even how the Microsoft team are achieving the typing you're talking about. The special sauce here isn't as special as you think.
> I think we're reaching a bit of an impasse
Not really. I was just saying that what you're talking about is totally possible with metaprogramming. Not making some rhetorical play.
> José Valim's blog post on static typing ended with an announcement that they've sponsored a PhD student to work on these problems.
That's specifically because strict static typing with guards^ is at least difficult, but maybe impossible, and that's what the PhDs are trying to figure out. However, dialyzer isn't as hobbled as you imagine/purport it to be – the crux of my counterargument – and actual experience trumps speculation or casual reading on the topic.
FYI, guards and pattern matching are one of the reasons why Elixir is so damn expressive, and this kind of function typing isn't available/possible in most non-functional languages
I'm not talking past you. I never said metaprogramming is compile time text processing. You said "Elixir can already accomplish compile time text processing with metaprogramming", and I was just pointing out that the text processing itself is not the most interesting part of the example, it's the resultant types.
> No. It's probably even how the Microsoft team are achieving the typing you're talking about. The special sauce here isn't as special as you think.
But I don't have to reimplement any of this, because Microsoft has already written it. With enough time maybe you could implement dependent typing using metaprogramming for example, but would you then say that Elixir is just as good at dependent typing as Idris, which has that feature built in?
> I was just saying that what you're talking about is totally possible with metaprogramming.
You've graduated from saying "I think someone more savvy with Elixir would know more." to "totally possible"[1]. This does not sound like an argument from experience.
> That's specifically because strict static typing with guards^ is at least difficult, but maybe impossible, and that's what the PhDs are trying to figure out.
A lot of what you're talking about has already been implemented in other languages. This is not untrodden ground. There's no syntactic sugar for guards in JS (which TS just adds type annotations to), but semantically it's very similar to type narrowing in TS [2], which I use daily. This is not something I've read about casually on the internet.
[1] https://news.ycombinator.com/item?id=35944988
[2]https://www.typescriptlang.org/docs/handbook/2/narrowing.htm...
Was speaking specifically about your opinions on Dialyzer and Elixir. It's very much a cursory Google around and having stronger opinions about the limits of a language than people who spend time with the language.
> You've graduated from saying "I think someone more savvy with Elixir would know more."
It took further reflection on the idea. It helps to start with at least some epistemic humility, once again the crux of this thread. I'm willing to revert to, "It ought to be possible" given that macros boil down to the final generated code (including typespecs) which is then analyzed by Dialyzer
> Elixir is just as good at dependent typing as Idris, which has that feature built in?
Metaprogramming allows features to become 'built in'. e.g., being able to integrate type checking and compiler errors for HTML/XML or other DSLs.
https://hexdocs.pm/eex/EEx.html#function_from_string/5 https://hexdocs.pm/phoenix_live_view/0.17.0/Phoenix.LiveView...
I think your experience with Elixir and Dialyzer (or perhaps emotional attachment to these tools) is blinding you to the fact that there are features that your favorite language lacks. I gave you a concrete example of something that's a research topic in Elixir (inferring types from guards), which has an existent analogue in TS (type narrowing). You completely ignored that and fixated on the one sentence about experience. Your entire argument is that I should defer to your authority on the topic as an anonymous commenter, because you're supposedly very experienced. It feels like a very hollow appeal to authority.
> It took further reflection on the idea.
But by your standards simple reflection is not enough to establish authority on a topic. You need to have deep personal experience with using macros to parse text before you're qualified to make a firm judgement. You event went so far as to speculate on how TypeScript implements type checking and claimed that the "secret sauce" is probably metaprogramming, despite having never worked on the compiler. This all feels incredibly contradictory.
> Metaprogramming allows features to become 'built in'. e.g., being able to integrate type checking and compiler errors for HTML/XML or other DSLs.
Built in does not mean "I might hypothetically be able to write my own macro that does this". It means it already exists in the language today.
Honestly this discussion feels like it's devolved into repetitive bickering, which is why I let your original comment about metaprogramming stand unanswered, and you reacted by following me into a different comment thread to reiterate the point.
You've really misjudged me. I've been programming professionally for 20 years, I have used and still use a variety of languages including Typescript.
> Your entire argument is that I should defer to your authority
Never made such an argument. My argument is that you have less authority on the subject matter than someone who has spent years with the language.
> I gave you a concrete example of something that's a research topic in Elixir (inferring types from guards), which has an existent analogue in TS (type narrowing)
At best a subset of functionality than an analogue. This kind of research is being conducted by experts in type theory outside of Elixir. If it's a problem for Elixir, it's a problem for any other language that would attempt it, and absolutely I would defer to the authority of those experts who have spent years looking at type theory.
> despite having never worked on the compiler
Knowledge of meta-programming doesn't require the same skillset as writing a compiler. I'm certain you or any other capable software engineer, would be able to write a macro that parsed JSON, interrogated the typing of that JSON, and spat out a typespec with some knowledge of Elixir `defmacro`, `quote`, and `unquote`.
> Built in does not mean "I might hypothetically be able to write my own macro that does this".
Never made such a claim. My claim is that the JSON typing mechanism isn't really something that's dependent on the Typescript language and toolchain. As with my link on my last post of inline heex components, it's possible to take a text representation of something in Elixir and transform it into something that's available to the compiler (and hence also available to Dialyzer).
> bickering
Trust me there is no spite from my end.
Ironically the researcher that you're deferring to (Giuseppe Castagna) wrote a paper about the set theoretic types Elixir is hoping to implement one day, and he points to TypeScript as a practical existent example of these types. This is the same paper Elixir's creator recommends as an introduction to the research [1]. The paper is called "Programming with union, intersection, and negation types" [2]. Here's a quote from the conclusion of the paper (page 55):
"...the need of set-theoretic types naturally arises when trying to fit type-systems on dynamic languages: union and negations become necessary to capture the nature of branching and of pattern matching, intersections are often the only way to describe the polymorphic use of some functions whose definition lacks the uniformity required by parametric polymorphism. The development of languages such as Flow, TypeScript, and Typed Racket is the latest witness of this fact."
You might also want to take a look at the discussion that took place on HN about the paper when it was published [3].
[1] https://elixir-lang.org/blog/2022/10/05/my-future-with-elixi....
The point is, if a solution already exists, it's implementation details would simply be imitated and no further research would be required. The SOTA in set-theoretic types is not sufficient to capture typing for guards, which last I heard was a focus of Castagna's research specifically for Elixir, though the sophisticated typing of Elixir may also present other significant challenges.
Again, type narrowing isn't related to pattern matching on guards. As far as I'm aware, pattern matching on function signatures is impossible for Javascript as it doesn't even permit function overloading - though Typescript could do name-mangling like C++ does to get around this. Guards allow you to overload functions based on the shape of the data, and guards can be quite complex when chained together (e.g., 'and', 'or').
But to refer back to the stated project goals, its aims are more ambitious than the SOTA:
> - The Elixir team wants to go beyond type annotations and make type signatures a direct part of the language in a way to add new idioms and expressive power to the engineering teams that opt-in. This should open new perspectives in the theory of typing and the application of bi-directional typing.
> - The language and runtime of Elixir have a distinct focus on fault-tolerance, concurrency, and distribution, which can lead to interesting typing considerations and modelling
Imitating ideas from one language to improve another is a form of research, and can be fairly difficult.
> As far as I'm aware, pattern matching on function signatures is impossible for Javascript as it doesn't even permit function overloading
There's no syntactical support for function overloading in JS, but it's fairly trivial to accomplish the same semantics using normal control flow (i.e. conditionals). This is why type narrowing (which uses control flow analysis), is relevant here. The paper I referred to above actually points this out. On page 6 under the section labeled "FUNCTION OVERLOADING", the author wrote:
"However, intersection types let us express ad-hoc polymorphism (i.e., function overloading) if coupled with some mechanism that allows functions to test the type of their arguments."
The author then goes on to describe this technique in more detail on the very next page:
"OCCURRENCE TYPING: Occurrence typing or flow typing [59, 46, 17] is a typing technique pioneered by Typed Racket that uses the information provided by a type test to specialize the type of some variables in the branches of a conditional...This method of refining types according to conditionals is important in type systems for dynamic languages and in those that enforce null safety: some examples include Ceylon [38], Dart [27], Flow, Kotlin [37], Typed Racket, TypeScript, and Whiley [47]."
> But to refer back to the stated project goals:
Thank you for linking to the list of project goals. The second bullet point under the list of goals says:
"To capture some programming patterns used in Elixir programming one needs to resort to the techniques of gradual typing, parametric polymorphism, and occurrence typing, three tools whose theoretical aspects have been already studied for semantic subtyping".
TS of course exhibits gradual typing and parametric polymorphism (generics), but did you notice the mention of occurrence typing? The same form of typing I referred to above, and which the author explicitly mentions exists in TypeScript (amongst others).
I honestly don't have the time to devote to this conversation any longer. I will most likely allow you to have the final word, because this discussion could carry on ad infinitum and is incredibly unproductive. As Paul Graham once said this might be taken as a tacit admission of failure [1], but I have things to do.
[1] https://news.ycombinator.com/item?id=34043420#:~:text=when%2....
Strictly speaking, Dialyzer has two big benefits over more traditional type systems:
1. It doesn't require type definitions to be present, it can type check code having none at all.
2. "Dialyzer is never wrong", i.e. it never returns false positives, i.e. it's an underapproximating checker.
These are the design decisions taken by the tool authors. There's rationale behind them, though the experience of using checkers for other languages might not be 1-to-1 applicable because of that. These decisions come with downsides, of course, some of which are covered in this thread, so I won't repeat them, but in general they were taken deliberately due to characteristics of the language(s) (it was originally created for Erlang, not Elixir) and the state of type system theory at the time. Please bear in mind Dialyzer was created in 2006, some 6-7 years before TypeScript.
Ecto is good but I think the Django ORM does it better. The ORM in Django[0] is dead simple and easy to understand. The only caveat being that Django doesn't have changesets, which I do like.
It also integrates nicely with the Django Admin. You setup your models, and you can very easily generate an admin interface to CRUD on those models.
One annoyance I have with Ecto is that migration files are handwritten. There's a generator tool, but it only gives you a skeleton migration that you then have to manually fill in. This gives you 2 sources of truth, which is error prone. Automating this process like in Django solves this issue.
Ash[1] (which uses Ecto under the hood) does solve most of these issues, although at the moment the documentation can be hard to navigate.
[0] https://docs.djangoproject.com/en/4.2/topics/db/queries/
Oh no, I strongly disagree. I know both very well, and Ecto is much better and flexible. SQLAlchemy was great, when I last used it two aeons ago, but still subpar compared to Ecto.
But also apples and oranges. Django has an ORM (i.e. stateful), Ecto is a stateless data mapper.
This is why there was so much push in Rust to get async anything up and running.
There are a LOT of people fed up with the current web backend ecosystems.
This is a huge disadvantage, and extremely dangerous, and in my opinion should render the entire language useless as a backend language for any serious project.
Elixir may not be a silver bullet. But Node.js is definitely a vampire...that isn't on your side.
I also think that livebook has a couple of really killer features now that make anyone who spends a significant amount of their time in Jupyter notebooks to stop and take a second look (eg it’s so easy to share a livebook compared with sharing a jupyter notebook, or even easier to extract code from a livebook prototype to turn into a resilient production elixir module).
Yes there is still a ways to go to catch up to the maturity and breadth of the community that Python has in this domain, but that gap is closing way faster than I could have thought.
I think the biggest challenge for adoption is its functional nature. If you take the time to learn about elixir and the BEAM you are rewarded with a pretty complete runtime which ships with its own DB, in-memory Store and message queue (with a very easy path to adopting more battle tested versions of these when you outgrow what is included in OTP). This means spin up time prototyping new systems is spent less on vendor choices (which type of DB should I use) and more on solving your problem domain.
I wish I had know about all this stuff 5 years ago…
The topology is exactly the same
In fact, porting your vetted numeric code to Elixir is probably going to make the risk go sky high, but running the same code with supervision structure in Elixir would in fact reduce risk so is a no-brainer
Just my thoughts of course
I think pattern matching and the pipe operator combined with the let it crash approach work very well in practice. There are amazing libraries for Erlang/Elixir including database client, dataframes, JWT, http, AMQP client and many more. I haven't had a single library that was not working out of the box the way we wanted.
The code is pretty easy to follow and we can onboard a new engineer with ease. The usual ration for our codebase comparing to Python is 1:10 in favour of Elixir. The engineers on the team have 10+ years Python experience and only used Elixir for toy project before.
And yeah, resilient design de-risks itself and that alone is a huge gain because it also reduces other forms of complexity.
Still, I think we have to keep pushing to debunk other myths from "it's impossible to find talent" and the fear of "no one uses" that are common myths about any technology that is not the #1 most used.
[1] https://survey.stackoverflow.co/2022/#technology-most-popula... [2] https://madnight.github.io/githut/#/pull_requests/2023/1
More good & relevant information + libraries -> more acceptance (i.e. my use case is no longer a tech uncertainty)
The other thought would be that if you didn’t have to do anything yourself then you must have no moat.
Also, I think we’re agreeing here, but there are a huge set of things that you may need to build an application that aren’t the core value prop of your company. Buying into a more mature ecosystem makes it more likely that you don’t have to build those things and can spend more time on the moat stuff.
https://survey.stackoverflow.co/2022/#most-loved-dreaded-and...
Simply put, many developers don't yet know they're gonna love it!
Actual response: Some similarities, different goals, different strengths, different abstraction levels. Go is cool I hear but I don't see a need from over here doing Elixir.
21 years of experience in total, including in C/C++ and Java.
It's been 4 months and counting that I've been trying to land an Elixir freelance contract.
My theory is that dev. happiness and productivity is so great in Elixir land, there is not much churn, so talent is available, jobs, not so much.
(j/k, no worries napsterbr)
Of course, those with more experience tend to have heard of Elixir and be keen to work in it.
People with less experience are more concerned about whether investing in this language is a good career move, so we emphasise how learning something like Elixir will ultimately make you a more valuable developer.
If you’re considering adopting Elixir for your new project/startup and worried about finding talented developers, I don’t think Elixir will make it harder and might even make it easier.
It depends on how you measure sustainable. What would happen to Elixir without its BDFL? Has Elixir achieved escape velocity and will continue? Rust was pushed out of the Mozilla nest and managed to live on. I don't see a similar capacity for Elixir handling such existential risk. Are there succession plans? Is there mentoring among members of the core?
I also think that it's necessary to point out that just because OTP facilitates resilience doesn't mean that the system you'll build with it will exhibit that behavior. You have to design your system to use OTP effectively. Easier said than done. I've seen Elixir systems reach a critical failure state following maximum restarts of its internal processes. However, at least Elixir has the potential of being hardened. It's just not going to act as expected without a lot of up front wisdom applied.
We do test systems to the point of breakage. It is not something we skip just because the language is good.
As to existential risk:
I will try to be succinct. If there has been no exposure then there is no need to hedge.
Further, there are commercial firms which will maintain your critical systems. In other words, a long term maintenance contract for Elixir systems can be procured from more than one source.
If you already work with Elixir, I have couple of posts that might be helpful[0].
What I love most about elixir is the ridiculously dead-simple vertical scaling. A close second is The World’s Best Take On ORMs (aka Ecto).
Horizontal scaling is what's hard for a lot of systems, but simple with Erlang and Elixir since they are, by design, meant to support concurrent and distributed workloads. That's why I was asking if they meant horizontal and not vertical.
Erlang and Elixir are not uniquely easy to scale horizontally, but they are (if you don't fight their nature and produce programs that consist of one giant execution thread) ridiculously easy to scale horizontally by tacking on additional CPU cores or physical CPUs.
2014 - https://www.infoq.com/presentations/whatsapp-scalability/
There is also Lasp, which is very interesting: https://lasp-lang.readme.io/docs
My interpretation was vertical scaling is within a single node and adding cores or CPUs to a single node is vertical. Multi-core and multi-node really present similar problems, so adding more cores or adding more nodes isn't terribly different from a design perspective.
> ...easy to scale horizontally by tacking on additional CPU cores or physical CPUs.
If you define adding more cores to a single machine is actually horizontal scaling, then we agree in principle, just not in the definition of scaling.
Not true in practice. You can scale hardware easily to hundreds of cores, if your app is able to spread the load, but the performance ceiling is very low if your app is single-threaded.
The BEAM with its design is very parallel, and the more cores, the happier it is.
The challenge is the prevalence of hypes, which as pointed out insightfully, can lead to irrational choices.
I think Elixir/Phoenix could use a little bit of Javascript (and modern PHP) enthusiasm to help with adoption - this all seems a bit…dry.
[1]
[...]
./deps/phoenix_live_view/assets/js/phoenix_live_view/view_hook.js
./deps/phoenix_live_reload/priv/static/phoenix_live_reload.js
./assets/package-lock.json
./assets/vendor/topbar.js
./assets/js
./assets/js/app.js
[...]Previously have seen: https://github.com/patrick-steele-idem/morphdom
I do not know how true it is today.
If you want to handle a websocket and have a 2 way bindings, then no way around it. But that is not phoenix fault. It is the reality of the browsers api. It is pretty minimal js though.
My stack these days is just Phoenix, Liveview and a sprinkle of client-side interactivity with Stimulus.js, which is super-simple, composable, progressive (i.e. just add it where you need it) and plays excellently with Liveview.
I am designing a super interactive, fully synchronized in real time dashboard with no more than 100 lines of Javascript code to handle: dropdown menus, show/hide burger menu, scroll to top after pagination. Just as it was originally designed, JS should only be the icing on the cake, not the entire buffet.
Yeah, I'm so hipster I'm already living in a post-Javascript world. Feels good, man.
I hate code generators with a passion. They obfuscate the inner workings of the library. Have you tried to find a tutorial which do not use the Mix task "phx.new"? (an up-to-date one).
For example, if I need an API, I will pass `--no-assets --no-html --no-live --no-mailer --no-dashboard` and the generator still creates a static files folder and static files related code.
If I want the strict minimum, I have to take some time to remove the un-necessary code.
Don't misinterpret me, I think Phoenix is a great framework, just like Python/Django, and Ruby on Rails are.
But sometimes, I just want Flask/FastAPI. You can say "just use plug_cowboy", but IMHO, "plug_cowboy" is more like "express.js", not really a FastAPI-like framework.
Also, absinthe is the only GraphQL implementation that natively supports subscriptions (via phoenix+phoenix_pubsub). In Django (graphene), you need so much more boilerplate with django-channels and yet another library. In JS land, Apollo Server also do not support it natively, you need graphql-ws and a pubsub implementation as well. And most of those pubsub implementations will need a Redis (or whatever) to distribute the queue between multiple web workers, while Erlang/Elixir have been designed for this (which actually simplify your infrastructure).
In the end, I often go with Django Async (via ASGI) with Server-Sent-Event (via an async StreamingHTTPResponse) to avoid the complexity of GraphQL subscriptions. But even that, I need a pubsub via Redis to make my multiple workers aware of each other. While with Elixir, a simple libcluster dependency, and a `receive` in the endpoint handler would be enough :)
But speaking of generators, I love this site: https://www.phoenixdiff.org/ — diff your local Phoenix version with the latest one to bring your codebase up to date. I use it every major release to adopt the latest best practices, even if Phoenix has decent backwards compatibility.
What do you mean? You can use other libraries, for instance Guardian or POW.
Take a look at how django does it. `django-admin startproject` creates:
- settings.py
- wsgi.py
- asgi.py
- urls.py
Even with the codegen tools, you still need to learn what a plug router is, what a phoenix controller/view is, what an ecto repository is, etc...
The codegen tools just overwhelm you with irrelevant code (i'm making an API, why the hell do you generate static files? and why do i need to edit so many files to take it out?)
Another point I don't like about Ecto is that you can't easily switch the database backend depending on the environment.
With django, I can have an sqlite3 db for the dev environment, and a pgsql one for the production. I don't want to run a docker image for pgsql on my machine. I just need to read the DATABASE_URL environment variable, and the ORM takes care of the compatibility. If later on I need some specific pgsql feature, I will have to run the docker image on my computer anyway, but until them please keep the complexity out of my way.
The documentation explains every part of a Phoenix app pretty well.
The code generator is not there to learn you anything. That is your own job.
Adding a controller or schema to results in a small number o f new files. Git commit before you do and you can easily understand what was added.
I can argue the exact opposite: you can just look at the generated code to understand what is going on. No magic involved.
> For example, if I need an API, I will pass `--no-assets --no-html --no-live --no-mailer --no-dashboard` and the generator still creates a static files folder and static files related code.
To me it sounds more like an issue with phoenix rather than with codegen in general. I would expect the static folder to not be generated if we pass the "--no-assets" flag.
Definitely, although as someone who finds that level of "enthusiasm" or fanboism a turn off, I like that it's not there. But it has to be there to ensure the success and growth of the language/ecosystem.
It's a little bit better than what it might look like on the surface level as you do a lot of pattern matching and dialyzer. It's far from what you get from Rust of course, but it's better than with Python for instance.
I found maintaining the Typescript codebase in the same company much easier.
So, I leafed through this document, and did not see how it would help people sell Elixir. Engineers will not be offered reasons to prefer it to other languages (popular or otherwise), and managers / bean-counters will not be convinced by the claims of it being less risky and a robust choice for a glue language - when very few people know it, so maintaining the elixir code will depend on some local guru and/or training people to use elixir.
... and this all is not to disparage Elixir! I just don't feel I've been told almost anything about it.
For an experienced dev wanting to pick up Elixir I would suggest picking up Elixir in Action [1]. The third edition is currently in MEAP. Much more concise and great discussion of some Erlang/OTP internals too.
[0] https://github.com/DockYard-Academy/curriculum
[1] https://www.manning.com/books/elixir-in-action-third-edition
Haven't actually gone through it yet, but hoping to over the next month.
Use the code THANKYOU10 to get a $10 discount ;)
But I'll bite: What's wrong exactly with using Github's markdown renderer?
This is a plain markdown file, the author could host it on S3 and it'd look like:
https://gist.githubusercontent.com/evadne/c9aeca424d30f024ba...
There are a ton of markdown renderers, author could write one in a few hours for the feature used (#, ##, ###, >, and unordered lists), or even better, paste the thing in one of the many markdown-to-html tools (https://markdowntohtml.com/).
This is not like the author used some proprietary syntax, that only displays on Github.
GitHub has forked Markdown in a way that is incompatible with CommonMark, GitLab, etc. So the author might accidentally buy into that fork which there is no quality control for (just look at the lack of care put into their attempt at admonitions).
You can share it elsewhere if like
Resilience, within the context of this presentation, and the philosophy of Erlang, would be the capability to recover from errors while unattended.
The application runs out of memory due to too many transactions being processed at once. The problem happens once every day. However the system gracefully recovers, and the team is able to resolve the issue peacefully without the problem escalating to an active incident.
Hard crashes are not an acceptable substitute for observability, or continuous improvement.
This example doesn't even rise to the level of an "active incident" in the Erlang philosophy. In other words, it's not a bug, so there's no urgency to improve it.
Graceful recovery means that something handle that failure after these transactions failed. There is no data loss. They may have been slower, but i think we can agree that a slight temporary latency for no dataloss and graceful handling of unexpected stuff like your database machine being on fire is not so bad?
It's still a bug that should be fixed, it's just that the effects are better contained thanks to the ability to self-heal.
But in practice, if you're on the BEAM and use OTP as intended, your code is fully parallel, split into fault-tolerant pieces, and one crash does not affect the entire system.
I have been responsible of two Elixir monoliths in the last 6 years, and I have not seen them crash even once. Not even during prototyping, not even during load testing, never once. Without having to adopt defensive coding strategies. I write 1/4 the error checking code than if it were Rust, and 1/10 than if it were Python or Go.
How many platforms can realistically promise this?
Note, I haven't said my code is bug free. It's buggy because I'm only human and make plenty of mistakes. But the BEAM saves me from myself, saves me from third-party JSON endpoints that for a split second return corrupted data, saves me because the DB connection dropped for a second, and saves me from the myriad of transient heisenbugs that happen in production.
The bugs are still there. But the system is fault-tolerant because it has been designed to be.
--
If Saša Jurić can't prove it to you, then no one can: https://youtu.be/JvBT4XBdoUE
(I'm pretty sure every time someone posts this link, the number of Elixir converts increases a notch)
Trying to continue this conversation is a waste of our time. Feel free not to use Elixir.
It is quite frustrating when someone has already decided something they don't know is dumb or makes no sense. It is more plausible the unknown unknowns are blinding your judgement, than Erlang and Elixir developers worldwide suffering a collective hallucination, is it not?
Empty your cup.
To me these phrases are more about the guarantees—if it does crash it recovers—and less about dogmatic advice that everyone follows.
But there's time and place for different levels of defensive programs. I get to pick and choose based on where I see risk, but I know the whole app isn't going to blow up if I missed something.
With OTP, you have what we call a "supervision tree". Your software is divided into components (erlang processes) and organized in a tree where the leafs are your components, and the nodes are the "supervisors" which will catch the error, and restart/retry the operation.
What's the result? Imagine you have a component that is polling data from the database and then sending a message to another component based on that data. For a split second:
- there is a network error
- you lose the connection to the database
- the component crashes
- its supervisor notice the crash, restart the process
If the network error was not temporary, the parent supervisor will notice that the component crashes far too often, and will decide to crash as a result. Letting the parent supervisor of the supervisor deal with the problem.
In Erlang/Elixir, processes can "monitor" other processes. They will receive a message when the target processes dies (normally, or abnormally). This allows you to delegate the error handling so yes, you write less defensive code, because that code is located in a dedicated component that will be notified automatically.
There is a limit to how defensive you can make your code. As soon as you call out to another service for data, the universe of errorinf situations they can create goes to infinity. For example they could happily send you a 200 but return mistyped JSON. What do you do in that case? Are you genuinely going to write that as a test condition to make sure the transactional chain in your system handles this and rolls back the db inserts that preceded it? I can tell you that in 99% of cases idiomatically written elixir will "do the right thing" if this happens, and that is a consequence of "let it fail" philosophy. Moreover, if you try to write error handling code in the traditional sense, you will probably get it wrong, it will be hard to understand, and it will be hard to debug if your implementation is not quite right.
I love the OTP framework. So much that I suffer from "erlang envy" in every other language I'm using.
Which is why I made this a couple years ago: https://linkdd.github.io/triotp/ (not benchmarked, not tested in production, only an experiment)
Seems like most of their issues would not apply to a greenfield project and now that things are stable, they're quite happy with the choice.
Traffic estimate: https://www.similarweb.com/website/cars.com/#overview
If people build multi-billion dollar companies on top of PHP and Node.js, I would not even be concerned about scaling Phoenix.
I think it would have been better if he had more time. The first part was nothing special as such, it could have been easily replicated in even crusty old things like Delphi, never mind modern C# or similar.
It only became interesting when he showed the introspection and when he scaled out to another instance. These things you cannot do so easily in other systems. So would have been more interesting to spend more time on that stuff.
I'm sure there are other videos doing just that, just my thoughts on this one as a "PR video".
Who says data has been lost? Only if your protocol is fire-and-forget with no checking. But then you'd have data loss no matter the language(s) you implemented it in if your system became inundated with requests and crashed.
Databases have WAL -> Systems have task queues
The executor itself can fail, if the job is not lost :)