But then again, who does not write unit tests to get 100% coverage against 110% of possible user inputs, in case you made a typo somewhere. Right ? Right ? </sarcasm>
Interestingly, coming to Elixir from another dynamically typed language (Python), I tend to do much fewer runtime/typing errors than ever before. I might be more experienced, or the tooling and compiler are good enough to catch the most common bugs.
I get a few "no matching clause" with my data ingestion tool on my Sentry log because external APIs are flaky and a cosmic ray flipped a bit so that I'm reading incorrect data. A GenServer somewhere crashes, restarts and the data comes back clean, without any user disruption at all.
But again, I'm completely ready to hear that it's just my setup, or how we failed to build the app.
Just wanted to confirm that it's completely possible to shoot yourself in the foot, which I'm surprised hardly anyone mention when talking about elixir.
Anyone interested should give it a try.
"Let it crash" is commonly misunderstood. It doesn't mean that you shouldn't treat the known unknown cases of your business (like bad input), it means errors that are generally not known a prior (a database crash or network partition, for example) does not need to have a treatment since the Elixir/Erlang will simple crash and restart.
Things like K8s also brings a similar thing that Elixir brings to you, however having this in language is powerful since it allows ulyou to treat it better, if you want.
Your comments read like "but I was promised magic and I found no magic" to me.
Well, of course there's no magic. What there is is a really good runtime and a good tooling that allows you to avoid some mistakes preliminarily.
Maybe try VS Code with the Elixir LS plugin. It runs dialyzer automatically and highlights errors which also catches a ton of those runtime type issues. It’s pretty nice, often more expressive than standard typed languages.
http://erlang.org/doc/design_principles/sup_princ.html#maxim...
However, this is not my experience. When something bad happens in my situations, it's usually in response to a bad input that is going to be repeated a couple milliseconds later. So the app just crashes in loops.
I don't understand how the "let it crash" philosophy is supposed to happen when the only thing that can happen after a crash is... Another crash. Maybe my case is just too strange.
As far as helping prevent bugs, let it crash won't help. It is not a replacement for a type system either. A type system for Elixir would definitely help you prevent bugs (but unfortunately implementing one is challenging).
Let it crash is a mental model and a technique to handle bugs and avoid undefined behaviour when all other attempts to prevent them (be it via static types, testing, code reviews, etc) failed. Imagine some data validation functionality had a flaw and that causes bad data to reach a core component, would you rather:
1. have the core component continue functioning with said bad input, which may have unknown repercussions?
2. have the core component crash and restart itself with a known state?
So let it crash can be useful to guarantee your system is not going to venture into undefined behaviour.
I wrote a small example of this in a blog post about assertive code (http://blog.plataformatec.com.br/2014/09/writing-assertive-c...). If you compare the two get_token implementations, you are 100% correct that in the second implementation, if you receive bad input, the code will crash over and over again. But I would say that I prefer the crash, so I can think about how I want to handle said input, rather than have an implementation that works by accident (and the accident may be a happy or an unhappy one). Would I prefer to have foreseen the bad input in the first case? Of course! And do type systems help avoid undefined behaviour? Definitely! But not all of them. It would be great if we could have both.
On a larger scale, especially when building complex systems, you are going to interface with client/server, database, apis, etc and you will run into unexpected scenarios. You likely can't prevent, model, or foresee all of them, so "let it crash" works as a default behaviour to handle them. It is also a legit strategy for things you know that may happen, but the best way to handle them is to start over anyway. It provides a mental model for you to reason when things go wrong at each individual component, instead of trying to reason how an exception may affect the whole system.
I am not sure if this helps, so questions and feedback is appreciated!
There's no code in the world that can automagically recover from unexpected and not-coded-for user input, though.
I think it boils down to this:
> Here I'm classifying bohrbugs as repeatable, and heisenbugs as transient.
> If you have bohrbugs in your system's core features, they should usually be very easy to find before reaching production.
I don't know about "bohrbugs", but the vast majority of my bugs are "i-am-an-idiot-bugs", and boil down to "some combination of the handling one of the many possible inputs is going to trigger some code path where a function does not have the proper arity because I made a typo somewhere because I'm an idiot."
I claim that those are not "easy to find before reaching production" as a human being. (And I argue a large part of them are trivially found by a decent type static analyzer. Which, unfortunately, dialyzer was not last time I checked, 2 years ago.)
> By virtue of being repeatable, and often on a critical path, you should encounter them sooner or later, and fix them before shipping.
I disagree with that. My bugs happen on critical paths, but with non obvious shape of data. (I receive input from untrustworthy, aka "real-world", sources.)
I've spent enough time fighting bugs that were in the critical part of the code, were tested multiple times at the unit, integration and acceptance level, with different kind of simulators, but still broke on the first real-life usage because one function down a stack was not handling a nil or expected a string instead of a number, somewhere.
Encountering those issues "sooner" becomes a matter of "having enough imagination when writing automated tests".
I've heard property testing could be a tool to "automate" such imagination. But I'd rather have something telling me I just made a typo ;)
> So really, how efficient is restarting as a strategy?
> Well for repeatable bugs on core features, restarting is useless.
Amen.
> if the feature is a thing very important to a very small amount of users, restarting won't do much. If it's a side-feature used by everyone, but to a degree they don't care much about, then restarting or ignoring the failure altogether can work well.
It's hard to not read this as "Let it crash" being a good philosophy - provided it does not really matter, if your application works ;) (I know this is not the intended message, but still...)
I had great expectations about this section of the article:
> > I like statically typed language, and I restart my daemon after unhandled exception. What is Erlang gonna win me on fault tolerance
> This question was asked to me on a forum where I was discussing programming stuff and discussing the Erlang model. I copied it verbatim because it's a great example of a question a lot of people ask when they hear about restarting and Erlang's features.
The author follows with question (which is, basically, my opinion), with a very insightful explanation of supervision tree and OTP, but unfortunately, does not seem to talk about the "statically typed" part.
Which is a shame ! Message-passing, supervision and OTP are the great aspects of Erlang.
They're extremely powerful when used appropriately (which, in my experience, mean "sparingly, and as a plumbing-layer over purely functional code".)
My whole frustration with Elixir is that I want static typing "behind" the boundaries of GenServers.
Sure, I understand "GenServer.send" can not be typed, in general, and I'm ready to do some level of input validation at the boundaries, but beyond that point, please, If I tell you to add two apples, and I mispelled the name of one of them, don't even let me run, okay ?
I also know that there are efforts to do that (alpaca & co), but they're just too early-stages for my taste.
As said earlier, if you're solving the kind of problems where you never those kind of "i-am-an-idiot-bugs", or you're just not as idiots as I am, you may not find this problematic. I do.
But then, for dramatic purpose, the article ends with this gem:
> At some point, a team member got a very expensive credit card bill for the logging service we were using to aggregate exceptions. That's when we took a look at it and saw the horror on the leftmost side of the diagram: we were generating between 500,000 to 1,200,000 exceptions a day! Holy cow, that was a lot. But was it? If the issue was a heisenbug, and our system was seeing, say 100,000 requests a second, what were the odds of it happening? Something between 1/17000 and 1/7000. Somewhat very frequent, but because it had no impact on service, we didn't notice it until the bandwidth and storage bill came through.
Really ? Where those 500,000 to 1,200,000 exceptions not blocking anyone ? Not wasting someone's time ? Not causing improper or delayed results ?
Or, worst, just slowly making people lose faith into your software, or in software in general ? (Okay, I'm being slighly grandiloquant here, time to wrap up.)
Thanks for the link.
There are certainly ways you can get run away reactions that eventually take down the otp tree regardless of your settings and I probably do more input validation than strictly called for my the erlang guide books but not spending all of my time worrying about those edge cases does have it's upside with out negatively impacting other clients.
Or is your cluster more of an agnostic broker / message pump that lets other systems do the logic ?