Elixir v1.7 released
elixir-lang.org
elixir-lang.org
I first came across Erlang a few years back, while I was researching more exotic programming languages, but for some reason Erlang never felt like home. I know plenty of people like it, but personally, I found it difficult to get accustomed to its syntax, especially after having spent most of my time in Python.
But as soon as I discovered Elixir, I got hooked. All the advantages (and disadvantages) of using the Erlang/OTP programming model, while experiencing a nicer, developer-friendlier, and more familiar interface.
Our work is seldom, if ever, not dependent upon the sweat and labor of other people.
So, thank you Elixir team! Elixir is truly a joy.
However, I'm curious: does the lack of types (both Elixir and Erlang are dynamically typed as you know) affect long term productivity on the platform? The lack of proper static typing does make large codebases very difficult to build and maintain, don't you think?
People have mentioned dialyzer a lot in reference to Elixir/Erlang but how good/practical is it?
Sometimes I wonder: how fault tolerant can the OTP platform be anyways if your process can crash due to an unmatched pattern? A type checker should tell you at compile time that you've not tested all possible combinations and this should not happen at runtime. For a process to crash on an unmatched message and have a replacement process restarted by the supervisor to take its place ("let it crash") is definitely cool but seems like an extreme solution. What could have been fixed forever by a type checker is being deferred to the runtime. Thats really my main gripe.
There are probably many scenarios where the whole OTP philosophy of crash recovery is definitely going to help with the robustness of the platform but I suspect a lot of crashes of processes are due to fairly prosaic reasons like unmatched patterns, wrong arguments to functions etc.
In the end a lot of your crashes are going to be "other people's software, like, say, "an unexpected race condition in nfs". Or "some other library crashes when parsing a really big json".
In the end these are not really solved by a static type system. The idea that static types will save you is flawed by giving you a false sense of security and tying up developer time outside of the 80-20 Pareto law... In my short so far experience as an elixir developer I've delivered all of my products on time. Running it I know there are errors since they floated by the console in the background... I didn't care about the 1% error since it did not significantly affect my performance. Not worrying about every last problem has been great for my sanity.
Finally, there is a possibily antipattern Dev cycle, I've settled in to. Unit test the happy path, write happy path, conjure errors, unit test them, solve them with an eye for recovery, spec out the type system, sweep away last lingering errors with dialyzer. Occasionally totally refactor.
I can usually demo any time after step 2.
Add in dialyzer and you have something that gives you the flexibility loose typing and a fair amount of the guarantees of a more formal type system.
And with Elixir you have a way of starting processes again should you still crash with an error.
Never mind that microservices and cross node messaging are built in as standard, as well as confidence in being able to scale your system with things like Flow or OTP.
I'm surprised not to see more traction (especially in startups) and I love a project this complex only has 16 bugs on github!
static typing is somewhat of a programming religious wars. I mean sure, extremely weak typing is IMO horrible (python, ruby, js, I'm looking at you) but being dynamic does mean that quickly testing out an alternative code path. Just today I realized I needed to forward some information to a somewhat polymorphic data structure, a process which I hadn't planned for, I found a map structure that is encapsulated state in a gen_server in exactly the right place, so I just shoved it into the map under a new key. That is effectively breaking what could have been a static type system. And it was two lines of elegant code, no refactoring necessary, and probably a line of documentation saying that in this corner case the structure will contain this extra info.
IMO the right approach is compromise. You could do this in a statically typed system by just making everything a map... which is effectively what the BEAM languages do, but at least there's a type analysis tool that can introspect deep into the built in "flexible" types.
Ruby: duck, dynamic, strong.
Python: dynamic, strong.
JavaScript: dynamic, duck, weak.
Elixir: dynamic, strong.
https://wiki.python.org/moin/Why%20is%20Python%20a%20dynamic...
From this perpective, a type system is a nice to have and a recovery mechanism is a must have.
However, a type system is much more than preventing a certain class of errors. It is documentation, it is a design tool, it can improve performance, and others. Plus, if you can catch errors earlier, why not? At the end of the day, I would love a type system in Elixir, but I am certainly happier that the recovery system is there right now.
Elixir/Erlang do have typespecs, which helps on the documentation aspect, but a type system would give much more on the performance and guarantees departments.
At the time I was attempting to use it, it was also super slow, and libraries were not keeping their typespecs up-to-date. Also a big one: no parametric polymorphism results in `any`s galore. :/
My limited point was: it would be interesting to see statistically what % of time the ERTS (in real deployed production systems) is dealing with errors that could have been avoided at compile time (had their been a static typing system) and how many times are there real runtime errors (which can not be detected by a static type system). Of course with those kind of runtime errors the whole concept of supervision trees etc. is invaluable.
Having said that, I know of no language that offers concurrency via the actor model – which is easy to reason about – fault tolerance, advanced static typing and distribution out of the box, all at the same time.
Thus, I'm wiling to forgo the static typing, in favor of all other features.; no to mention other cool Elixir/Erlang features such as live runtime introspection and tracing, hot code reloading, the OTP, excellent multicore support, and soft-realtime capabilities due to preemptive scheduling and per process garbage collection.
And btw, people have written large systems in languages less stricter than Elixir – e.g. Python, Ruby, etc - thus I think large codebases in Elixir should be fine.
The related works section mention some languages that get fairly close to being actors with types (e.g. Pony), but you're right that we're not there yet.
The tricky thing is that we need types that evolve as messages are being sent - an actor might change behaviour when it receives a message. Academica is still figuring out the kinks of how to do that.
My work assigns a static type per actor and my conclusion is that that's decent - but quite annoying to program with and without the guarantee of deadlock freedom.
https://www.dropbox.com/s/lczrcqu2m9p6osv/Master_Thesis.pdf?...
Thanks for the links :)
I'd also be really interested in versioning, and upgrading a cluster over time, without bringing the whole thing down (like with Cloud Haskell). Would be super handy to be able to verify that your migration works first, before actually going ahead with it. I'd also love it if the types could be used in some way to make this kind of thing easier to reason about.
Are you also aware of the work advancing in Akka Typed? After a number of iterations they made some modifications that made things simpler - not sure how it compares with your work though: https://doc.akka.io/docs/akka/2.5/typed/actors.html#relation...
Personally I don't think Scala is the right tool for the job. Once we figure this out we should design a special purpose language - this is too complex to expose as a library (bad error messages etc.); and while we're experimenting I think Scala's type system is not expressive enough - we need dependent types.
I hope to see someone continue in my spirits but based on "mailbox types for unordered interactions".
Versioning, upgrading and errors are really difficult problems. I think it should be possible to do upgrades in a typed setting but it will not come easy. We'll need to start simple here, e.g. upgrades that don't change any types.
The fundamental error that only "let it crash" can handle is hardware failure. No amount of type checking or exception catching will help if the server melts in a fire. In that case, the only solution is for a different process on a different machine to deal with the failure. This will be true for any programming stack.
Erlang's approach is, since we have to delegate to a different process to handle errors in the most extreme case, let's use that approach to handle all errors, so we only have one mental model for error handling and don't have to do a lot of extra work to handle hardware failure. (Joe Armstrong refers to "errors" as times when the programmer doesn't know what to do, as opposed to exceptions, which can be rescued.)
The performance story is similar: no matter your language, if you want to scale, eventually you'll need multiple processes on different machines with no shared memory passing messages to each other. For consistency, Erlang uses that model all the time.
Erlang's fault tolerance decisions are what lead to its fairly good, and especially consistent, performance characteristics.
I've written more about this on the DockYard blog: https://dockyard.com/blog/2018/07/18/all-for-reliability-ref...
If you're talking about ERTS, sure. If you're talking about Elixir, I disagree. I love the language, but consider:
# Opening a file
f = File.open("foo")
# Reading four bytes from said file descriptor
IO.binread(f, 4)
# Seeking four bytes ahead from said file descriptor
:file.position(f, {:cur, 4})
It'd be like if core use-cases in Clojure required you to use the Java call syntax. Kind of strange. Still a joy, but strange.
> does the lack of types (both Elixir and Erlang are dynamically typed as you know) affect long term productivity on the platform
I wouldn't say so.
The "idiomatic use-case" of Erlang/Elixir is IO code. It's wire-protocol parsing and wire-protocol generation. (Take "wire protocol" here to also include on-disk formats.) Essentially, it's the code that decides what the types of untyped binary streams should be, by doing heuristic things to it and passing around weakly-tagged intermediate representations in the mean time; and it's the code that generates those untyped binary streams.
When Erlang/Elixir are being used for these idiomatic use-cases, static typing, or even reified dynamic typing, doesn't really make sense. You don't know yet what the system-internal types are. Erlang's type system, such as it is (one product type, called "term", which can be an integer, a binary buffer, a boolean, an array/tuple of terms, a linked-list of terms, a map from terms to terms, etc.) is the type system of a parser/generator. In fact, to write parser code in e.g. Java, you must first reimplement exactly that type system, and then express your logic in terms of it. You're Greenspunning Erlang!
(Other high-level languages, while requiring this greenspunning to write custom parsers, make using certain syntax-level marshalling features for particular, limited kinds of marshalling really easy. See, for example, Go's generic "encoder" lib, that works with many individual serialization format libraries, and relies on struct field annotations to do its job. If encoder-style structs fit your use-case, this code is amazingly succinct! If, on the other hand, you're e.g. deep-inspecting network packets like a firewall would do, and you don't want to decode the whole network packet to decide what to do with it because that's way too much overhead, then you've got to give up on encoder and write your own parser code. And it's ugly. Erlang/Elixir shines here.)
All that being said, you can write non-IO code (business logic requiring typed inputs/outputs) in Erlang/Elixir. It's less idiomatic, especially in Erlang, but it's not like it's clumsy. And it's certainly a better experience in Elixir (with struct types, for example.) But Erlang and Elixir still always kind of assume they're either being used as a "control plane" and that the data is flowing in some other way; or that they're the "boundary" between the (raw binary) outside world and some sort of business engine process or library written in native code.
The most idiomatic Erlang (or Elixir) you can write, takes data from the wire, figures out what it means, spawns a process that hands it off to a port program or NIF, receives a response, and then encodes that response and sends it back up the wire. When Erlang/Elixir is also doing the job of that inner business engine as well as being the glue, it's less streamlined, and you start having to think about things like using HiPE and function-inlining annotations to make your BEAM modules run faster (which doesn't really matter when those modules are just there to do IO and never compute anything intensive.)
> a lot of crashes of processes are due to fairly prosaic reasons like unmatched patterns
Fun consideration: crashing on unmatched patterns is how Erlang "gracefully" handles the case where you have a distributed system composed of multiple ERTS nodes running your application, and you're doing a rolling update so that some of them are on different versions of your application, and they're actively RPCing with one-another, but with different application-layer messages without guaranteed mutual understanding.
Try to figure out how you'd handle that scenario in a language where you're sending around marshalled typed messages that must be decoded by a matching local marshaller, rather than simple tagged tuples. Especially try to figure out how you'd handle partially parsing a message where you understand the outer struct's shape, but don't understand one of the optional inner structs.
In any other language, the only option there is to just keep around the old version of the decoder, such that the new release of the app can decode both the old and new data formats, and ensures that it won't generate new-style messages for old-style peers. Then you roll your update over the whole network, and then create a release which rips out all the graceful-upgrade code you just wrote. It's a whole production.
Erlang's distribution protocol, and its external term format, are designed to obviate these challenges, such that your code (which, remember, is idiomatically IO-boundary-layer code trying to decide what to do with packets) can still make decisions in the face of data you don't have 100% of the relevant schema mappings for. The let-it-crash philosophy, and the supervision tree, are how you idiomatically handle these messages that have successfully been delivered from your peers, and which are "speaking your language, but not your dialect." Used together, you never have to do the graceful-upgrade protocol-version-juggling foofaraw.
I'm building a very large complicated product in a very boring market that traditionally has old tech. Building it with Elixir helps me write less code with fewer bugs and test coverage. I love it.
It feels strange how I remember Elixir was a "what if" back when Meteor was in vogue, now I couldn't be happy at my job if not for Elixir. It's the best!
Jose's vision of developer UX first is really something else and it shines through when you work with Elixir.
Ive worker on Elixir over a year and it is great, but there are 0 devs and learning it is hard for newbies
Yes, it's functional, but it doesn't have a lot of complex concepts and is much easier to learn than , eg, Ocaml or Haskell. (at least in my opinion).
A good dev should probably be productive within a week.
While is true that the local concepts are quite simple and that the language is quite small, how all those ideas play together create a very complex environment rarely seen outside the BEAM/Akka environment.
I don't believe that good developers without prior experience in such systems can become productive in a week. They could be able to write code that compiles, but the week after that they are going to throw it all out and restart from scratch.
So.... pretty much like programming in the most popular languages, too. ;)
- Creating an actor system manually
- Wrangling an ExecutionContext
- Getting replies from actors in the form of `Future[SomeReply]`
- Making sure you don't do blocking operations in basically any of the concurrency tools.
Compared to doing all that, the Erlang/Elixir runtime can be explained to a newcomer quite simply–processes are cheap, start them up (e.g. Task, GenServer) and do whatever. The nuances can come in time.
There is even https://elixirschool.com/en/ for a crash course.
Two years later that has proved to be a fantastic choice and it has not hindered hiring in any way. In fact, most candidates are very excited about it.
It is? Why?
It's easy to learn Python because everyone has a blog with Python code on it.
There is also a wealth of tutorials, books, and interactive learning resources for Elixir that are up to date. I'm not really sure what you're referring to
Maybe this is the problem, then. I'm not the only one saying it, it was said in the article and other people are saying it here too. You can pretend a problem doesn't exist, but that doesn't make it true.
Which article?
There’s just a lot that’s different. Now, I’m looking for a job and I mostly know Elixir, and it’s making it difficult to find stuff when all you really know is how to write code in a (good, but) relatively unused language.
Elixir sits right now in a stage of its lifecycle where adoption is increasing, lots of developers are interested in picking it up but don't have much experience with it yet. Rails went through the same thing circa 2008-2010. I've introduced it now at several client companies which have gone on to readily adopt it more widely within the company, and positions on those teams that have adopted it are greatly coveted. Moreover, learning it well enough to be productive (though perhaps not proficient) takes only a week or two of coaching from a motivated developer.
I'm not sure what experience you've had with it that leads you to this conclusion, but your position is by no means definitive.
https://trends.google.nl/trends/explore?geo=NL&q=%2Fm%2F0pl0... You see that line that barely moves away from the 0-line? It's Elixir.
Elixir is a great language, but there are hardly any developers to be found and creating an application in Elixir is just a not a business decision. It's a fun decision. But telling that on a platform that mainly consists of developers gets you downvotes for sure :)
No, but Erlang is listed among the most-loved, most-wanted, and highest-paying technologies.
> there are hardly any developers to be found and creating an application in Elixir is just a not a business decision. It's a fun decision.
WhatsApp is a famous success for Erlang. Cisco is using it broadly as well. A large portion of the phone system runs on Erlang because of its technical advantages. Bleacher Report went from 150 servers to 5 by switching from Ruby to Elixir, and their whole content delivery model is enabled by using Elixir, since most of their stuff is personalized and caching doesn't really work.
Elixir is basically "friendlier Erlang"; all the advantages of the BEAM are there. Elixir and Erlang devs together attend conferences like Code BEAM. You could probably hire Erlang devs to do write Elixir, and if you had an Elixir app and Elixir died, you could switch to Erlang.
I think there are some good business reasons to use Elixir. Of course, I'm a dev who has spent significant time learning it, which makes me somewhat biased. But I did give it some thought before jumping in.
Lots of solid advice for how to introduce Elixir as a new language to an established team using a different language.
It would be hard for something like Nim, Ocaml, which are brilliant languages but lack the some support.
But Elixir is built on top of Erlang. Erlang! The whole Networking world depends on it on one way or another. It is literally some of the most important pieces of technology. You can replaces your OS with something else, you can replaces your lib with alternatives. But I am not aware of another ecosystem that is an alternative of Erlang and it is battle tested the hack of it.
And Elixir is built on top of Erlang / OTP.
Erlang is not popular in the FAANG companies, not used in their CDN devices and LBs, and not used by CDN vendors (e.g. Arista).
Mainly because the OTP does not solve most classes of problems you encounter at a large scale.
Now you can get errors like this:
________________________________________________________________________________
apps/activity_feed/lib/activity_feed/push_notification_listener.ex:10:callback_type_mismatch
Callback mismatch for @callback push/2 in ActivityFeed.ListenerBehaviour behaviour.
Expected type:
{:error, _} | {:ok, _}
Actual type:
:ok
________________________________________________________________________________
apps/activity_feed/lib/activity_feed/push_notification_listener.ex:24:pattern_match
The pattern
_ = {:error, _}
can never match the type
:error | :okThe lion's share of the work is now extracted into Erlex [1] which can be incorporated into other tools as needed.
I'm sure there are plenty of tricky traps that regulars have learned to work around and "just know how to use it without breaking it"; I have a suspicion a lot of unsatisfied people are those who come with different expectations of what the tool should do and how it should behave.
In these cases, they'll have a frustrating experience. I'd be interested to learn what it is so that we can make the whole thing better.
Ah good point. And rebar3 along with docs, and books are probably the first thing a newcomer will see when they explore Erlang the first time.
Languages like Go, Rust, and Haskell have to start from scratch. Getting to the point where they work at all is a huge achievement (though of course they do more than that).
Languages like Clojure run on an existing virtual machine, but want a very different programming model, so they have to invent a lot.
By contrast, Elixir runs on the BEAM and gets 99% of its features from Erlang: you still use processes, supervisors, pattern matching, etc.
So in a sense, the only reason for Elixir to exist is to provide a great developer experience for the BEAM. If it doesn't do that, you might as well use Erlang.
That said, there's one design choice that I simply never understood: the dynamic typing. Bear with me, I'm not trying to start another holy war here, I have an argument. Thing is, Elixir very seldomly uses its dynamic typing. Ruby has method_missing and monkey patching, JavaScript (finally) has Proxies, but Elixir has none of these. In fact, most Elixir code is particularly undynamic in nature.
All Elixir code that I've seen looks more like a non-OO version of typical Java or C# code. Notably, variables and seldomly change type. Function arguments are seldomly "a string or an array" like is common in JavaScript. Structured arguments are typically either used as a dictionary, or as a struct (the Elixir word for object, record, whatever), but not really ever "a little bit of both". Modules always have the same set of functions, the functions always have the same signatures.
Elixir replaces Ruby's monkey patching with macros. They feel a bit similar, but notably macros are entirely compile time. There's nothing dynamic about them.
I feel like if Elixir had unambitious static typing, more akin to C# or TypeScript than to Haskell, it'd have significantly better tool support, code would be much much easier to read and follow, and virtually all code that works now would still, well, Just Work except that you'd need to specify types on function arguments of course.
Of course, there's typespecs, but anyone who's ever tried to use them knows how abysmal the development experience is. Dialyzer error messages are impossible to figure out, often showing errors multiple functions down the stack from where the typing or programming error actually is. The syntax is weird, minimally documented and not integrated with the rest of the language. I'm impressed by how well most bigger libraries are typespec'ed, given how hard it is to get that right.
All that said, I'm well aware that even an "unambitious" type system would be hard to accomplish. Making a dynamically typed language surely must be easier than an ergonomic statically typed one - especially if the underlying VM does not depend on static types. Maybe the simple reason Elixir lacks a good static type system is that it's a lot of work.
There are some typed language on the BEAM but far from being reliable on prod.
Quite simply, noone knows how to efficiently type the actor model on the BEAM yet. It is that simple
To recap, one of the main Haskell designers tried to come up with a type system for Erlang but couldn't cover the inter-process communication, probably because a process which has another process' pid is allowed to send it literally any type of message.
In Elixir, you notice the dynamic typing mostly when you're defining callbacks that take MFA-style function specs, e.g. def handle_click(m, f, a) ...
Last point, you are right about Dialyzer error messages, but people are working on it right now. Look for messages in this discussion from _asummers.
Apparently Typed Akka is finally starting to arrive at a good design, after a number of iterations over the years, but seems like there is still lots of interesting research to be done in distributed type systems.
> In Elixir, you notice the dynamic typing mostly when you're defining callbacks that take MFA-style function specs, e.g. def handle_click(m, f, a)
I really noticed it in the `Enum` module - so many `any`s, on simple things like `map`. Kind of ridiculous. And protocols are not type directed, so you don't get the huge win of return type polymorphism. Kind of kills lots of useful patterns :/
BEAM's concurrency model can be helpful even at small scale. Eg, suppose you have a web app where requests to one endpoint are computationally expensive (by design, or only for certain input, or because there's a bug). If you use (eg) Rails with Unicorn, you have a smallish, fixed number of processes available to serve requests. If N is 10 and you get 10 requests to that endpoint, every other request to your site is now standing in line, waiting to be served.
By contrast, Cowboy (the web server a Phoenix app would use) will spawn a new, tiny BEAM process per incoming request, and each process gets to run a fixed number of instructions before it is paused by the scheduler and goes back to the end of the line. So even on a single-core system, a few expensive requests wouldn't affect typical requests noticeably, and the effect would be gradually increasing latency as you got larger numbers of expensive requests, not a sudden and dramatic spike.
Maybe this won't matter for your site, but it's something that you don't have to worry about.
That said, I suppose a Haskell designer and me have rather different expectations from a type system. I want great TypeScript/Java level tooling, catching stupid type-style bugs, and easier to understand code. Going for a fully covered, sound, type system sounds like quite a challenge indeed and I can imagine why it didn't work out.
I disagree that "The type system can't guarantee the types of what gets sent between processes" implies "a type system would be useless". Most of my code is not busy sending and receiving messages. It's just, well, regular boring single-threaded code that manipulates data, performs some side effects maybe, etc. I'd be perfectly happy with a type system that just makes me specify, at compile time, what message types a process is expecting to receive. Somebody sends something else? My problem, runtime error. That way, each process can be decently typechecked individually, the process type can just be "pid", and you can skip the entire "how do we type the actor model" problem. I'm sure I'm wrong here in some subtly detailed way, but I'm convinced that Elixir would fit a TypeScript-like type system like a glove if you just forget about typed messaging.
Does this guarantee "if it compiles, it works"? Nop. Does it allow for readable code, extremely powerful tooling and catching stupid errors? Sure thing.
For example if you wanted some function to work with both a string and list you could use pattern matching and "when".
Something like:
def myfunc(input) when is_binary(input) do: # ...
def myfunc(input) when is_list(input) do: # ...
Of course it's not the same and you lose a bunch of useful static analysis tools, but from a usability POV, you could skim function heads and quickly see what types are supported. To me, this is a big win compared to having a massive elseif or case statement buried into a single function where at call time you have no context on what's expected.
Thanks!
In particular since the final product is on-prem, the highly available, let-it-fail philosophy is appealing to reduce the number of issue tickets we're likely to have to deal with.
Don't take my strategy as gospel - I am on an open-ended, highly divergent branch of our company's product line and have been given a lot of product leeway by management that is.... "open-minded".
https://github.com/cogini/nerves_system_ec2
I'm excited to play with it in the future.
It works pretty well, and I get to ship artifacts rather than source code.
Also a simple Dockerfile to push to AWS
There's a PaaS called Gigalixir that has a lot of promise, but you'll be limited to one app at a time, and it must be re-deployed at least every 30 days or it gets shut down.
Also been using Heroku for some Elixir apps in production.
Our CI builds our elixir artifacts, updates them to GCS, then our minions pull the artifacts, untar them and move them into the right place. Aside from deploying go binaries, Elixir binaries have been the next easiest thing.
We have one (very low throughput) service that runs Elixir inside of a docker container, however, the rest just run BEAM on the VM directly, with no containerization. BEAM at higher load does not enjoy sharing cores with anyone. Most of our Elixir services sit at 80-100% of CPU on all cores. BEAM likes to spin for work, and is CPU topology aware. Two BEAM scheduler threads running on the same core is a recipe for disaster. Since we're fully utilizing our VMs, and shipping the tars is simple enough, Docker gave us too much operational overhead for it to be worth it.
We've also had our fair share of troubles running the BEAM VM on GCP at scale. I could go into this in more detail if anyone's curious.
Please do!
- CPU utilization can differ wildly on Haswell & Skylake. On Skylake processors our CPU utilization jump by 20-30% due to Skylake using more cycles to spin. Luckily, all of that CPU time was spent in spinning, and our actual "scheduler utilization" metric remained roughly the same (actually, on Skylake it was lower!).
- Default allocator settings can call malloc/mmap a lot, and is sensitive to latency on those calls. Under host memory bandwidth pressure, BEAM can grind to a halt. Tuning BEAM's allocator settings is imperative to avoid this. Namely, MHlmbcs, MHsbct, MHsmbcs and MMscs. This was especially noticeable after meltdown was patched.
- Live host migrations and BEAM sometimes are not friends. Two weeks ago, we discovered a defect in GCP live migrations that would cause a 80-90% performance degradation on one of our services during source-brownout migration phase.
GCP Support/Engineering has been excellent in helping us with these issues and taking our reports seriously.
I thought that GCP live migrations were completely transparent for the kernel and the processes running in the VM. I'd be happy to read a bit more about the defect that made BEAM unhappy.
Excessive allocations and memory bandwidth are two very different things. Often then don't overlap, because to max out memory bandwidth you have to write a fairly optimized program.
Also, are the allocations because of BEAM or is it because what you are running allocates a lot of memory?
We were able to simulate this failure condition synthetically by inducing memory bandwidth pressure on the guest VM.
We noticed that during certain periods of time, not caused by any workload we ran, the time spent doing malloc/mmap would 10-20x, but the # of calls would not.
I'm curious what tools you used to discover this
Docker allows me to run various Phoenix/Elixir apps with various versions of the language and framework since I don't have time to update all of them. My method also allows me to compile .tar.gz releases on my local machine since the lower-tier servers that I can afford don't always have enough memory to compile there.
Here's a tutorial I wrote on how I deploy my Phoenix apps if anyone is interested: https://gist.github.com/jswny/83e03537830b0d997924e8f1965d88....
Or will Elixir only be used for server-side programs, that don't require a graphical user interface?
I would like to program video games with Elixir. But the examples I see online are rather immature.
And it doesn't seem like anyone is interested in using Elixir to program desktop GUI programs.
That said, I've written a CLI in Elixir now and it's pretty good for that. Also the Elixir community if full of hackers so I wouldn't be surprised if somebody makes this comment obsolete very quickly.
https://elixirforum.com/t/3-elixirconf-2017-elixir-native-ui...
There are a few projects working on creating a purely elixir web dev experience for both front and back end, but no mature solution exists yet.
Wings3D created a 3d modeling program with application, including UI, written in erlang. http://www.wings3d.com/
I personally wouldn't choose Windows, but there are areas where that is a hard requirement.
And the reasoning is that one should instead be using `@callback` and `@macrocallback` which also are typespec attributes: https://hexdocs.pm/elixir/Module.html#module-typespec-attrib...
To you use typespecs and Dializer? (I haven't looked into those yet.)
My experience is that a good share of bugs is avoided thanks to immutability, but I still get nil-related issues now and again.
In Ruby, if I'm trying to find the source of a dynamically generated method in a dynamically generated class, that usually requires me to run add a debug statement, run the program, and see what's being passed. Even if I have a stacktrace, I often find it difficult to understand what the code is doing without running it.
I find Elixir code easier to read, and grepping my way through the code base is often enough. And to be fair, a debugger is often less than useful in macros involving AST & when dealing with a process that got a random message from who knows where.
What exactly makes you say that Rails is a clear win over Phoenix for CRUD stuff?
My biggest concern is payment processing libraries are waaaaay behind in Elixir. For example, Stripe doesn't have an official Elixir library (but they do for Ruby and Python), so that leaves you with a bunch of community driven packages that are all in various incomplete states.
Likewise PayPal has an official SDK for Ruby and Python but nothing for Elixir and it's very slim pickings on the community side of things here, to the point where I would feel uncomfortable using any of them.
But other than that, things have been good so far. A lot of common CRUD problems are solved, for example with pagination Rails has Kaminari and Elixir has scrivener which took like 2 minutes to set up. You'll find well maintained libraries for a lot of common web dev functionality. Plus, things like Turbolinks works great with Phoenix if you're into that.
Generally speaking you'll end up writing a little more code with Phoenix than with Rails, but I find it much easier to grasp what's happening in Phoenix because there's so much less magic (but not so little that you need to implement a million things yourself).
The language itself is good, the tooling is pretty excellent, but there are issues for things like deployment, ops, and frankly a general excitement about neat things instead of focusing on bread-and-butter stuff like, say, timezones and timestamp formatting (something coming in a couple of releases, allegedly, but which was always an obvious absence).
Using Heroku (not good practice and I wouldn't use in prod), it would be really quick.
That's really the only deal-breaker though. Docker for Heroku makes it really easy to ship artifacts rather than source which I think is important as well.
It was 2 years ago, I know the community tried to improve things, but it's still not there. I read some excerpt of the upcoming book "Phoenix in Action", and in chapter 1, the author himself mentions "deployment" as a drawback of using Phoenix since it's still painful.
Releases are not mainline yet (soon!) and are kind of off to the side in a semi-officially supported way (bitwalker is amazing!). There are guides for this, but not the same type of guides as would exist if it were in the core repos. Then you have hot code reloading, which exists, but is usually said to be shied away from generally, so there's even fewer guides about all the gotchas that come with those sorts of systems. Some people even run mix phx.server on their prod boxes!
I LOVE this language, and all these problems have solutions and workarounds, but one can't tell the story without being honest about the current release/deployment quirks. Things are definitely better than they were a few months ago, and much better than a few months before that. But they're not quite at the level they need to be for something that's supposed to be so fundamental to your application as its configuration and release. They'll get there.
Everybody wants this tooling for deploying appliances when frankly we are almost all just doing boring webshit.
Even as a newbie to the ecosystem it didn't take long to discover that time functionality was lacking in Elixir but the Timex library makes that a solved problem with little effort.
Looking forward to seeing what comes up in the standard library for that in the future.
A modern language should have first-class support for these things. :|