Elixir at Ramp
engineering.ramp.com
engineering.ramp.com
> My hot take about most dynamic languages is that they are a poor fit for startups who have intentions of being long-term businesses
I don't think that's fair to say. There's plenty of long term businesses running at scale using dynamic languages. That's employee and load scale too.
Dropbox is ~13 years old, has ~4 million lines of Python and operate at crazy scale. I ended up talking to one of their ex-engineers last month about how they build and deploy their set up at: https://runninginproduction.com/podcast/82-dropbox-gives-you...
Then there's Basecamp which is one of the oldest SAAS apps around using Rails and GitHub too which isn't quite as old but definitely falls into the long term business category.
The classic rebuttal to this is that the company needs the best chance of surviving today, and can fix it tomorrow.
There are problems with this approach, it's definitely harder to fix these problems later, but "at scale" is that really a problem? The big problem is getting to the scale, surviving long enough.
For lower level / systems programming, though, I think that's somewhat less true. Think about database plugins like timescale, or VPN solutions, etc - stuff that, if it breaks, somebody's having a really bad day. It's probably worth spending more time on robustness and then a statically typed compiled language can really help eliminate many classes of errors off the bat.
BEAM at least has some compelling features working in its favor that make it worth exploring operationally, but also it doesn't really operate at that very large scale. It's well-adapted for smaller scale systems.
I wonder what they switch to at WhatsApp when they become medium scale.
There are very real and well-known scalability limits of Erlang clusters and applications that run in them. This isn't someone talking shit, but instead just operating within the limitations imposed by reality. Message passing has overhead -- it's not free.
WhatsApp probably doesn't need nearly as much hardware as you think.
Well yes you would argue, you did. That doesn't make your argument true or important.
At the end of the day what you need is a statically linked binary. There are tools like cx_freeze and Nuitka that try and get you there but ultimately aren't worth the effort and give you other problems to solve as well. Or you could use cython, but again, more problems. You're simply better off using a language that reduces your (business) risks from the onset.
I'm sorry that that makes you feel bad, but that's facts and anyone working at this scale will tell you so. We pretty much only deal with Python where we have to (in the ML space). If a company is really stubborn about it though it's often better to just not take those jobs.
There are many reasons why I feel Erlang or Go or Rust shines brighter in many situations but my issue is that I not only need to get up to speed, build scrappy prototypes fast, but also hopefully, hire other engineers including juniors.
There are many ways one can build systems with multiple languages and take the benefit that each has to offer in specific cases. Python itself calls C in multiple libraries to speed up code.
Some of the safety related paradigms are really practical but to adopt a new language means a lot of risk for the startup too. Not every tech leader is so capable as to juggle both the delivery stress of features and building idiomatic software in a new language, I know I would struggle.
So I keep learning and I will wait for when specific cases demand me to use another language and use something like gRPC in between my existing codebase and the one in a new language. I do not (should not) have to move away from what already works well to adopt new tools or paradigms.
The problem could be with senior developers: if we had no experience with Elixir we probably would have resisted starting a new project (the main project of the company) with a new language. But this is the same with any language. Would I be confident to build a company on Python or Ruby or JS if I didn't know them well? Not at all. Would I start with Java? Not after all the years I have not used it.
Having said that, I would not recommend Elixir or Scala or anything similar for startups. The 3 main reasons:
1) It's a hiring problem.
There aren't enough qualified people who can fill these roles. Expect a lot of on-the-job training that's more challenging than you'd think since these languages/frameworks are different paradigms. Famously, the complaint I heard in Scala circles was "There are too many ways to shoot yourself in the foot." I worked at a well-known everyday consumer tech company where the payments platform went down for the better part of a day as people scrambled to figure out what went wrong. Noone had read the de-facto Scala bible which warned that catching `Throwable` could break your code.[1]
2) You really, really don't need to get things perfectly right from the get-go.
> My hot take about most dynamic languages is that they are a poor fit for startups who have intentions of being long-term businesses: you’ve created an environment that’s optimized for your founding engineers to build something quickly in the first 7 months, but the price is a set of recurring obstacles which your engineers will pay down over the next 7 years.
With rare exceptions, most companies throw away or rewrite their initial infra once they hit scaling issues. Having these scaling issues means you've likely hit product-market fit, secured more funding, and hired a badass team to solve said scaling issues.
3) The ecosystem is smaller.
Because the language/framework is deeper, it requires more investment to learn. In turn, fewer production-level libraries get built on top of it. That's when you end up needing to build a lot of things yourself. At which point - you've slowed down _even more_ vs. DynamicLang where you can run `dynamic-package add audit-trail` or something and get the 80/20 solution in 5 minutes.
[1] http://www.tzavellas.com/techblog/2010/09/20/catching-throwa...
It's not entirely clear whether your experience is only with Scala or whether you've used both. If it's the former then I'd suggest that others try Elixir for themselves rather than take a recommendation based on experience with another language.
The points I raised earlier in the context of choosing a language for building a company apply to all of those. I don't see why Elixir is exempt - do you mind explaining?
It runs in a different VM, it's a different paradigm (or mash-up of paradigms which, in turn, is the source of one of your contention points when learning/ramping hires), it's a statically typed language so requires usually more upfront thinking and a better understanding of domain modelling (getting it right from the start), it's notorious for being complex (don't know if it's warranted or not, but it's a normal complaint even from people who seem to like the language).
Elixir/Erlang does need a bit of honest study to hone (this is the same with everything though, JS for instance is quite complex when you take a step back and look at what you need to understand to write decent maintainable code), but the payoff is much higher because it doesn't change with every tide. Code you wrote 4 years ago, if idiomatic, will be idiomatic today, and I'll go off on a limb here and say it will be in 4 years time too.
(btw didn't downvote you)
I'm not trying to derail into discussing how Scala is a shade better or worse than Elixir in some aspect - the main point I'm trying to drive home is that they're both in an entirely different class compared to Ruby/Node/Python.
Your points don't necessarily apply super well.
Point 1 is about hiring and teaching. There's going to be a different candidate pool of people interested in learning Elixir vs Scala, and while I don't know Scala my understanding (heh, see what I'm doing here?) is that it's a pretty large, complex language. So your experience onboarding and training new devs doesn't really apply. In particular your comment about "many ways to shoot yourself in the foot" doesn't resonate with me as an Elixir developer at all. It's a small, simple language, and there's really only one way to do most things.
Point 3 also doesn't necessarily apply. Yes, Elixir is smaller, but for all I know the libraries are better than in Scala or vice versa. I haven't run into too many missing libraries, and the ones we use are generally "production quality", though I don't really know what that means. They have tests, documentation, etc. But again, maybe that's a language culture thing. Dead simple testing, a great doc generation system, and compiler help, combined with general practices prevalent in the community, mean libraries are pretty robust and easy to use. Furthermore, erlang interop is super easy and done a lot. I know Scala has the JVM in principle, but I don't know to what extent you can lean on those libraries, to what extent it's typically done culturally, and to how smoothly they work with the rest of the code.
In short, I think "my Elixir was Scala" is an interesting take and worth sharing. But I think you're leaning too hard on that relationship, without clearly demonstrating that you know Elixir itself, in order to make strong recommendations against using Elixir.
That said, I agree with you if you are planning to be a unicorn and scale to 1000+ engineers then you definitely should go with the well-worn path just because it's the only way to satisfy your insatiable hiring funnel.
On the other hand, if your aspirations are to focus on a simple but great product and keep the team tight rather than chase growth at all costs, Elixir/Erlang could be a super power to scale a cohesive product to massive scale with a very small team (eg. WhatsApp).
What about for not-startups (whatever those are)? I'm attempting to use Elixir for a new component of a kind of startup like product in a 50 year old company. So kinda startupie, kind very not. But more generally, I'm curious what changes with the "for startups" qualifier?
If you have a guaranteed internal market so you need to launch with scale, then that’s more important
So...not Elixir.
I think your first and third bullets are correct, but not entirely relevant.
That is, yes, it's hard to find people that know Elixir already, but if you can take a dev and get them competent in two weeks does it matter? Your experience with Scala != experience with Elixir.
Likewise, when it comes to ecosystem - maybe it matters, maybe it doesn't. In Scala you can leverage the JVM; I feel like the ecosystem isn't really a problem then. In Elixir, you have a lot of things available both natively in Elixir, and in Erlang. Less than the JVM, for sure, but unless your problem domain includes something that doesn't exist for Elixir, it doesn't matter.
For your second...that seems more how you write code than your language choice. And a claim that Erlang/Elixir's approach to 'let it fail' means you are spending too much time writing robust code seems...false; if anything it's less than having to catch and think about Java's checked exceptions, let alone if you ever ask "wait, what happens if we go off the happy path here?"
There are also upsides that you have to weigh up when choosing a language. I don’t find your arguments particularly persuasive.
I'd not recommend Scala for other reasons: being multi paradigm makes it hard to learn, read, re-read and collaborate on. Kotlin would be a more sensible choice on the JVM imho (no implicit nulls, fully OO at the core with FP where it makes sense in that context, sum-types, exhaustiveness checks, no exception/annotation frenzy like Java).
And Erlang misses types. Gleam[1] looks cool, and seems to be picking up steam.
Rust is an amazing choice if you need that kind of low-level control over performance in all it's dimensions (except built times).
There is a great danger in this. Hence I prefer a language that is smaller and helps me to avoid cruft being checked in to version control. Linters are a solution, I've seen it work in JS/TS project. Not sure if C++/Scala have linters up to the feature level of JS/TS linters.
Compared to what? Java? Kotlin?
And only initial development? Or also maintenance?
And why potentially and not certainly? What can make that difference?
“Different” than what? Its not the 1990s where everyone learns only C++/Java-style static class-based OOP anymore.
> I worked at a well-known everyday consumer tech company where the payments platform went down for the better part of a day as people scrambled to figure out what went wrong. Noone had read the de-facto Scala bible which warned that catching `Throwable` could break your code.
That’s a pretty horrid example for “Scala is a hiring problem because its a different paradigm”; lesson one for exception handling in every language I’ve used that has exceptions is never catch everything, only catch specific exceptions that you know what to do with. And in virtually every language, catch-everything is a minefield waiting to blow up in your face.
I would recommend Elixir for a startup.
I made a similar comment on a previous article about Elixir: https://news.ycombinator.com/item?id=27194517
I also didn't want to make things "just work" and refactor later, because I think that's just overall terrible advice. The cost of going back and rewriting your architecture is enormous. Might as well use tools you feel confident can scale with your startup than hack something together and have a complete mess on your hands.
Good on them for finding Elixir useful, but none of the benefits they talk about seem very compelling for me. Most of them are available as table stakes in other languages (rerunning failed tests as an USP of mix test, really?), or very situational/subjective benefits. I find the import system in Elixir to be very opaque, making it hard to know where a function is coming from after it's been aliased and use:d. I subjectively find explicit import statements much clearer when debugging code.
The final drops for me are the missing explicit return statement, and the extraordinarily bad tracebacks on error. If I could I would excise Elixir from our company.
I will also add that goroutines (and actors and other first-class concurrency constructs) built into a language doesn't really help with web startups that aren't targeting large scale consumer traffic or systems level apps, and even if it did, multiprocessing GIL languages has never been an issue. It's a cool feature for sure, but language fundamentalism is just a way for insecure devs to feel cooler.
In fact, I think you should go pitch this idea to all OSes developers, nobody is doing more than one thing at any given point in time, so clearly multitasking is unnecessary cruft, pthreads, selects, etc just cater to insecure devs that want to feel cooler and pretend they're not in the stone age of computing anymore.
Tracebacks are not so bad but they tend to be useless when the cause of an error is in a different process than the one that failed. I know that my GenServer failed at line x with data y, z but I don't know what sent that x to the GenServer. I have no idea if there is a way around this except extensive logging.
Ah, you were using casts instead of calls. Use calls as much as possible, even if you always send back :ok. This needs to be documented as part of best practices.