As someone with primarily a Ruby / Rails background, I’m choosing a language for a new API project and considering Elixir. I’m interested in hearing some counterpoints to Elixir, especially in how a smaller ecosystem Of 3rd party libraries slowed down development.
Not Elixir, but a cautionary tale from our Erlang project. ~8 years ago a our IoT backend was written in Erlang, this was the early days of IoT, so sure it made sense as a technology, could scale well, handle all the symmetric sessions, etc. It was a good tool for the job and in theory could scale well.
But, there's always a but. We're a Python shop on the backend and C on embedded devices. Engineers move on, there some lean times, and after a few years there was no one left who could support an Erlang system. So we hired for the position, and it's key infrastructure, but not really a full time project, so we hired a Python+Erland person.
But! Now they're on call 24/7 since the regular on call roster are python people and when the Erlang service goes wrong, it's call the one engineer. So, do you hire 2 or 3 people to support the service? No, you design it out. At the time is was 2018, and IoT services were a plenty, so we could use an existing service and python.
Another way of looking at it, let's say it's a critical service and you need at least 3 people to support it/be on call. If each engineer costs $200k/year, that's $600k/year. Does this new language save you that much over using a more generic and widly known language in the org?
* Healthcare startup that basically outsourced its entire infra/security team from a consulting company. The consultants rewrote everything and took over engineering leadership, basically acting as gatekeepers for anything SRE-related.
* Worked at an older DNS company that hired a bunch of consultants to help engineers migrate their web framework from Play to Spring. Said engineers used Play because it was shiny but didn't have the Scala expertise to keep maintaining it, so they moved to pure Java.
You don't need someone "learning" Erlang. You need someone who works with it.
If you're an up-and-coming Golang developer, how happy are you going to be when your company takes you off of a Golang project and makes you the new Erlang person?
Now how happy are you going to be when you're informed that you're learning Erlang to maintain an old, flakey codebase written by 2 guys who left the company and are now unreachable?
Oh, and you're going to need to be on call for this because no one else knows how to fix it.
If I'm a python developer, I'm now spending less time on my core competency, giving other python developers an edge. Seems like you're actually narrowing my career, unless I wish to apply for jobs seeking mediocre Erlang programmers...
so this argues against the theory that learning other languages and other programming paradigms makes you a better programmer in the languages you already know.
This might be true based on your specific situation (relatively new to the language, but already know some other varied languages); I don't know.
The workers are settled and and don't want change. And there's nothing wrong with clocking in and out and not giving a shit about your "career". But there is something wrong with the managers falling for this.
There are lots of senior roles that require oversight of multiple projects that are working with different tech stacks. Some companies specialise in a single language. But lots of companies have say 2 or 3 backend languages (say C++, Go, and Python), plus a website in JavaScript plus mobile apps in Java/Kotlin and Swift/Objective-C. Being able to dive into all of those makes you incredibly valuable to this kind of company.
For Java it might be enterprise Java, it might be Android, in which case you need familiarity with those frameworks to be productive. For Swift it'll be iOS.
No one's "diving into" the codebase for an iOS app starting from "I mostly do backend web app development in Golang".
Are you an "expert" in all those things? I doubt it. In fact the worst possible thing you can generally write anywhere is that you're an expert in C++ which at this point is oh-so-many subdialects.
I mean, I've certainly been expected to at previous jobs (my background being backend and frontend web). It was a steep learning curve, but in retrospect it was also an excellent learning experience.
> Are you an "expert" in all those things? I doubt it.
No, but give me time. I'd consider myself an expert at frontend JavaScript/TypeScript and backend JavaScript/PHP. I have a decent amount of experience (enough to get a job an X developer, not enough to call myself an expert) in a number of other technologies (e.g. Python, Rust). And at my current day job, I'm responsible for a React-Native project through which I'm getting exposed to a lot of the quirks of the Android and iOS platforms.
Go I've not used. But Go's a relatively small ecosystem to learn. I don't put Go on my CV, but I'd be confident applying to a Go job. C++ I've done a little of, but I'm absolutely not an expert. Although Rust experience means that I understand most of the low-level concepts and best-practises that aren't C++-specific. Still, I absolutely don't advertise that on my CV either. One day though maybe. I'd like to do some embedded stuff at some point.
> In fact the worst possible thing you can generally write anywhere is that you're an expert in C++ which at this point is oh-so-many subdialects.
C++ is a bit of a special case there though. Because
1. C++ is huge: it takes a lot longer to master C++ than it does other technologies.
2. C++ is wildly unsafe. Therefore the consequences of not knowing what you are doing with C++ are much greater. You'll likely end up with a crashy program riddled with security holes.
In contrast, I wrote a production Rust project with no prior experience, and all the things that would have been crashes or security issues in C++ were compile errors. So the only consequence of my lack of experience was that it took longer to complete the project.
So that everyone knows you're full of shit? It takes years in a technology to become an expert at it. There's no replacement for the experience of taking multiple projects from inception to production to scale. That's how you learn about the inscrutable errors, fiddly configuration issues, and pitfalls that make you an expert at something.
The OP said "I'm a python programmer", and discounted the value of learning something new like elixir because it would undermine this due to the opportunity cost, which implies they may be falling into the trap I mentioned. All other things being equal, someone who is reasonably competent at both Python and Elixir is going to probably solve many problems in a better way than someone who didn't have a broader perspective. (In part because learning Elixir stretches well beyond learning a programming language, it is an introduction to an entire distinct continent of distributed systems knowledge and technology - OTP, BEAM, etc.)
Other comments say an engineer is supposed to solve issues with the tools given to them. I can solve issues in other languages, I just don't want to. How does that make me a bad engineer?
Second, because you are also likely to favor recruitment of other devs based on this cultural choice not on their merit, eventually creating a clique.
If I run a Java shop I want a clique of Java devs, not Golang or Rust devs.
A "multi cultural", Jack-of-all-trades kind of team, that have a broader knowledge of many tools and techniques but less in depth familiarity with any of them, is more likely to use less features from the languages and tools that they have picked for a few of their strong points only, and less likely to indulge themselves with a level of sophistication that benefits their ego (and job security) more than the project.
> If I run a Jave shop
So I know a Python shop must be a business that sells reptiles, but what does a Java shop sell?
>So I know a Python shop must be a business that sells reptiles, but what does a Java shop sell?
Coffee obviously.
I'm not a <language> programmer. I'm a programmer period.
If your product-critical cloud backend depends on people being knowledgeable in a specific programming language, you don't make it a 1/3 time side project for multiple engineers.
Also, if you start giving people "1/3 time" responsibilities, you're one step away from "everything is top priority" territory.
The point is: If you're going to write critical pieces of your infrastructure in a specific language, it's a requirement to have at least one person full-time dedicated to that piece of infrastructure.
When things go wrong at scale, you can't depend on a group of people for whom this is a part-time side project.
You'll have to trust me that we didn't just throw our hands up in the air when dealing with the Erlang situation. We tried a lot of the suggestions here and more.
Erlang is one of those languages that, for whatever reason, lulls engineers into a false expectation that they can pick it up over a few weekends and start knocking out IoT-scale solutions serving 6- and 7-figure connection counts. Speaking from direct experience, that's not only untrue, but it's a dangerous misconception that leads to a lot of pitfalls.
Don't get me wrong, Erlang is great if your problem set matches up neatly to core OTP functionality and well-defined patterns, but you don't have to stray far until you start finding difficult pitfalls that aren't easily addressed by engineers learning Erlang on the fly.
> Also, if you start giving people "1/3 time" responsibilities, you're one step away from "everything is top priority" territory.
I like to maintain and improve legacy systems and usually happy to do it. When I got a separate side-project I had constantly fight with my team why I should work on project X instead of contributing to our core project. If it was my whole team responsibility they were happy that somebody does it. Also, they wanted to invest enough time so it won't break.
In my mind good developers are not <language> developer, they are engineers that solve problems with the right tools.
Given infinite time and no deadlines, sure.
But in the real, you can't expect your engineers to learn entire programming languages and associated ecosystems on the fly as they debug issues in production.
If your Erlang-based cloud backend starts going down, you can't afford to wait around while engineers teach themselves Erlang so they can begin to even debug the problem.
The point is that if you write critical infrastructure in a specific language/framework/ecosystem, you need to have people proficient in that ecosystem who are ready to go. When dealing with production systems at scale, it would be downright negligent to assume you could simply learn the language and framework on the fly if any problems come up.
Maybe the emphasis should be more on the disappeal of your role being changed to maintain an old, flakey codebase than on the language.
Side note: Pagerduty comes with many hilarious sounds, including a very realistic meow. Thus we practice meow driven development: I don't want my phone to meow desperately.
Erlang / Elixir are languages that are easy to begin learning, but difficult to master. In our case, the team was able to get the basics up and running quickly, but the service became increasingly difficult to manage as our product requirements expanded beyond the strengths of the Erlang ecosystem's core competencies.
In our case, the original team of developers was excited to write a greenfield service in Erlang, but much less excited to maintain and scale it long-term. Some of them used their experience to speak at Erlang conferences, which led to them being recruited away by companies who were desperate for Erlang developers to maintain their legacy code after the core team left. Ironic.
At first we tried to train existing engineers on Erlang. Doesn't work. Not because Erlang is bad, but because people don't want their companies changing the direction of their career development just to support someone else's legacy project that requires an on-call team to support.
So your two options are: Hire at least 2, preferably 3 Erlang engineers for the reasons discussed above. Harder than it sounds, because AFAICT there are a lot of companies with 2010-era Erlang backends that made sense at the time but now require dedicated maintenance engineers.
Or you can rewrite it in a more common language, which is the route we chose. Rewrites are a difficult decision for well-known reasons, but ultimately it was the correct choice. Being able to drop 10+ engineers into a project as needed is a huge accelerator compared to funneling 100% of your work through the 2 Erlang gurus in the corner who maintain a codebase that no one else wants to touch.
Hiring is nearly impossible, come support our legacy system, and what ever you do, don't introduce Erlang into any other part of the company. Why? We feel burnt on this one project, and god forbid we get any deeper than we are now. What this isn't appealing?
I'm just beginning to get interested in the Erlang ecosystem. Could you please add more color to this and give some examples where Erlang shines? IIRC, Whatsapp backend used to be on Erlang.
You can hot swap all languages.
"All languages" provide you the ability to run arbitrary code, and thus theoretically support hot-swapping, but if you try to put it into practice you'll soon run into real obstacles you hadn't considered that either bite you or at the least make your job a lot harder. Erlang provides those practical solutions that make hot-swapping easy (in other words, practically possible).
TL;DR: On the question of buy vs. build when it comes to hot-swapping abilities, Erlang falls on the left while "all languages" start at the right.
Hot swapping Erlang in production is rarely done, and if it is done, it mostly done during debugging, not serving production traffic.
I invite you to do a straw-poll on the erlang mailing list.
I'm going to have to call your bluff and ask you to substantiate your claim. Please show where Python provides, as a part of the language and/or the standard library, the equivalents of these Erlang built-ins:
1. Module versioning (via vsn)
2. The ability to run multiple versions of the same module at the same time, transparently and for as long as needed.
3. The versioning (via Module vsn), tracking (via handle_call/3 and derivatives), and migrating (via code_change/3) of State.
4. The dependency tracking and ordering between code updates of multiple modules (via release_handler)
These are the native features that make hot-swapping in production not only feasible, but also powerful and easy. Of course, these are built on top of foundational primitives that are possible in any language, sure, but not all languages (including Python) have these natively available.
----
> Hot swapping Erlang in production is rarely done, and if it is done, it mostly done during debugging, not serving production traffic.
> I invite you to do a straw-poll on the erlang mailing list.
I'm sorry, but this point you're introducing now is unrelated to the discussion we're having, so I'll refuse to respond to this in any way except to remind you of the point we _are_ discussing:
1. adrr's claim: " Erlang was developed for telecommunication systems where you need performance and the ability to hot swap code." https://news.ycombinator.com/item?id=23286646
2. To which you responded: "Erlang's hotswap and Python's hotswap are of comparable power, ie, both are crappy. Being able to do something isn't the same as doing it well.
You can hot swap all languages." https://news.ycombinator.com/item?id=23287624
Used to be? I know WhatsApp switched from FreeBSD to Linux and swapped hundreds bare metal servers to thousands of VMs. I dont think WhatsApp ever swapped / rewrote their system in another language under Facebook.
For example good static typing is not just about adding type declarations but about taking advantage of the type system to the right degree. (Like don't ever overdo it, it's killing productivity most times).
But what degree falls into the productivity boosting spot and from which point on it will hinder productivity depends a lot on the team.
The problem often starts when the team changes and e.g. a pure scala/rust/C++/Ocml team get people cross entering from languages without any usefull static typing.
What had been reasonable and useful might now be a hindrance to the team!
I ran into a situation where this caused massive delay. (Well through the actual root cause was miscommunications, people not complaining about problems and instead trying to find ways to fix them behind the back of other people force pushing nice looking simple solutions which just happen to fail many edge cases, killed the write, refactor, test cycle and increase maintenance cost longtime :( )
So well 1st never overuse type systems. 2nd communicate problems, especially if you have problems understanding anything even if it just takes longer then you think it should.
What I mean is code which not only uses types and takes advantage of the type system but overdoes it. In many languages with a powerful type system you can encode much more then just basic types and this is useful to reduce possible error cases. But if you overdo it the type system part becomes too complex, type error messages too unreadable and it hinders productivity and on-boarding.
I'm a big fan of static typing and believe that more encoding more constraints in the type system is _generally_ good. But if the resulting types become to complex it often not worth it.
With simpler, more gradual designs like what are in Typescript or Haxe or even static, native-code languages like D, you have more escape hatches available, so you can iterate on the specification a few times and get code running quickly but also aim to make it tighter over time.
If you have $200k "python engineers" on the payroll who wouldn't jump at the opportunity to do some additional Erlang, maybe it's time to reconsider your hiring practices and that is the real cautionary tale.
I have made mistakes like that (introducing monads/future and making the team learn guava) and I still regret it. Its not to say futures are useless - there were a lot of async logic in the codebase and futures solve a real problem, but its not clear if the benefits outweigh the cost.
Back in the day people would balk at hiring Python programmers saying, "there are so many more Java programmers", and I used to say, "Why would you hire a Java programmer who was unwilling or unable to learn Python?"
Same logic applies here: Why would you hire a Python programmer who was unwilling or unable to learn Erlang? (Especially if you're going to pay them to do it!)
If you can't switch languages you're a kind of technician not a programmer.
FWIW, having done it professionally for ~15 years I'll never write another large project in Python again. Erlang's runtime is so much better than Python's it's just ridiculous. I feel stupid for not realizing this sooner.
I have met lots of engineers who are smack dab in the middle of the flock when it comes to tech stacks, or even _behind_, but they were more than capable. Hire for what you need. But what you think you need and what you actually need is probably not know to you.
I've been a happy Python programmer for over fifteen years, so this comes from a place of love and respect.
Beyond roughly 150k LoC you start to run into difficulties keeping everything straight over time, especially if you use a lot of indirection or other magic. The aspects of the language that help you go fast when you're prototyping (is "RAD" still a thing? Rapid Application Development) start to trip you up when the codebase matures. There are a lot of fancy footguns in Python that catch you if you stop dancing.
Tooling is getting a little better (MyPy, et. al.) but at the same time the overhead of adding explicit type information erodes the advantages Python has over, say, Java or Haskell.
There was at least 1 really large error that slipped into production because we were mocking an untyped service incorrectly (a mistyped property name), and the test passed, but prod exploded. It was embarrassing, and I certainly would have caught it with more mypy discipline, but overall the pros outweighed the cons.
At big companies that use mypy, however, this is problematic. If teams only define types for the straightforward parts of the system, then the types become less useful, especially as the company scales. From what I’ve seen, individual teams use an all-or-nothing approach with MyPy, so it’s unusual for an individual dev to sparsely type anything.
The moment we can get rid of it, you bet we will. But it's much easier tearing it down when you understand it enough to discern designs driven by the limitations of the tech vs requirements for the product. Black box ground up replacements are hard and expensive compared to ports. Also, thankfully it's Python and nothing something more difficult/complex.
At the end of the day, it's a job, and there's an expectation of professionalism to do what's necessary. A lot of people on HN like treating tech jobs like a paid hobby which I think is detrimental to career development.
I get being interested in Erlang if you're doing Python. But interested in doing Python as a Java dev? Probably not.
And no fat-binaries. Nor self executing binaries. It's just as old and having em basically just boils down to enforcing conventions. But nope, nothing. There have been attempts by some such as shiv, but really... None are even remotely as usable as maven is.
And no, it's not the same. Not even remotely. The only thing that helps with are quick and easy deployments.
The thing which makes maven special is the ease of producing artefacts for deployments, not the final step of starting it up on the server.
Your example of Java to Python doesn't represent the gravity of switching from Java to say Erlang or Haskell. You assume someone says no because they are unwilling or unable. We are unwilling, sometimes, because of the opportunity cost we have to pay for your fuck up.
Why would learning a new tool narrow rather than widen your opportunities?
It's not like you forgot all you know about Java.
Languages and specially the tool chain and the way of doing things around them evolve. If you spend like 5 years doing something else and come back to it, you'll find things have changed a lot. You may argue that it's easy to relearn this. But from a job perspective, the guy who has been using the exact same tech stack in the recent years will often be a better bet than a guy who wrote Java in version 1. This is very true for C#. Syntax, design principles, tools, method of deployments all have changed rapidly. Professionally, you want to be as up to date as possible in the area you worked in or you get rusty. You can learn new things on your time for their own utility. Flushing down 40-60 hours a week on a useless piece of knowledge is bad all round to me.
That is only if it's actually useless. I haven't programmed in haskell for many years, yet I am very glad I did. It was hard at first, but it made me think about code, data and transformation from a different perspective, expanding the repertoire of paradigms, regardless of which language I code in.
YMMV, but personally, I'd much rather hire or work with someone with 3y java + 1y erlang experience, than just 5y java experience. In general a polyglot rather than monoglot, especially if it's a different coding paradigm. Nevertheless, I may not actually want them coding random bits of work in erlang or whatever random language.
I've almost exclusively worked on the usual procedural /OO languages professionally (java /c#)..I like to experiment and I've picked up a bunch of languages along the way (perl, python, coffeescript, f#, nim etc).. And its usually because I want to see what's different and spend enough time to do something useful (but not large) with it. Also, it helps you become a better programmer IMO. otoh, if I'm hired for my skills but then have to work on js( don't actually like the js eco system) or haskell (I gave up after trying and I doubt I'd become very good at it anytime soon if I were forced into it) I'd be not happy either (and IMO, with valid cause that doesn't reflect on me or my abilities)
They can put only "Python" on their résumés, right?
For example, I wouldn't want PHP or Fortran on my resume, and, consistently with that, I avoid them. I don't have an opportunity to actively avoid them, but that defense mechanism would leap into action, if called upon.
However, I know a thing or two about these; I'm not simply applying "meme-based reasoning".
Would you want to pass on Lisp because of the ramblings of a rookie web developer with no Lisp experience? That's what you might do if you believe the "Lisp Curse".
Ah, sure they might be willing, but as a lead I'm not willing to invest in it.
1. I'm not sure I want someone who's got 4 weeks of experience in a language working on system critical infrastructure. Pick your time, does it take 6mo?
2. They have existing work, learning and working on an entire new language _AND_ new project is a huge task. Especially for non-trivial projects.
We could, but in our case it was a better engineering decision was to retire the old system and move on.
This is a great question to ask. It’s worth extending beyond language to frameworks and libraries as well.
It’s important to optimize for the development experience across all the apps in a team. Especially once the excitement of using a new language is gone and there are bugs to fix.
I‘ve rewritten a few projects in a common language for the org. Each time the biggest benefit was how straightforward it became to work on. Problems you’ve found somewhere else become easy to spot (and search for). Writing another test is the same as the last project you had open. Anyone in the org can contribute with a minimal learning curve. This can be the difference in getting bugs fixed in a day or a month.
Key phrase right there. Because it doesn’t matter what language a system is written in: if management doesn’t bother to hire replacements until after everyone who understands that system and why it works the way it does has already left, of course it’s going to crater shortly thereafter.
That’s not a language problem, that’s a business continuity problem. And it’s lamentably common as dirt.
Any monkey can learn to churn code. Learning the business domain, what its problems are, and how to solve them; that’s the bit that actually matters.
PEBKAC, at every level.
Then consider how hard it would be to get more people that have used such language/ecosystem in anger.
In a large company with lots of services in many languages, avoid putting business logic in your Elixir code. Business logic adds coupling to related code in other systems that often all need to change together, or get changed in a hurry. Either way, the engineer trying to do the change probably won’t know Elixir. Likely they were appointed to make the change by a PM who doesn’t know or care much about which systems were built in which languages and who knows those languages. And that engineer won’t know Elixir, and they’ll complain about being unable to work on your Elixir system, and it will give Elixir a bad reputation at your company in the eyes of managers and PMs. It won’t matter much if your system has high uptime or low tail latency or whatever.
The solution to this is to only use Elixir for building infrastructure. Nobody cares much what language PostgreSQL or RabbitMQ or Kafka are written in. You don’t modify the source code for those things typically, you just use them through some kind of API. Similarly if your company needs a notification delivery system, you can build that in Elixir. You just have to do it in such a way that nobody will ever be blocked trying to launch a new kind of notification because they need to learn Elixir first. Make it fully accessible and extensible through its API so they never need to know what language you used.
The first project was an API that was intended to serve as a middleman between a few legacy services. Basically the company that hired us was building a new JSON API but didn't want to rewrite all their old code and our job was to consume the output from their ugly legacy APIs and produce nice JSON.
Elixir/Phoenix worked for the first service, but after that we ran into problem after problem. One of the legacy services used HMAC authentication and Phoenix (at the time?) didn't really support that. We were eventually able to hack around it without too much pain. Another service used XML and we had a lot of trouble finding XML libraries. There were some but nothing remotely as nice as Nokogiri. While all of this was happening, adding to our frustrations, a new version of Phoenix was released with a lot of changes. The only documentation for how to migrate an existing app was a typo laden GitHub Gist by the author of Phoenix. The final straw was when we got a call from the client about another service we had to integrate. This one used JSON but the order of the keys was important (please for the love of whatever god you believe in never do this). When we had to decide whether to write a custom JSON parser or move to Ruby, we rewrote the thing in Rails.
The second project was already a few thousand lines of code when we started working on it. It was a dumpster fire of a project but the devs who built it praised Elixir to the skies and said it was way better than their old Rails version. So take that for what it's worth.
We've decided not to do any new projects in Elixir for now but we're leaving it open for the future. It is a nice language to work with and, as you can see from the other comments, there are some projects where it shines.
I have two suggestions for you:
1. Talk to people in real life about Elixir/Phoenix. Most other devs I talk to in person at other agencies that tried Elixir have (surprisingly) neutral or slightly negative feelings about Elixir/Phoenix. Some percentage of people on Hacker News absolutely love Elixir (and often hate Ruby/Rails) and they usually flood the comment section of any Elixir article.
2. If you don't fully know the problem domain, use something with good library support. I don't know what your project is, but if you control the front and back ends or if you know exactly what features you'll need, there will probably be no problems choosing Elixir. Plus, it's always fun to learn something new. If your project is "some kind of API that needs to do X and maybe some other things we don't know yet" I would stick with Rails (or Python/Java/something with a lot of libraries). The worst place to be is not being able to do something simple because of a lack of library support and having to explain to your client/manager/coworker that it's because you read on Hacker News that writing "a |> b" instead of "a.b" is the future.
I think this is completely fair. As my friend says, "I don't know what library I'll need for every project, but I know where will be one for Python."
These days there's a multitude of both XML and JSON libs for Elixir.
> Most other devs I talk to in person at other agencies that tried Elixir have (surprisingly) neutral or slightly negative feelings about Elixir/Phoenix. Some percentage of people on Hacker News absolutely love Elixir (and often hate Ruby/Rails) and they usually flood the comment section of any Elixir article.
This mirrors our experience, albeit with Erlang.
Every time I try to talk about negative experiences with Erlang online, people come out of the woodwork to pick apart my stories and assume we were just incompetent. Erlang/Elixir have an almost mythical reputation in online communities. A lot of people have completed some basic Erlang or Elixir tutorials and assume they're just a few steps away from running a massive WhatsApp-style backend, but it's not that simple.
If a company wants to go all-in on Erlang/Elixir and maintain a staff of multiple Erlang/Elixir devs for the lifetime of the service, that's one thing. However, decisions to write core features in these less popular languages should not be taken lightly.
My boss really wanted to use Elixir. Our CTO said he could only if he could convince 3 senior engineers and one ops person to agree to work on and support it. So he flew a whole team of us to ElixirConf. He organized a hackathon team around it. We rewrote some non-critical some services in it. All on company time and money.
At the end, none of the engineers could sign off on using it professionally at the company. Elixir is a cool language and it was a fun learning experience, but it's such a shoddy abstraction over Erlang and that's not a pleasant language. The operational complexity is really high, and it didn't solve our problems better than our current core languages did. The ecosystem was also really lacking in unavoidable ways for us, which meant we spent more time developing libraries that would've existed in any other ecosystem than actually writing application logic.
My big problem is with companies that don't do any exploratory work like that. Write a prototype in new language/framework with the explicit idea enforced in project management that it's going to be a throwaway. Do a study through hackathon etc like your team did.
Don't just go on meme-driven development, which covers "CTO read a fancy shmancy content marketing piece in colourful magazine while on the flight"
1. Elixir is still a pretty leaky abstraction over Erlang. In my experience it's not enough to learn just Elixir, I regularly had to dive into Erlang library source code to debug or answer questions. This somewhat negates the benefit of a small, stripped down syntax when you often have to learn another one in conjunction.
2. Maturity of ecosystem. This should improve over time but I've had challenges finding high quality libraries, especially for things like database connection drivers or making network requests. It's often hard to tell how well-supported or complete a library is and regressions were a regular occurrence.
3. Documentation. In practice I rarely found official documentation complete or even helpful (outside of big projects like Phoenix / Ecto). Even core Erlang libraries had surprising chunks missing. It's been awhile but I remember it being very hard to figure out what options were supported in Erlang's TLS wrapper. I ended up stitching together pieces from the OpenSSL documentation, Erlang source code, and lots of trial and error.
4. OTP overlap with other scheduling systems. This isn't a design flaw as much as a potential footgun depending on how you deploy Elixir code, but there is a lot of overlap between the cluster management support in Erlang/OTP and, for example, container orchestration in Kubernetes. Both support scheduling some concept of tasks, configuring redundant copies of tasks, detecting problems, and restarting tasks according to policy. Deploying an OTP application on top of Kubernetes (on top of Linux) results in 3 layers of OS-like task scheduling to learn, teach, maintain, and debug through, all with distinct tooling, terminology, and limitations.
All in all, I found Erlang/OTP to be a pretty interesting and compelling platform, especially for certain special purpose applications. If I ever use it again I'll probably skip the extra layer of Elixir syntax and write straight Erlang.
I've gotten to the point I can adapt to missing libraries but what it does at runtime is very different from anything else.
I don't do a lot of server side node.js but I have a lot of confidence if someone called me at midnight saying their node service was down I could jump in and start making progress fast. I think it would take some time running Erlang in production to get there.
I've certainly found it to be much more specialist than I think many cheerleaders would have it. It can do most things out of the box (and that I guess is the point of OTP), but to make it do those things requires quite a deep understanding of the platform (and that it is a platform, that the language is just a way to interact with that platform). Elixir is my favourite of any language I've worked with, so I'm not down on it, but it's not general purpose
1. Dialyzer is just a huge piece of shit. It's an incredibly impressive piece of shit, and makes it possible to pretend that Elixir supports typing, but it has so many edge cases that we've run into them writing even trivial code - and it misses a lot of things that, e.g., Java or Typescript would have no problems with.
2. Testing is kind of fiddly and annoying. It's very difficult to consistently mock things out. And yes, yes, it's a functional language so in theory every function should be small and trivially testable, but unless you're writing toy code, eventually you will have dependencies you would like to mock out in tests, and you will have complicated business logic living _somewhere_.
On the flip side, I prototyped out a drone flight controller in Elixir thinking that the “fail and restart” portion would be awesome for reliability (if the RTK receiver craps out and restarts, no big deal, the EKF can just fall back to using the IMU until the RTK is back online). That part of it all worked great, but doing real-time numerical computing in Elixir is painful. As was generating the signals for the ESCs. I was making C modules to try to talk to the ESCs, and errors in those modules had the ability to corrupt the BEAM. I started looking at ports too (where the C code runs as a separate process), but gave up when it felt like everything was feeling way too complex and fragile. Let it crash is great, but “it crashes all the time” is not. I could have probably gotten it working, but... not a great domain for it.
Erlang is great at soft real-time but does not work at all for hard real-time, there simply are no guarantees, just very good average latency.
If you want hard real time (such as for in-flight control of a drone) then you should look elsewhere.
let it crash != robustness not required
Even if let it crash worked, it seems philosophically inappropriate for a drone controller.
What is completely incompatible with disposable workers is complex numeric processing. Unless you can segment it with small non-communicating unities, of course. That's the kind of application that asks for a low level language.
In embedded systems, certainly safety-critical ones, you have to consider the possibility of a reset path at all points in your flow of control. Your code resetting is a valid flow of control.
This is absolutely necessary to get a properly robust system.
Your drone system needs to be able to handle a reset at any point and still maintain adequate control of the aircraft. This is "Let it crash".
"Let it crash" is essentially the same thing. Take account of the necessity of reset, because you can't stop it happening.
Of you want to build dynamic web applications, phoenix+LiveView is imo _the_ best Option currently. It also keeps the promises: reliable (zero-maintenance apps running for years on heroku), productive (prototyping things is really fast: like rails 2/3 days), fast (templates render fast even when complex). It is also great to have actor processes and ETS at your fingertips.
Having said that: dynamic typing truly sucks for bigger projects/large teams. It is awful how much you have to write tests for, and these tests will slow down further velocity when things will change... stellar opposite to Rust.
Never attempt to do any serious number crunching in Elixir/Erlang, the BEAM truly is not made for this stuff. Plan to use/write NIFs or Ports for computational tasks. Even compiling larger Phoenix codebases will take minutes if you aren’t cautious of using „import“ in modules.
Finally: deployment is really the major pain point. I love the single file deployment story of rust and go.
I do regret choosing Elixir at times for the following reasons:
1. I am new to the language (I've been using it for "years" but only very part time (not part of my day job)). While the functional approach does lead to much more elegant code, it can also be super esoteric to me as I'm new to and somewhat uncomfortable with the idea of recursion for everything. There are pre-written macros to help of course, but it's just training wheels. This will improve with time and experience. I also don't fully grok how to write my own macros. What I really need to do is spend a couple of days deep diving until I get it.
2. Phoenix is changing quickly (tho slowing down). I wrote some stuff if Phoenix 1.2, 1.3, and the upgrade path is a bit involved (no more so than a rails upgrade was tho). I suspect it's stabilizing, but when I first created one project it used brunch for assets, now it uses webpack (which is a good choice, but as a non-frontend person I'm not well equipped to make the change myself to upgrade my project). 1.3 (IIRC) also dropped models, so that required some refactor to work as a 1.3 project.
3. Deployment changes quite a bit. The old rules for Ruby go out the window. You can still treat your Phoenix app like a rails app if you want, but it's like buying a Ferrari to drive around the neighborhood at 25 mph. This takes some learning, and while much of the old rules about scalability, etc are still applicable, they do change a bit and it requires making new mistakes and learning from them.
4. I get irritated now when working with non-functional code in other languages. My tolerance for side-effects is down to nearly zero, and I get irritated when there's a dozen lines to do something that in Elixir is a couple of pipes.
That said, overall I much prefer Phoenix/Elixir. None of the downsides are the fault of Elixir, so I believe with time It'll be a no-brainer. I am already at the point where I don't start any green field projects in Rails (tho I do use Ruby extensively for scripting as it's by far the best scripting language IMHO).
I can't tolerate OOP stuff that `obj1.calls(obj2)` and changes `obj2` anymore. I didn't care before, now it makes me want to refactor everything or not write it at all.
Using Elixir kind of made me not want to use anything else.
I really don't know what I'd prefer to it though, every language has its pros and cons and static typing systems while nice also can introduce their own problems. I've spent days tinkering with type abstractions in Haskell rather than getting work done - not because I had to but because I knew more expressive solutions were possible and the allure of them was too hard to resist. Elixir is a fantastic language for getting work done.
If anything holds me back its my experience with Scala. I used Scala for awhile (actually it was my introduction to functional programming) and I like the language but everything about it that seemed to make me unhappy was related to the tooling and JVM ecosystem in general. I should give Kotlin a real chance on a small project though as I haven't done that yet, I've just read the language guides.
Lots of great ideas in Elixir, Phoenix, and Ecto, but it just didn't hit the critical mass or completeness that makes the Rails ecosystem impossible to walk away from.
And because abandoning Rails is not a practical option, maintaining brainspace for an additional thing that is not widely used and nearly identical in function is not worth the cost.
Sadly.
- If your engineers are used to the Ruby, JS, Python etc. ecosystems of finding a library for everything, they may become frustrated with the smaller Elixir ecosystem. In my experience, this is actually fine because it means you have fewer external dependencies to wrangle.
- Differences between compile-time and run-time configuration is a foot gun. We ended up moving almost everything to run-time configuration, which is not what you see recommended by many libraries and examples. This is slowly changing however.
- Deployment is a bit unique, and working with building releases can be confusing for those used to deploying dynamic languages that are not compiled.
- Functional programming requires unlearning many OOP best practices. This can slow down engineers new to the language.
A lot of these issues IMO comes from how Elixir was initially marketed as a "fast Ruby" and Phoneix "fast Rails". Once you get past the syntactical similarities you realize the semantics are very, very different. Elixir is much closer to Erlang and Clojure than it is to Ruby.
Elixir is an excellent language for motivated engineers solving novel problems. If you are building a standard CRUD app, I think you will find more success with Rails or Django in 2020.
For that reason, I was pretty comfortable coming to Elixir and implementing Somewhat esoteric, games-specific networking stuff myself. It would have been nice if the library had already existed, but I am so glad I went with Elixir rather than Node or PHP (which both do have the game networking library I needed).
My experience has basically been that the “fat head” of open source libraries exists in Elixir, but the long tail of more esoteric stuff does not. By its nature, though, you won’t need 99% of the esoteric stuff available in other languages.
Although, actually, I'm converting it all to C#, since we don't really have a lot of elixir people in the shop (I had to learn Elixir just to do the maintenance when I came onboard.)
The lack of APIs can be painful at times. What is there sometimes isn't the most fun to deal with outside of the ecosystem (the driving force of the language conversion was issues using Phoenix Channels reliably with non-elixir consumers, and during the conversion we are dealing with the minor pain of pulling certain data out of an mnesia data store.)
But, I mostly enjoyed my time playing with it.
Biggest observation was that the IDE Tooling didn't feel as nice as, say, F#, which makes a bit of a difference when you're not dealing with the language day-to-day and need helpful reminders for how you're getting certain things wrong.
My biggest annoyance with Elixir is that the ecosystem seems to impose a lot of opinion on how things should be done, and sometimes actively makes it hard to do things differently. In some edge cases I couldn't do the easy thing because it's "wrong", and couldn't do the right thing because it was not yet implemented. I believe this is with good intention and is mainly a side effect of the relatively small and young community, as in many areas the de facto libraries are made by the same few people. I suppose this problem will get better as there are more libraries to choose from and/or more contributors to those libraries.
Another, milder source of annoyance for me is the lack of functional programming features in a functional programming language. To invoke a function that has been passed as a parameter you need a dot. No currying etc. Still, you can get things done and with these limitations it's harder to get yourself into a mess of functions being passed around (but not too hard if you try).
All in all, to me what Elixir brings to the table is more than enough to offset its annoyances. However when it comes to choosing Elixir, I agree with other comments suggesting to choose something the team is comfortable with.
I came from a Rails background too and Elixir is so much better than even Ruby (which I'm a huge fan of already) simply because it forces you to think in terms of functions rather than OOP based. Most people who try and give up early on Elixir - including myself (I gave up initially after a few weeks actually, before going back to it again) was simply because I was trying to code things like I would normally do in an OOP setting.
For example, you don't use FOR loops in Elixir, you don't nest switch cases. If you try to be in an OOP mindset with this language, you WILL fail. And that's a good thing. The language constantly challenges you to rethink a lot about your code. And it always results in better code when you do it functionally. Eg. using pattern matching instead of multiple if else clauses.
The downside of Elixir is the ecosystem isn't as mature as Ruby, so, for some of the things you're trying to accomplish, you won't find ready made libraries, or they won't be maintained anymore. Sometimes, you will be forced to write something from scratch and that can cost valuable developer time. Integration of stuff like GraphQL can be too verbose compared to other programming environments as well. But, the language itself? It's hard to complain. My entire consulting career has been built around Elixir since I made the switch ~3+ years ago. That's how good it is.
Hard disagree here. Functional code is often better, but certainly not always.
In our particular case,
1, we did not have good libraries for file uploads to Azure and to handle the signing for securely displaying them.
This was solved by 2 very simple modules. In the end, both things are fairy simple unless you want them complicated (ActiveStorage can sure do "more" out-of-the box).
This also means the code it's super straight-forward and understandable.
2, we did not have a good library for SOAP.
We started to build a simple SOAP library. This I would consider a pain, but there is always option to avoid it (see no. 3).
3, we did not have good PDF libraries.
I happened to package my Ruby library (using Prawn) as a Docker container and simply call its API. This could also simply solve no. 2.
On the other hand hand,
1, we got one of the best GraphQL libraries out there (Absinthe)
2, we got a really fast test suite (once everything compiles, that is)
3, we are getting great tooling like Phoenix LiveDashboard for free
Also I want to note that deployment is now quite easy with "mix releases" (as compare to what it was). You even get a remote console that connects to your running cluster. No problem.
[0] https://nts.strzibny.name/phoenix-livedashboard-request-logg...
- Elixir and the Phoenix framework is perfect when building concurrency systems. Our company had made a ticketing system, solving the tickets for "Cirque du Soleil - Hongkong 2018". During two months of the event, our system (one Elixir/Phoenix server - scale up to two - one load balancer) can handle more than 5000 tickets per second. We were so excited. Unfortunately, the client's company closed after that. So the project was closed too.
- The second project was a product of a finance startup. Not so much thing I can share in this project, but the main feature is chatting. We used the Phoenix Channel and Graphql Subscription for this feature.
- In the side project, one member of the team used Phoenix Liveview to build a Coup game (board game).
So, in general, when I work with Elixir, I see a few pros and cons:
Pros:
- Fast. Functional programming is really fun. I love pattern matching.
- Not so hard to write tests. It's easy to mock external APIs. (Check out this blog: http://blog.plataformatec.com.br/2015/10/mocks-and-explicit-...)
- Easy to deploy. After dockerize the application, I can run it anywhere. The ticketing system was run on AliCloud, and the second project was run on Kubernetes (AWS).
Cons:
- Lack of supported libraries for external services. Sometimes, our team has to write a wrapper library to call directly to external services' APIs.
- Not easy to hire an Elixir developer in Singapore. Our team tried to hire Ruby on Rails developers and train them. But they were not feeling interesting in learning Elixir, so they left the team.
1. It actually takes a while to get your head around the fact that the biggest advantage of elixir is having access to the battle tested features in OTP. It takes longer to realize that these benefits don't come for free and elixir might not be the best solution to your problem.
2. The learning curve for writing functional code is different for different people. As a result, it's impossible to figure out before hand how long it will take a team used to writing pythonic code, for example, to start writing functional, elixir code.
3. You're able to emulate something like type checking between pattern matching and ecto schemas but it's not type checking. Be prepared to spend time coming up with best practices for writing tests.
Coupled with the relative slowness of the compiler, this means your average development session is a loop of
1. trying to get something that pass syntax check
2. starting your application
3. making a request
4. `no match of right hand side...` because you had a function that expected `{:error {:ok {:foo %{bar: 0}}}` down a five-level-deep stack, and you typed `%{"bar" => 0}`, you fool ! Or maybe you typed "baar". Or maybe you did not type it, and some of your colleague did. How would she know ? `dialixir` was probably disabled a few years ago because the error messages were not helping anyone, and you're not going to write `@spec` anymore at this point.
I heard some people managed to overcome this by using the REPL, but in my experience it just means making the spelling mistakes in a terminal rather than in an editor.
Elixir is great for code that mostly does "routing" of messages from sources to sources.
Phoenix channels are a blasts.
Pattern matching would be great if the compiler could help you with the obvious mistakes.
Libraries are exactly how you would expect for a few-years-old language. Name one topic, and there are three competing libraries : one of them does not work, the other one is unmaintained, the third one has completely changed its API in latest alpha. You're going to pick the wrong one anyway.
If I could go back in time, I'd avoid writing any kind of business logic in it.
(Opinions mine.)
The main complaints are valid: hiring issues, lack of a good typesystem, small ecosystem. Yet I have been the most productive with Elixir and would rather move to something better than regress to the mean just because Elixir isn't a mainstream language yet (which would solve the hiring and small ecosystem issues).
Most Clojurists tout "libraries over frameworks", accordingly perhaps with the flexibility of the language itself. Personally, I think acceptance of frameworks would bring much needed standardization to the Clojure experience. On the frontend, I enjoy the lightweight Re-frame.
Elixir is also dynamic but with annotations that gives the illusion of types. It's really the worst of both worlds.
* Not enough engineers can write/understand elixir which kept leading to people being blocked
* Library ecosystem for scientific applications is lacking
* The language itself is confusing and the macro capability causes code to be uniquely incomprehensible in each different module
It was a worthwhile experiment but you just can't beat the advantages of having extensive documentation and expertise in the more mainstream languages unless you have very specific performance requirements.
What stopped us was not so much the lack of libraries, but programming ergonomics. E.g. when using keyword lists for function options, callers need to pass the keywords in the exact same order. Also mocking was difficult in tests.
Elixir is a great project with a friendly community, it just didn't work for us. YMMV so by all means try it out if you're interested!
That's not generally true in the language. Were you trying to pattern match options in your own functions? Newbies to the language can get excited about the pattern matching feature and try to apply it in places where it's not right (well I may be projecting there).
> Also mocking was difficult in tests.
This is true. Were you using Mox? Mox makes things considerably easier at the cost of some boilerplate, with the added benefit that you can run concurrent mocks. You also have to not think of mocks in elixir like Ruby, they are very different.
I'm sorry I wish someone more experienced with Elixir could have helped onboard you.
Proplists should be replaced with maps pretty much everywhere.
>Also mocking was difficult in tests
Interesting, what exactly was difficult? You just replace one function with another.
I'm not the person you're asking, but iirc, that replacement is global - so once you replace function/module X with MockX, you can't test X itself, nor anything else that relies on X.
It's a pity that the elixir community doesn't make this clearer, because I'm seeing "not being able to Mock" be a common complaint.
I completely agree. Not having an expert Elixirist able to help me: It took me a full six months of fooling around with Mox (and a cryptic feature announcement in the 1.8 release) to really dig into and grok it. At this stage in my elixir experience, I have even PR'd a feature that's been integrated into Mox, so it's not completely incomprhensible, it's a super-well-written library.
Honestly, It's a totally (alien/from-the-future)-technology (in both the good and bad senses) library. The bad senses could be almost trivially fixed with an in-depth, free, online video masterclass or, hell, even a conference lecture on it, and blog posts. More people that really know how it works should blog about how amazing Mox is. Unfortunately, I suspect that the people who are using Mox to its fullest extent are too busy pushing code to prod, ha.
I never used that Mox and never had a need to. I just use `Mock` which allows `passthrough` as I stated in another comment, what other clarity is needed I don't understand. Can you give any example of the problem you can not solve instead of pitiness?
Inside `with_mock` code will only access mocked function. In case you want to call original function in some condition - you have `passthrough` available inside a new function which will call the original one. So what is the problem exactly?
This would've been fixed by creating a map from the passed in list, which would remove the order dependency but still keep the ergonomics of a keyword list, or just querying the keyword-list with `Keyword.*` functions. Not that big of an issue, to be honest, and not a showstopper. There are much bigger issues with the language, but Ruby doesn't offer any upsides in those cases either.
The backends were swagger which allowed me to autogenerate the Elixir source code for the GraphQL resolvers from the swagger.json file. It was a project for a telecom company which also employed Erlang programmers.
My experience: stability was outstanding, performance reasonable. Got a lot of praise for it. Only got one big report in two years.
Would I recommend it? Only to experienced software engineers.
Would I recommend it to companies? Only if you understand that every new tool or stack is a strategic decision.
The one thing that we struggled with Elixir with is that it's quite hard to find people that know Elixir. For a long time I was the only one doing Elixir in the company, which introduced a bottleneck. That said, when you find someone who knows Elixir they tend to know Elixir (and a bit of infra) quite well.
In order to tell the type of various method signatures, it seems like you have to go looking at other areas of your program. I’m aware that it’s possible to annotate method signatures, but this still didn’t lead me to feel secure. I think I was hoping for/expecting something more Haskell-like.
It’s easier to recognize that a feature (dynamic typing) is an issue in some applications after trying it for a while rather than just taking a theoretical view. I’ve sometimes misunderstood how a feature works in practice by relying too much on my theoretical understanding.
Also, because Elixir allows type annotations, I was thinking that I might not need static typing. It turned out (at least for me) that I would prefer to have a language that insists on type annotations by default rather than allowing them as options.
However, I wouldn't use it for a real project:
1. The ecosystem doesn't compare to ruby/python/javascript. There's a lot of missing packages and abandoned libraries. I ended up having patch a bunch of libraries I used.
2. The tooling that exists is nice and 'just works' but there's a lot that's missing from ruby land: https://elixirforum.com/t/help-an-elixir-beginner-find-missi...
3. The deployment story is horrible. The heroku/dokku buildpacks didn't work by default and it's not obvious how you can easily deploy a mix/distillery release without writing your own deploy scripts. https://elixirforum.com/t/official-heroku-dokku-buildpack/28...
4. Phoenix != Rails. There's many many obvious things that are missing. `mail_to` for example.
However, I dislike the GenServer syntax, it feels too complicated with random tuples all over the place. Also, I feel like the job market is too small. I wasn't able to find a job using it (although maybe COVID is to blame here).
Probably the biggest advantage of Elixir over Ruby is the runtime. The Erlang VM has proper concurrency, and it provides really nice primitives for working with it.
The ways in which Elixir sucks tend to be the same as Ruby: dynamic types, somewhat slow runtime (compared to C, Java, Rust, Go, etc). I'd also add that library support is still a bit weak with Elixir, but it's always getting better.
Where Phoenix really shines is its support for fancy stuff like Websockets and distributed messaging in your backend. Trying to do these things in Rails leaves much to be desired.
+ Phoenix _better_ than Rails, which is very surprising
- less packages available, though almost everything covered by at least one package. ( hex.pm vs rubygems.org )
It's only slow if you are comparing a single-threaded operations, Erlang VM is designed to scale horizontally, not to be fast with one thread.
But some people will still walk away thinking that Go and Elixir perform about the same while Node is 4x slower.
The first one, I can't complaint. Every programming language have it's weakness, to solve that I'm offload processing stuff into Rust. Luckily, Elixir have NIF integration with Rust.
Elixir and its ilk tends to make it easy to do what you want to do.
In Go you have to re-write things over and over because the abstractions are low, but they're easy to do. In Elixir you can use the built in libs/abstractions, but you have have no idea how they work.
This is a gross simplification of both sides of course, but I often see Go-people arguing a different point than their counterpoint in <other language> is trying to make.
They consider themselves "productive" because in go "...the code just flies from my fingers...", when what they're doing is reimplementing the same things over and over because the language doesn't support it.
HAVING to type a lot of stuff, even if it's easy to type, isn't productive. It's just a lot of typing.
Debugging it it a nightmare especially when you have to roll up your sleeves and go into Erlang.
The biggest help was IO.inspect, and then the next biggest was elixir_ls, which had used typespec analysis and highlighted the error for me in my ide.
The language itself is pretty small and I felt like the first stage was pretty easy.
Learning the OTP is a much bigger challenge.
At the same time, I think you can gain a lot of the benefits of Elixir and the BEAM without needing to be an OTP expert, because frameworks like Phoenix leverage it for you.
In my experience, things like pattern matching and immutable data really contribute to a more maintainable code base, whether you find you need the actor model or not.
Part 2 learning Phoenix and OTP has been a long, ongoing thing. If I did it full time it would be done, but doing it off and on part time it is a long slog.
* cant find people for it and if you do they come because of the tech not because of the real world problems we are trying to solve.
* the no types thing makes people write tests that test for typos and make refactoring incredibly difficult because you have to fix gazillion pattern matching tests
* the tooling is just horrible, cant jump, because of pattern matching that makes it much more annoying
* phoenix was complete misfit for us as well, so i ended up hating it for no reason
so because of no-tooling and on-refactoring, accidental complexity only grows, and we ended up with much more fragile system with infinite failure domain.
i wrote briefly about why it was chosen at https://eng.rekki.com/kitchen/kitchen.txt and why we moved to go.
that being said, i am sad that this was my first real world experience with elixir, in a project where it was not a good fit, so now i am totally biased against it. when i tried it briefly for few hobby projects it seemed like a quite cool language.
-b
Elixir is an okay language at best, if you look at it in comparison with let's say Python:
1. It is not any safer than python 2. Not any faster than python 3. Abstracts away a lot of things away from you "magically" just like python 4. It is harder to read elixir cause you will have people writing all sorts of random macros.
Where Elixir is good: 1. Concurrency (Processes!) 2. Based on BEAM but you won't care for your simple API project and you shouldn't have to. 3. Other erlang goodness supervisors, otp etc.
Personally I detest Python, but you would be better off writing your app in Python cause using Elixir you have gained nothing much substantially but lost on all ecosystem and community support Python has gained in its long history.
As a sidenote, I hope you use a strongly typed language.
1. Hiring. Generally speaking, finding experienced Elixir engineers is not cheap nor easy. Also, chances of finding someone willing to work in an office is even more slim (very remote heavy talent pool)...
2. Deployment/monitoring. Shipping your first Elixir application to production will likely be one giant act of frustration if you're used to Ruby or other scripting languages. Compile time vs runtime settings and environment variables are a HUGE gotcha. Also, as fate would have it, BEAM is more like an operating system than a simple runtime and thinking you can toss it into container and be off to the races is a recipe for disaster. Hope you didn't want to use distributed Erlang... (though this has gotten better recently, tbh). A lot of common monitoring tools and methodologies have to be thrown out the window because of how BEAM works internally (there are great ones to accomplish most anything, but its just more to learn).
3. Distributed systems, make for distributed problems. Yes, Erlang/Elixir and BEAM give you some fucking amazing tools to build distributed systems... that doesn't make building distributed systems themselves "easy" or "quick."
4. Performance. Erlang/Elixir will probably take a hot-fast shit on most scripting languages when it comes to network requests; however, if you need to do any long-running number crunching or complex string manipulation, you'd be better off almost anywhere else. This could also be categorized as "use the right tool for the job, dummy."
5. Learning curve is real and vastly underestimated. You aren't just learning a language, you're sort of embarking on a quest to learn a new paradigm. Erlang/Elixir, BEAM, and OTP (I'm going to refer to them as the "Triforce") were designed together to solve a specific set of problems. The Triforce has been in-use, 24/7, 365 days a year, for 30 years, in real, critical, production systems. If you ever wanted to learn the "Actor" pattern, BEAM is the most metal implementation I know of... Also, really sit down and make sure you understand it's all "non-blocking" thanks to the scheduler being able to pre-empt processes.
----
With regards to 3rd party libraries, I find in Elixir and Erlang, I need and use way less of them... BEAM and OTP give you soooooooo much it's kind of absurd how little you end up missing or wanting.
----
Lastly, while a lot of the success stories here are great to read, a lot of them have bad code, or things that are just silly to the trained eye. And that's totally okay, the authors are learning, we all have do it, but I'd probably not recommend the code as study material. In general, try and find examples from people who have seem to have been using it for a few years.
----
Addendum (via edit):
FWIW, I've been writing Elixir for about six years now (just checked some commits), and have been employed full-time writing Elixir for five of those. I wrote Ruby before that, and have never looked back... If you want my number one selling point to any Rubyist :trollface: (but also very real), is in Elixir a "horrible, slow, wretched monolithic test suite" still will take less than 90 seconds to run.
The main question is always the same, “how much effort it will take me to find Febe if I need them”.
And my answer is always the same, “make them come and work with you, and focus people excited and open to learn a new technology”.
My only real complaint with Elixir (and this covers Phoenix and Live View too) is that things tend to get announced WAY before they're anywhere near being production ready, so you may find yourself postponing building anything because you're waiting for the announced thing to be ready to use.
For example Live View was talked about in the later months of 2018 but there's still some things missing today that make it pretty sketchy to really use it in a non-trivial production app, and the docs leave much to be desired. Things like LV based file uploads as being part of the library was in the "near future" or "on the horizon" a year ago, etc..
I only spent a weekend using it for a project to play with about 6 weeks ago and ended up uncovering some bugs and even some feature requests that quickly ended up making its way into the master branch of LV after reporting them. But these are for things that seemed very basic to me in the sense that if the folks who were developing it were using it in larger production apps, there's no way those things wouldn't have made it into the library sooner.
In other words, I don't feel comfortable or confident at all to use it in production because I'm afraid if I spend some real time working with it, I'm going to end up being patient zero for so many bugs and edge cases. I want to focus on building my apps and trust the libraries I use, not be a pioneer (software is already difficult enough without bugs!).
On the flip side, Rails is a champion here and IMO it's very much why it's so popular and will remain popular 5 or even 10 years from now. It's because most of the things in Rails come from tech focused businesses using it to drive their services and the core team also use it to drive their business. Everything feels like it's really battle hardened, production ready when features are talked about and was carefully designed based on months of real world usage.
With Elixir, Phoenix and LV that feeling isn't there -- at least not to me. It sort of kind of feels like the libraries are being built from a theoretical point of view. I mean, I know the creators of those libraries do consulting work, but it's super behind closed doors. There's never any talk about what drives the development of practical features and it's kind of a bummer to get glimpses of things on the horizon, but then years later they aren't ready for production.
That's the thing that worries me about the future of the ecosystem. I also think it's partly why it hasn't exploded in popularity. It's a nice ecosystem for sure, but it's missing critical components for it to be adopted by the masses.
That and I think generally speaking the language is very hard to pick up. I know I struggle hard with Elixir with ~20 years of general dev experience across half a dozen non-functional languages before touching Elixir. I still feel like I have trouble reading Elixir code I've written 2 months ago when it uses some "real" functional bits that I had to ask for help on.
I often feel like I hit 100% dead ends and have to ask another human for help. I don't think I would be using the language if it wasn't for the IRC / Slack channels. There's some folks going above and beyond to help people out there, including Jose himself (the creator of Elixir).
With Python and Ruby I rarely encounter situations where I had to ask for help like this. Google always had the answer for everything I couldn't figure out based on tinkering and reading the docs.
It's not just due to less blog posts existing for Elixir / Phoenix too. It's the language overall. I want to love it, but for whatever reason my brain won't accept it naturally. I get hung up so frequently. I know this is just my brain being an idiot, but I do wonder if anyone else feels the same.
My gut tells me yes because otherwise the language would already be one of the most popular choices for building web apps. There's a lot of great things surrounding the language.
- Debugging is hard. You can’t throw a REPL wherever your want in your code and pipe states are hard to inspect.
- Serving static files in Phoenix is oddly super hacky.
- Deploy is hard. Mostly by the lack of support out there in term of documentation and services.
In my experience, I found I could throw a `require IEx; IEx.pry` pretty much anywhere in the code to get a REPL. This seemed like a unique superpower of Elixir to me. Where have you found that this isn't possible?
I agree with the pipe states point.
My main complaint with Elixir is that I'd discover really stupid mistakes with run time errors. The sort of things a bunch of other languages I've used would preclude at compile time. I never dug into more advanced debugging or Dialyzer though.
abc
|> foo()
|> bar()
abc |> IO.inspect(label: "a")
|> foo() |> IO.inspect(label: "b")
|> bar() |> IO.inspect(label: "c")
2- agreed
3- Mix releases have solved 80% of this problem, for me. Since it's trivially extensible, I do a series of compile-time checks, include verifying that the current branch is on master and correctly git-tagged; and then uploading to Amazon S3. I haven't done this yet, but it's going to eventually trigger an automated blue-green deploy.
I clearly hadn't bothered to read the 'Getting Started - Debugging' page on the Elixir website at any point.
Any codebase beyond fairly small will be harder and harder to work with to an unreasonable degree, in my experience, and any perceived "velocity" gained from the dynamic nature of it is paid for doubly so by the lack of safety you get beyond toy projects.
I'm only slightly more for Erlang as a choice mostly because it's simpler than Elixir and doesn't have as much obfuscation of logic added to it that Elixir does, but in reality it's also a bad choice for any bigger system. The runtime is absolutely great, but the languages on it are generally not good enough.
[1] This looks fine to me? https://elixir-lang.org/getting-started/typespecs-and-behavi...
[2] I have limited experience with Haskell, but my impression is that the type capabilities do actually allow you to forego testing in a lot of cases. The next best static type system I've used is C#'s, which I used professionally for years, and in C# I think automated testing is still very necessary.
EDIT: I regret even talking about type checking here--that's not my point. My point is that for modeling domains, Elixir's type system is totally adequate. I'll agree that Elixir's type checking could be better, but that's not what the comment I was responding to said.
Maybe ruby is different than python in this regard, but with python I see a lot of “assert isinstance(arg, dict)” in functions which would not be necessary with type checking.
Feeding an unexpected type into a function, and having it carry on as normal, is really scary.
That is how duck typing works. Generally you shouldn't use "is instance" but more use a set of "callable" checks to make sure the structure has the required functions.
Things like Go Interface types are, of course, a formalisation of duck typing and is what has helped static typing come back into fashion.
The fact is, in most cases a Python function won't carry on as normal if you feed it an unexpected type, because while Python's type system isn't static, it is fairly strong. In general I am a fan of strong types and I would like it if Python's type system were a lot stronger.
But ultimately this isn't what my post was about: I'm talking about modeling your domain with types, not type checking.
~/$ python
Python 3.7.4 (default, Oct 12 2019, 18:55:28)
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> hello = 'Hello, world'
>>> one = 1
>>> hello + 1
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: can only concatenate str (not "int") to str
1. Strong static types (makes a lot of assertions about type, checks these assertions at compile time; Haskell, ML, OCaml, Rust, C# are reasonable examples--Haskell folks would probably laugh at me including C# here, but it's the only example in widespread non-academic use). Example (C#):
var hello = "Hello, world";
var one = 1;
var result = hello + one // Doesn't compile
3. Weak static types (makes few assertions about type, checks these assertions at compile time; C, C++ are good examples, although it's arguable that C actually does more checking at run time than at compile time). Example (C):
char* hello = "Hello, world";
int one = 1;
char* result = hello + one; // Happily compiles
>>> var hello = "Hello, world"
undefined
>>> var one = 1
undefined
>>> hello + 1
"Hello, world1"
You can run mypy which will list problems. Also PyCharm understands the annotations and does all the things I mentioned it do: proper autocomplete, proper refactoring functionality and highlighting errors
I strongly disagree. There are plenty of ways to write maintainable code. Static types can help, but it's quite possible to write maintainable code in dynamically typed languages.
And, for the record, running unit tests doesn't require me to run the whole program.
And, even if I never explicitly test types (assert isinstance(foo, str) is an antipattern anyway), behavioral tests of my code will turn up most type errors.
The obsession with checking types at compile time rather than 30 seconds later while running the unit tests doesn't serve anyone.
Specs/contracts are cool but ultimately don't afford the same kind of descriptive and expressive power that a static type system does.
> static types in most languages[2] don't reduce this burden much: there isn't an alternative to thorough testing.
However, there definitely is a burden about how much testing you have to write. I generally don't want to have to test every branch of my program to make sure a string doesn't slip through where an int should be or that variables are initialized and not null, etc.
The correct completion to this sentence is "irrelevant.", because that's not what anyone is proposing.
The fact is, behavioral tests catch a lot of type errors even without intending to, and more to the point, if you test all the behavior you care about, then you don't care if there are type errors, because they only occur in situations where you don't care.
It's impossible to defeat the claim that "with good tests you don't need static type checking", since every counter-example can be dismissed by arguing that better developers would have covered that test case.
I can claim just the same that "with correct code you don't need tests", and dismiss every counter-example by arguing that better developers wouldn't have made that mistake.
Obviously we know that developers sometimes make mistakes, and sometimes these mistakes are in the tests themselves. So trust your experience regarding how this all works out.
In my experience this idea always brings a strawman, "Well in statically typed languages you still have to test", which is obviously true. But the type of tests and the content of the tests is very different.
Maybe someone is saying that, but I didn't say that.
My question isn't whether you can test enough to catch all bugs. My question is whether time spent wrangling types gets you more value than time spent writing tests.
> But the type of tests and the content of the tests is very different.
Are they? How so?
Again, unit tests of the form `assert isinstance(foo, type)` are an antipattern--that's not what I'm proposing.
Yes, by a tremendous amount, in my experience.
> Are they? How so?
Tests are only as good as the person writing them. I could see a model working where the person that wrote the code isn't the person that writes the test, but that's definitely not how most development orgs work. If a dev is good enough/capable of writing comprehensive enough tests to accurately test the correctness of their code, that's great, but almost none are (I say almost because I actually mean "actually none" but am leaving room for my own error). If you're an average dev, you'll write average tests (neither of these are insults), but that means you still won't catch everything (by a lot).
I think another comment of yours on my posts actually summarizes the core disconnect between your thinking and mine.
> But in the vast majority of modern software, it mostly just matters that you catch and fix bugs quickly--whether you catch those bugs at compile time or runtime is usually not as critical.
I couldn't possibly disagree more with this statement.
> Yes, by a tremendous amount, in my experience.
Well, then I'd have to ask what your experience is that causes you to believe this? I don't mean years, I mean what languages, and what, more specifically, you observed.
> Tests are only as good as the person writing them. I could see a model working where the person that wrote the code isn't the person that writes the test, but that's definitely not how most development orgs work. If a dev is good enough/capable of writing comprehensive enough tests to accurately test the correctness of their code, that's great, but almost none are (I say almost because I actually mean "actually none" but am leaving room for my own error).
Static types are also only as good as the person writing them. C's type system, for example, lets through a wide variety of type errors. And users of a type system can easily bypass a type system or extend it poorly: I've written a lot of C#, and while C# has a type system which, when effectively used, can be extremely effective, I've also seen it used with dependency injection to cause all sorts of tricky bugs.
I don't think we can conclude much from bad programmers doing bad things except that good programmers are better, which is practically tautological.
> If you're an average dev, you'll write average tests (neither of these are insults), but that means you still won't catch everything (by a lot).
So? "Catching everything" isn't a thing--if you're talking about that, you're not talking about reality. There are two systems I've ever heard of which might not have any bugs--and in both cases an absurd amount of effort was put into verification (far beyond static types), which, even late in the process, still caught a few bugs. Static types aren't adequate to catch everything either.
> > But in the vast majority of modern software, it mostly just matters that you catch and fix bugs quickly--whether you catch those bugs at compile time or runtime is usually not as critical.
> I couldn't possibly disagree more with this statement.
shrug Okay... To be clear, "runtime" doesn't mean "in production".
Probably 20 Rails developers across 3 companies
> So? "Catching everything" isn't a thing
The "everything" I'm referring to is less about "all bugs", and more about "Type errors, spelling mistakes, missing imports, etc". All code has bugs. Not all languages allow remedial spelling errors to make it into a build.
> shrug Okay... To be clear, "runtime" doesn't mean "in production".
It sure does not. I'm not sure that changes anything. It seems like being difficult for the sake of it to argue that it's the same value to catch potential bugs now vs. later. Obviously the answer is now.
f = fn
{:ok, message} -> "It worked #{message}"
{:eror, message} -> "There was an error #{message}"
end
iex(2)> f.({:ok, "yay"})
"It worked yay"
iex(3)> f.({:error, "oh no"})
** (FunctionClauseError) no function clause matching in :erl_eval."-inside-an-interpreted-fun-"/1
Contrast with Scala, where using Either (which can be Left or Right):
def f(x: Either[String, String]) = x match {
case Right(x) => s"It worked $x"
case Left(x) => s"There was an error $x"
}
scala> def f(x: Either[String, String]) = x match {
| case Right(x) => s"It worked $x"
| }
^
warning: match may not be exhaustive.
It would fail on the following input: Left(_)
scala> sealed trait Status
| case object Ok extends Status
| case object Error extends Status
trait Status
object Ok
object Error
scala> def f2(x: (Status, String)) = x match {
| case (Ok, msg: String) => s"It worked $msg"
| case (Error, msg: String) => s"There was an error $msg"
| }
def f2(x: (Status, String)): String
scala> def f2(x: (Status, String)) = x match {
| case (Ok, msg: String) => s"It worked $msg"
| }
^
warning: match may not be exhaustive.
It would fail on the following input: (Error, _)
def f2(x: (Status, String)): String
scala> def f2(x: (Status, String)) = x match {
| case (Ok, msg: String) => s"It worked $msg"
| case (Eror, msg: String) => s"There was an error $msg"
| }
case (Eror, msg: String) => s"There was an error $msg"
^
On line 3: error: not found: value Eror
If I test all the situations I care about, the one situation I thought I didn't care about is going to fuck me in production.
It's not useful to talk about a binary "bugs versus no bugs", because "no bugs" isn't plausible in most codebases.
It's also not useful to talk about "more bugs versus fewer bugs" because that's only part of the picture: the other parts of the picture are how much development effort was necessary to achieve the level of bugs you have, and whether the number of bugs you have, and when you have them, is acceptable.
If it's a life or death application where any bugs at runtime are unacceptable, then of course we want static types, but static types aren't enough: I'd also want a theorem prover, fuzzer, a large number of human testers, and a bunch of other things that require way too much effort to be useful in an average software project.
The vast majority of software projects, runtime bugs are acceptable as long as they don't lose data, cause downtime, or expose private information. If you catch these bugs during unit testing instead of 30 seconds earlier at compile time, that's fine. Static types might catch a few more bugs, but it is very much not in evidence that the level of effort involved is lower than equivalent unit testing in situations where reliability requirements are typical.
Not to say I haven't had benefits from catching type errors but generally not as great as those I have from having an automated GUI test running.
While I personally prefer certain statically typed languages over dynamic ones for new projects, In practice, for small to medium sized projects, runtime type errors is in my opinion much less of an issue as some people make it out to be. Except for null exceptions they rarely make it into production and if, are easy to track down and fix.
Interestingly, a lot of the people I have seen constantly running into type problems in say Python, are the ones coming from statically typed languages that keep insisting doing thing the way they are used to instead of embracing the duck.
"If you drive carefully enough, a car without seatbelts shouldn't be a problem!"
If you're writing software that's life and death critical like wearing a seatbelt, you should absolutely be using a strong, statically typed language, because catching errors at runtime is completely unacceptable. But incidentally, none of the proponents of static types on this thread have talked about any languages that I would actually use for this situation. Java or C-family languages certainly aren't strongly typed enough. Type systems aren't a magic bullet.
But in the vast majority of modern software, it mostly just matters that you catch and fix bugs quickly--whether you catch those bugs at compile time or runtime is usually not as critical.
What are we proponents of static types being asked to prove?
> But in the vast majority of modern software, it mostly just matters that you catch and fix bugs quickly--whether you catch those bugs at compile time or runtime is usually not as critical.
I would argue that catching bugs at compile time, before you ship them, is vastly preferable to catching them at run time.
Your arguments for why you think static types are better.
> I would argue that catching bugs at compile time, before you ship them, is vastly preferable to catching them at run time.
I would argue that you're only doing the benefit part of a cost-benefit analysis, which isn't very useful.
Really? You're going to go with "most languages implement static typing"?
That seems like a bold statement.
What exactly dialyzer type checking lacks compared to static type systems?
However, there definitely is a burden about how much
testing you have to write. I generally don't want to
have to test every branch of my program to make sure
a string doesn't slip through where an int should be
or that variables are initialized and not null, etc.
I've been writing Ruby full-time for about six years (including one of the largest Rails apps in the world) and I don't find this particular aspect of Ruby to be a problem whatsoever relative to statically-typed language.
Ruby is famously easy to write tests for. Creating mocks in a dynamic language is such a breeze. There are plenty of problems with Ruby but an increased test suite burden is NOT one of them. "Confident Ruby" by Avdi Grimm is a great read in this vein; not about testing specifically but writing confident Ruby code in general.
Part of it is simple, human-friendly code hygiene. Give your arguments descriptive names and use keyword params when possible. If you have a method:
foo(user_name:, height_cm:)
foo(user_name: 157, height_cm: "Smith")
Of course, I do get type errors all the time in Ruby, but that's when I'm parsing JSON or something and that's generally not something static typing's gonna help you with anyway.
Now... there ARE places where I miss strong typing in Ruby.
One, I miss having extremely intelligent IDEs like you can have with Java or C#. I absolutely loved Visual Studio and Resharper in the C# world. The amount of reasoning it can do about your code and the assistance it can give you is absolutely bonkers.
Two, obviously, there is a price in runtime execution speed and RAM usage to be paid for dynamic typing. I don't find raw execution speed to be much of a bottleneck in a Ruby app because Postgres/Redis/etc are doing all my heavy lifting anyway. RAM usage and app startup times with large applications is more of a real-world issue.
First, if you're writing any kind of big code, than somewhere, in your code base, someone else is writing a code where the user name is spelled `username`. Or `user`. You don't have to be "asleep at the wheel" to not remember, or not know, which is which. So you're going to type the wrong one (not necessarily on purpose), and you'll get a runtime error. Not that bad, sure, but you'll get it.
Also, at some point, you'll realize that a string is not the ideal way to represent a user name [1], and some of the functions that deal with users are going to start returning a Struct %User{first_name: ..., middle_name: ....}. Or will it be a Map %{"first_name" => ...} ? And surely you're going to track all the calls of `foo` to fix them. And all the calls of all the calls of `foo`, because, who knows ? And suddenly you're doing the mental job of a static type checker. Surely you're not "asleep at the wheel" anymore, because you're doing the work of the wheel by manipulating the gears by hand.
Bottom line: I'll gladly admit that I'm too old and stupid to do that anymore. I had typechecking in the 90s. Give it back.
[1]: https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-...
So you're going to type the wrong one (not
necessarily on purpose), and you'll get a runtime
error. Not that bad, sure, but you'll get it.
Now, it's certainly true that in a static language, your compiler would catch this for you. In a decent IDE it would be pointed out to you while you type.
However, static or dynamic, you're going to be writing tests anyway, so I can't view this as some sort of increased burden when it comes to writing tests.
Also, at some point, you'll realize that a string is not the
ideal way to represent a user name [1]
I don't think your examples are realistic, at all, though.
1. If you replace `User#first_name` and `User#last_name` with `User#name` (which returns a `Name` object) your test suite is going to blow up all over the place every time you call the deleted `User#first_name` or `User#last_name` methods anyway. And now you have a list of places where you need to fix your code.
2. But, what if you update the internal structure of `Name` over time? Some of the above applies. But also callers of `Name` shouldn't know too much about `Name` anyway - it should be providing some kind of `Name#display_name` or whatever function that handles all of the complexity and returns a dang string anyway.
Everything I've written here presumes the existence of a test suite, of course. Which of course takes time to write and maintain. But any nontrivial project needs one anyway regardless of language or type paradigm, right?
Bottom line: I'll gladly admit that I'm too old and
stupid to do that anymore. I had typechecking in
the 90s. Give it back.
I miss static types when I'm writing code. I want my IDE to tell me the types and type signatures when I type, and draw a little squiggly line under my code when I get it wrong. In Ruby, I wind up having 10 different files open at a time in my text editor so I can see what various methods are expecting.
This is mitigated somewhat by simply bashing out a lot of my code in irb/pry directly, since pry's `show-source` can at least tell me stuff.
Huh, I thought this was normal. Not everyone does this?
My highly subjective belief and experience is that Ruby and its ecosystem has enough other perks to make up for this loss. But, I definitely accept others might feel otherwise.
I don't think that's a reasonable assumption at all. Even if, as you assert, the average person doesn't understand that types exist in dynamically-typed languages, I don't think that means I have to conform to common misconceptions.
> Specs/contracts are cool but ultimately don't afford the same kind of descriptive and expressive power that a static type system does.
True, but not what I was talking about.
Could you explain what descriptive and expressive power is missing here?
class Square(Rectangle):
def __init__(self, side_length):
self.side_length = side_length
@property
def height(self):
return self.side_length
@property
def width(self):
return self.side_lengthAgain, this is needlessly uncharitable as to what people mean when they talk about type systems, which is obviously about the capabilities of defining new types for program analysis, not that the runtime has an internal conception of types.
> Could you explain what descriptive and expressive power is missing here?
Sure! Looking at this, I have no idea what the size of side_length is, whether it's possible for it to be negative, or whether it's possible it to be null.
Of course, unless you're writing purely square based software, most domains are more complex than this. But I still think it's pretty helpful to know the properties of the parameters being passed to build your square are!
> Again, this is needlessly uncharitable as to what people mean when they talk about type systems, which is obviously about the capabilities of defining new types for program analysis, not that the runtime has an internal conception of types.
There is nothing "internal" about the example I gave. That's valid Python code that creates a type.
My definition (which happens to be the actual definition of the word) charitably assumes that people know dynamic type systems exist, so I don't think you can really accuse me of being uncharitable. If anything I'm being too charitable, as evidenced by this conversation.
> Sure! Looking at this, I have no idea what the size of side_length is, whether it's possible for it to be negative, or whether it's possible it to be null.
> Of course, unless you're writing purely square based software, most domains are more complex than this. But I still think it's pretty helpful to know the properties of the parameters being passed to build your square are!
Do you really not know whether side_length can be negative or null? Or are you just saying that to be argumentative? If we're pretending we don't know obvious things, why not just go all the way and pretend we don't know that side_length is a number?
As for not knowing the size: why would you want to have to know the size? The fact that this square will work with any numeric type is a feature, not a bug.
Now, consider this (disclaimer: my C++ is rusty, and I didn't syntax check this):
class Square: Rectangle {
private:
double sideLength;
public:
Square(double sideLength) {
this.sideLength = sideLength;
}
}
1. The size of sideLength. Well, yes, this example does tell you what size it is. Which is rather annoying, since now you have to cast if you want to pass in an integer, and you might overflow your bounds. This is an annoyance, not a feature.[1]
2. We know that sideLength can't be negative because we know what squares are. The type doesn't enforce that. You could enforce that by using unsigned int, but then you can't handle decimals. And in either case, you can't use very very large numbers. I haven't worked in a static typed codebase which has an unsigned BigDecimal type, have you?[1]
3. We know that sideLength can't be null because we know what squares are. The type system technically also tells us that, but the type system telling us what we already know isn't particularly useful.
[1] Haskell's numeric type can actually handle this much more cleanly than C++, as I mentioned upthread. But in typical Haskell, you'd probably just let the types be implicit in a lot of cases.
tldr: i haven't seen a runtime typechecker that handles generics and function types in a satisfactory manner.
i've used various Python libraries for runtime type-checking (based on `typing` annotations) like `typeguard`. and they work okay for simple types, but suck for anything involving generics and function types. checking if something is a `List[int]` every time it's passed as a parameter is too expensive, because you have to go through the whole list (and you're out of luck if it's an Iterator[int] - can't traverse that without exhausting it). runtime typecheckers don't have enough information to check if something is a valid `CustomList[int]`. and they have no way of checking if e.g. a function (passed as a parameter) is actually a `str -> int`, at best you'll find out when you call it.
and runtime checkers, at least the ones i've used, often end up requiring more annotations than i'd have to write in a type-inferred language. it might be possible to work around that to some degree, but I think that's a fundamental limitation – unlike a static checker, they only have info about code that already ran. so you'll have to annotate code like this:
f xs = cons 'a' xs
Let's just stop right there, since it's immediately clear you aren't answering the question I asked. You're talking about type checking, not descriptive and expressive power.
The topic is whether dynamic types are suitable for domain modeling, not whether dynamic types provide static type checking. We all agree that dynamic types don't provide static type checking.
> Specs/contracts are cool but ultimately don't afford the same kind of descriptive and expressive power that a static type system does.
i understand that as "the ability to describe/enforce the domain's rules".
now, i guess i made a bit of leap, jumping to runtime typechecking. my thinking was that while modelling your domain, you might want to specify that e.g. `width` and `height` must at least be numeric (i know i would!); `typeguard` et al are a concise way of doing that in Python, but have their limitations. so static types can be more "expressive" if the tools i mentioned (generics and function types) are useful for describing your domain model, which i often find to be the case.
looks like i missed the mark there; but in that case, i'm not sure what point you're making with that Square class. it's hard to say what's missing (or not) because there's not a lot there.
And if you're using a mainstream statically typed programming language, you probably aren't actually going to express that it's a numeric using the type system. 90% of the time someone will just throw `int` on there and call it a day, and that type expresses a bunch of lies about the height and width: it says it can be negative, and it says it can't be a decimal, and it says it can't be greater than the INT_MAX of you machine. `unsigned long` and `double` are both a little better, but are still expressing a bunch of lies. And that's if you are in a more modern language: in C, for example, you're telling the compiler that it's totally cool to add the side_length to a char* and (depending on you settings) not even warn about it.
So I have to ask, where exactly is this expressive power you're talking about? You're only gaining the ability to express things to the compiler, not humans, and it doesn't actually give you the ability to express what you want to express to the compiler. You've eliminated the narrow class of bugs where:
1. The code passes in a non-numeric.
2. The bug occurs on a path not traversed in normal testing.
And you've done this at the cost of your code not being able to handle a lot of the numeric values you might want to be able to handle.
i just wanted to make a point about how checking types at runtime doesn't mesh well with generics and functions, so it's always going to be limited on that axis. and perhaps there's a dynamically typed nirvana where you just handle that differently, but it's a problem i personally had
[edit: removed unnecessary snarkiness]
Most of this is incorrect, both for and against your argument. Elixir has a very incapable and incomplete type system. It's optional but can be checked via `dialyzer` but never confidently and exhaustively checked because `dialyzer` will sometimes plain not work, sometimes be foiled by bad library specs and sometimes fall back to "Everything must be OK because I can't prove this incorrect".
All in all it's not a useful substitute for a real type system used to model things.
Type specs aren't useful for domain modelling but serve mostly as extra documentation more than anything.
Testing is important no matter what language you use (except maybe dependently typed ones, I don't know?). It's not a substitute for a good type system. Even disregarding type-directed property tests with automatically generated test cases, there's a lot of value to be had in tests that actually just test logic, not basic types.
>All in all it's not a useful substitute for a real type system used to model things.
>Type specs aren't useful for domain modelling
Can you give me some examples?
No. This is just plain false. Having used dialyzer since 2016 I can tell you it's not useful as a substitute for statically checked types via a compiler, with real strictness (which includes "When I don't have enough info, that's a type error").
> This might not catch all the errors because of the nature of messages, but this is a pretty small part of a system usually and can be covered by tests.
Not only will it not catch obvious type errors, it will also report false types when the core team doesn't use dialyzer in their libraries (because it doesn't work). These will bubble up to you instead and while you're mad they're not using dialyzer you'll still have to admit you understand why they don't.
I'm not sure what you mean by give you examples of how Elixir type specs aren't useful for domain modelling. They're ad-hoc, serve as documentation at best and even if they had structural power enough to express basic things they aren't checked at all, so you're getting none of the guarantees you would get in a statically checked language where your model changes and you can safely go forward by seeing what needs changing additionally to adapt.
What they are is how you use them, can you show how you can model something with static types and how it is different from dialyzer?
Could you explain where you think the Elixir type system falls short for modeling domains? Your post only gives examples of problems with type checking.
The most potent data modelling tool you have in Elixir is a `struct`, which is just a `Map` with `atom`s for keys. You can then associate this map with an ad-hoc `atom` tag in some return value if you want, and that's as far as we can go.
Modelling behavior in Elixir is easier and better, with processes, but having absolutely zero guarantee that what you're doing makes sense when it comes to data hierarchies and composition is just not useful when trying to outline what data represent and how it can be represented.
What about this?
-type int_or_str() :: {a, integer()} | {b, string()}.Static languages fell out of favour last time round the dynamic/static merry-go-round because the typing started to get in the way. You ended up writing more type boilerplate than actual code - and not breaking your methods/functions down into smaller functions to avoid having to write out even more type boilerplate.
I often wonder whether type annotations that allow a processor to generate tests that can integrate into your test suite would be a useful thing to have. Sort of like a C preprocessor.
Then you could add ML type headers to Elixir functions and get a set of ExUnit tests back.
We already have this in things like `Arbitrary` type classes, `QuickCheck`, `Hedgehog`, etc.. Hundreds of automatic test cases generated so that you don't focus only on "I thought of these 5 test cases" type of testing.
There is a PureScript compiler called `purerl` for the BEAM which can be used to write BEAM code in a ML derivative. It works and I personally would use it if I had to deliver BEAM code next week. For the most part it's the answer to the "you can only write BEAM code in bad languages" issue.
Maybe in a very strongly-typed language, but having used a pretty wide variety of statically-typed languages, I can confidently say none of them had strong enough type systems that the assertions made by the type system could be considered "comprehensive". You still need a test suite.
Static type checking does have some advantages, it can ease some of the low level testing burden. For instance, using a tool like Dialyxir (dialyzer) is helpful in elixir to give you that extra security of static type checking.
For me it’s always ended up being that I’m more productive with dynamically typed languages and the flexibility they offer.
Dialyzer isn't an answer to static type checking. It barely works half the time and cannot be relied on to check things exhaustively. It's so bad that even libraries created by core contributors have type spec errors in them regularly. Why? Because they don't use it, because it's not reliable.
Both `Ecto` and `StreamData` had these issues several times and I can only assume you could find many more if you were even more invested in using larger parts of the eco-system.
It’s not a false sense of security; it’s automation to cut out the stupid bottom 75% or so of boring-to-write, disastrous-to-screw-up test garbage that we all have better things to do with our time than duct-tape together.
(Also, Dialyzer is pretty bad by comparison to a real type checker. At best it’s spotty and inconsistent, it fails open in weird ways, and libraries rarely implement it. Elixir has an argument for “the least worst dynamically-typed language in common use” but Dialyzer ain’t it.)
I can tell you with 20 years of programming experience that static type systems don’t make me more efficient... and I’ve used them all.
And just because I can see the reply coming, I’ve worked with programmers using static type checking that have turned out horrible bug ridden code. It’s just not a cure all.
So can we please accept each other’s choices about type systems. “Can’t we all just get along”
Ultimately if it matters to you, then choose what works. But don't assume or try and tell everyone it makes one language definitely better than the other, because things aren’t that simple.
Then those meatbags realize that they have to refactor their code and the world ends.
The computer is infinitely better at crossing the t’s and dotting the i’s than you are. Or I am. Relying on conscientiousness when you have tools to measure and correct is a sucker’s bet. And I will go so far to say that with the wonderful tools available to us in 2020, dynamically typed languages put “on time” and “limited bugs” in an unnecessary tension.
You’re the type of programmer that tanks projects because of that arrogance and an inability to work with other “meatbags”. Maybe for you it’s all about code perfection, but for the rest of humanity it’s about making a product that works, solves a problem, makes the world a better place, or earns a profit.
Too bad my suckers bet has paid off with success and money. Best of luck with your attitude in life.
Guess these Github meatbags have made a suckers bet on a dynamic language. No way they could be efficient or successful maintaining or refactoring a project of that scale. (Yes that’s sarcasm for the sarcastically impaired)
Also most PLs/frameworks have very bad testing ergonomics, so that doesn't help.
You can't publically admit to "hating tests" because that has very bad optics. So, being a static typing zealot is the next best thing (after all, it will protect you from some classes of errors that are covered by testing). You pay for not writing tests in a bit of boilerplate, but boilerplate doesn't feel like you're writing code twice.
But I also like writing tests to prove logic and rules, and not tests for data validation--because computers are incredibly good at nitpicky stupid shit, they can and should and must do it for me, and I have more important things to care about than "did they pass an int when I wanted a string?".
Modern statically-typed languages elide a great deal of that boilerplate you refer to, while still providing protection against foolish errors. And given that the rest of it acts as a runtime-introspective set of documentation for what you're doing, the rest of what you're describing as "boilerplate" provides other value as well. (The web framework I use, for example, uses your declared data types both as HTTP validation and to generate OpenAPI documentation for both humans and other computers. In a dynamically-typed language I'd still need to define these things as JSON Schema or the like--which is what happens under the hood here, it just actually provides for me dev-time and not just runtime benefits.)
Basically all of these things (or their rough equivalent) are easily testable in Elixir.
Your point is taken in the large (I wrote a zig NIF library that reads zig types and automatically generates the code that performs the data deserialization from erlang terms), but in the specific case you noted, if you're using type annotations to create your OpenAPI spec, you're doing it backwards. Best practice is to OpenAPI spec to generate endpoints that you code for. (I'm aware that phoenix-openapi does not do this correctly)
Also, I want to point out that I wrote a generalization, not an absolute statement about typing zealots. I think you're rare. I like types and even wrote an (unpublished) typing library for elixir (proof: https://github.com/ityonemo/typed_headers) that engaged during tests, with the intent to make it compile time checks. But almost every typing zealot I have worked with absolutely hated tests.
The rest--yeah sure, but you can write tests for those things in anything. What you must test is then only a subset of the things that yeah, you do really have to bust your ass to test in an Elixir or a Ruby or a Python, and then it really is boilerplate--just less useful boilerplate overall.
> Best practice is to OpenAPI spec to generate endpoints that you code for.
I disagree with that. Every project where I've ever done that has immediately shat down its leg when the spec of an endpoint changed because refactoring from external sources is bad even in languages where refactoring isn't a disaster from the word 'go'.
IMO, the code is the canonical source of functionality; the spec should reflect it. It's why my CI system for my current project builds new releases of libraries for major languages are pegged to the version of the OAS3 spec being emitted. If you want to go off-road and use a language I don't actively look at, you can still do that, and the semver of the spec (which remains a manual problem in either case, though you could probably get clever enough to make a good guess as to whether it should be major/minor/patch) will tell you as a consumer when you need to upgrade.
A semantically versioned spec is required whether or not you write the spec first or the application first
> But almost every typing zealot I have worked with absolutely hated tests.
Slices both ways pretty easy, tbh. Almost every dynamic-typing pusher I've worked with was an active danger to a project with more than one committer--the best and most effective developers on such projects have always been folks with the requisite fear and acknowledgement of their own fallibility and the absolute danger that comes with using dynamically-typed languages for anything you care about; the starry-eyed Ruby idealists (I had that phase too) are just going to be too clever by half and then you get to eat the resultant shit later.
How does pagerduty deal with it, I wonder.
And what use cases?
I heard Elixir & Phoenix are great for web apps.
1. Expensive to find talent. 2. Deploy story was kind of shit until recently with mix release. 3. In Rails or .NET you can find a package for literally anything you can think of. In Elixir-land, it's not 100%. It's more like 80%. One example, there's a Twilio elixir lib, but bad luck one of the calls we need isn't supported by the package. In rails it's all supported.
---
But on the other end of this, it's all rainbows and happiness. It's fast, predictable, boring. I love that about the language.
I still use Nim for hobby projects, but anything professional I will write in Elixir.