Why engineers can't be rational about programming languages
spf13.com
spf13.com
Then they turn around and claim that choosing a programming language is the most important thing you can do, and that you'll need to Like and Subscribe to learn more about it...
I've been through tens of rewrite projects, successful and unsuccessful, and seen projects and products at almost every scale, and I cannot agree that programming language choice is a primary driver in a product's success or failure. Even extending this thesis from language to framework and ecosystem, where there's perhaps a _tiny_ bit of signal, still doesn't really lead to a meaningful conversation. The main driver of a project's success is almost always driven by: the composition of employees working on the project, and the competence of the people architecting the project. Don't get me wrong - to an extent, some languages (especially more niche ones) drive hiring and what kind of employee you get, but this effect is dwarfed by who works on the project and how well it's managed.
Like, my team doesn't know anything about Java, but we COULD ship in Java if forced to. We can't ship if the feedback loop is a 30-minute CI pipeline because there is no way to have a local dev environment.
Just as heads up for others.
"Rock bottom is deeper than you think!"
This is my experience too. I’d go a bit further and say the leads are the primary driver of success. Because ultimately, if the composition of the people on a project is incorrect, it’s the lead’s responsibility to realize and change it.
> Don't get me wrong - to an extent, some languages (especially more niche ones) drive hiring and what kind of employee you get
In my experience, the community around a given language is going to significantly influence the sort of typical applicant you get for a job working in that language. Those profile vary a surprising amount, especially for, as you say, niche languages, but also for "beginner" languages.
I have seen businesses significantly harmed because they hired what I would term language specific technicians instead of engineers. That's a failure of leader, certainly, but that failure is a lot more likely for certain languages.
I have seen this too, and I really like the way you phrased it - I think I'll use that in the future!
I do think it's an easier trap to fall into with some languages, but I still don't think the language really drives it.
I worked on a large-scale Rust project that could probably have been a Go project a while ago and while Language Technicians were a big hiring hazard, after we got one or two we both learned how to manage them and stopped hiring that type of employee (since they weren't what our project needed) and things evened out and were successful in Rust.
Yeah, in the end poor hiring practices drive it. The language you choose just makes the probability of that failure possibility higher or lower.
> I worked on a large-scale Rust project that could probably have been a Go project a while ago and while Language Technicians were a big hiring hazard, after we got one or two we both learned how to manage them and stopped hiring that type of employee (since they weren't what our project needed) and things evened out and were successful in Rust.
That tracks with my experience, for sure. Once you learn to spot it, you can mitigate it.
I've seen it. There are definitely incorrect language choices for certain projects.
It would be fair to say that these cases are themselves often exceptions. Many projects can be equally well accomplished by teams skilled in any language. But there is definitely a set of problems for which you can make incorrect language decisions.
I'm going to exaggerate to make the point in an attempt to avoid too much argument about whether or the language would be suitable, but: You do not sit down to write an industry-leading, high-performance database whose top-level implementation language is Python. If your project spec involves running code provided at runtime by users, Go is a fairly poor choice. You can make things a lot harder for yourself trying to be too insistent about what language you'll do your mobile development in, rather than just accepting that there's a very dominant choice in those spaces.
I've also seen projects I couldn't prove to you beyond a shadow of a doubt failed due to language selection, but I am fairly certain the project I saw that chose Scala failed primarily for the choice of Scala where it was a bad fit, both technically and for the skillsets of the engineers involved.
I've also seen projects nearly fail because they chose databases incorrectly, which I would submit is a fairly similar thing. Mostly because of choosing a NoSQL database "because fast" when they should have used a relational DB. The projects in question didn't fail because they were able to switch in time, but it was a close thing.
Part of "the composition of the employees of a project" being responsible for its success is that good engineers pick at least a decent solution to a problem from day one. The aforementioned DB problem, for instance, should have been obvious from the very beginning that it was not the correct choice in their case. There are absolutely wrong choices, that can crash projects both quickly and slowly.
Don't go running around telling people that they can dig the Panama Canal with three toothpicks and a spare weekend, and if they fail, well by golly they just didn't have enough grit and gumption like us awesome folks who could have done it with only two. Tool choice matters. In fact I can hardly process how anyone can be an engineer and think that it doesn't, let alone how they can think it's some sort of engineering wisdom to claim that it doesn't matter what tools you use to do a project.
Of course, picking the tool is only the moment the project may fail. It is not the moment the project succeeds; there's still a lot of using it correctly that will be necessary and plenty of further opportunities to fail even with the correct tool. But at least success is within the range of possibilities. You can forstall that possibility entirely on day one with incorrect tool choices.
Just to be clear, I wasn't trying to claim this; tooling certainly matters, at the very least, for the happiness and welfare of an engineering team! But, the article tries to claim things like "choosing a programming language is the single most expensive economic decision your company will make" and outside of a few extreme edge cases, I just can't agree with that particular thesis. Even the examples of bad decision-making you pose in your sibling comments, like writing a database in Go or "almost failing" by using sketchy niche datastores, are actually examples of this exact thing: these projects made huge engineering mistakes only to achieve some level of success as a business. Would they have been more successful if they made better engineering decisions? Possibly, but again, language and framework just was not the most important decision or factor driving an outcome.
I'm not saying that means we shouldn't care about making good engineering choices; there are easy ways to do things and hard ways to do things, and certainly I'm going to advocate for and work with people and at companies that favor the easy ways to do things. But when it comes to overall outcomes, I'll stand by having seen far more projects sacrificed to analysis paralysis, rewrites, rewrite-related hand wringing, and language/tooling hubris than sabotaged by poor language and framework choices.
But is anyone REASONABLY competent going to do that? They might pick C/C++, Go, Rust, Java, etc. Those aren't a choice between "bulldozers and toothpicks" - they are more akin to choosing between Caterpillar, Volvo or Hitachi as the vendor of choice for construction equipment. They may have some gaps in their specific list of equipment, they may charge too much for one specific tool, your workers may have experience with one, not the other, etc..
Your example should be the textbook definition of a strawman argument..
Which is, I think, the key point of the OP's article.
And my response to this is:
We need objective, scientific, real-world measurements of PL efficiency, so that informed judgements can be made free of cognitive bias.
This is how science works. Especially medicine. But programmers seem to feel that they are above this stuff.
No, programmers are quite below this stuff. Budgets for medical research, treatments, trials are way more than IT budgets. And for typical IT project the underlying point is even if requirements were wrong, changed halfway , software is malleable enough, to be changed, refactored. And all while it can remain in use even during change cycle.
it is (almost) always people and (almost) never language/framework/…
One argument I would like to add to this original debate is that I have observed two types of developers: one type tries to stick to one programming language for everything (e.g. Java) and tries to write everythin in that language. They are specialist and they may accrue deep knowledge of the language versions, APIs and IDE(s). Another type of developer, in contrast, maintains active knowledge of a dozen programming languages (C/C++, Java/Kotlin, Python, bash, Rust, Go, PHP, JavaScript, ...), and is capable of delivering projects in each. They'd pick a language suitable for the task and stick with it for a project. They won't know any single language as intimately as the first type, but they benefit by virtue of their choice of language being more appropriate to the given project.
I guess we can all agree that writing your web application using a fortran framework to generate JS code is a bad idea.
But if you pick tfa's second example, picking Go vs. Rust for a new project, the language choice is secondary. Both languages were likely fine unless the project as a specific library requirement.
The main criteria to make the choice was likely whether the team had developers with some experience in that language, and whether using that language would make them feel dead inside in the morning when they check in ; and I'm pretty sure developers can be found that make either choice a great choice.
The point tfa's making, that picking a language defines culture, the hiring pipeline etc. is fitting neither the first example (team already there, and a rewrite is almost always a bad choice) nor the second example (team also already there, and the culture with them. Pipeline therefore irrelevant).
You mean they're writing their own database? Why? That's a huge job and available databases are pretty good. There are multiple open-source choices, all of which work.
If they think they're going to compete with Oracle, they need to read the history of Oracle.
They're just not competing with Oracle.
Of course, almost no one should attempt this. The number of people with the technical expertise to pull it off successfully is much, much smaller than the number of companies with workloads that would benefit from this.
It doesn't have to be an exotic workload. Sometimes the market is just full of weak implementations e.g. graph databases.
That being said, some programs can only be written in one of those. Browser code is JS exclusive, low-level needs C++, secure code needs not C++. Machine Learning needs Python and high performance can't use Python. Some Windows things need C#. Those cases are the obvious ones where there is basically no choice. Beyond those, it is mostly about the team.
Which of course ignores and contradicts the primary lesson of the chap who founded this company and pays for this website.
There is an entire book about it which documents the problems in lavish detail, and it's generally hailed as a classic and a must-read.
I had one time it was faster for the three of us to rewrite a web app in Rails than to make the Play framework accomplish what we wanted it to accomplish, even though none of the three of us had ever worked in Ruby before. Rewriting four months of work took two weeks because Rails neatly supported what we were doing and so much could be generated. We ended up with better test coverage too.
There are refreshing exceptions here and there but there's also something to be said about whether it's a good thing that frameworks don't dominate a certain PL (like Golang; everything is very readable if verbose but one does get tired of generating the same boilerplate all the time).
So, this goes for ~70% of all projects ever and ~95% PLs ever?
I would have been a staff Golang engineer, if not a VP of Eng even, somewhere at this point if I didn't fall in love with Erlang's concurrency & parallelism implementation.
I am kind of still offended by the entire IT area that we have multicore CPUs for what, 20 years now, and we still don't make true physical parallelism a first-class citizen in all of our mainstream PLs. Looking at you, Python, and looking at you, Golang, with your ability to crash the program for sending to a closed channel, and looking at you Rust for making async needlessly complex.
(RE: Python, yeah yeah, "GIL is going away next release" and stuff. With that tempo they'll catch up with Golang by 2050.)
Programming language alone should almost never be a big enough issue to force a rewrite, but if you already have serious other issues that force huge changes you might as well look at it at the same time.
For example, migrating a web app from a language that predates Unicode to something that won't require a bunch of scaffolding around every user input sometimes is worth it. Moving from LABVIEW to a real programming language that integrated with remotely modern development tooling was worth it. Switching from C++ to Rust? Probably not.
Counter example:
Weird. LabVIEW is a real programming language. And both LabVIEW and Rust make entire classes of bugs that you often hit in C++ go away, especially for concurrent programs.
> remotely modern development tooling
I would make the argument that it is the tooling, i.e., Git and Diff, that is ancient and not remotely modern. Continually demanding that anything worth source controlling is text-based, and even further line-based, is as antiquated as it gets.
Do you have examples of working version control for non-text programs, including the ability to combine work from two parallel workstreams (aka, merge)? I worked for a long time at probably one of the largest companies using non-text based programming and version control was never even close to solved, and my subsequent forays into the literature when I worked in the "no code workflow" space still came up empty handed for working solutions here. We always fell back to "try to represent the structure of the change in some text-based way."
No one ever names their wires, so there's no variable names anywhere. To see the name of a subvi (think function), you have to hover over it. Sure there's a little picture, but that just means instead of just merely naming my functions I have to draw a little picture. Don't even get me started on wire routing, which is a crucial, manual process for any vi of non-trivial complexity.
Then there's the fundamental engine bugs, which National Instruments' response to hearing about is approximately "get fucked". Like, for example, the GUI rendering breaking in subtle ways when Windows has an update queued. Imagine getting a bug report that consists of "We can't do feature X", and trying desperately to figure out if your code is wrong, or if your customer's just an idiot, only to discover that 0.1% of the buttons in your project just... don't exist sometimes.
Then there's version control, which is rudimentary at best. In order to compare two files, you need to load both files, which even with lazy loading takes what feels like an eternity. Merges? No automation whatsoever. Every merge is manual.
Extremely useful features (like metaprogramming, which is important because of just how miserable writing LABVIEW is) are locked behind shoving obscure keys into ini files like "ABANDONALLHOPEYEWHOENTERHERE=1" or "SuperSecretPrivateSpecialStuff=true" (That second one is real).
Don't get me wrong, LabVIEW has its place as a way to hack together a little control panel to talk to some instruments. But anyone suggesting adding a subvi should be shot, and the project should be rewritten in a real programming language.
That's exactly false. I've single-handedly developed multi-thousand VI projects on different systems, including heavily using the OOP system and actor frameworks.
Comparing language grammars on their own isn't enough any longer.
This and similar are common ideas for the people that never see the real whole world of programming, and maybe have the fortune of be in the "startup" circles.
I see the opposite, and is very good predictor to know how bad a product or a team is, using the programming language AND the main DB engine, but that is because I live in the world of "enterprise" code where for example:
* I'm called to do a rewrite
* I see the screenshot of the main app
* I guess correctly was made with vb (first big alarm) (how I know: I never see in my circle anybody that do vb, php, c, c++ anything resembling a sane UI. BTW just the use of colors was enough to guess)
* I worry, but confirm, that use Access as the main db
* I discover that part of the data was ALSO in a excel file, that is used with the equivalent of "joins", and was not surprised to see things like this
Even without knowing more about the people that do it, that is far enough signals to guess much.
BTW, there are very good predictors, if Use: MySql, MonGo, Php, Js (almost whatever you wanna add here in terms of frameworks), VB, Perl, Android (aka: Java android and android itself without using iOS alongside), is likely terrible. Then Java or C# taking turns how much worse, but not as bad as the ones before. I sweat if somebody say it use C or C++. Probably enough to straight refuse to take the project.
Any use of not-obscure tech in this sector and is a good predictor to be more or less not-that-bad.
BTW: Also complex infra and related boilerplate is now probably a stronger predictor after some langs like python, go, typescript and more modern java/kotlin/c# has spread (and also more pg and much less nosql, but too much "cloud")
Yes, you may not enjoy turning a legacy system that works into a nicely architected system, but the ones that get to that phase are clear successes IMO.
Contrast this with systems which had nice, clean, maintainable architecture from day one but bit the dust two years later.
The original article is a silly shill, as engineering managers have looked at economic cost of choosing a language and other technology since... forever.
And some of that "invisible" discussion happens visibly too (I've done that a number of times: "how do we keep our engineers motivated who want to explore a new hyped tech stack vs the cost of them being slower or leaving the company").
And because time to first release is often the difference between prosper and fail
It is not enough to be the best. You need to be the first
> the CTO couldn't be a first rate hacker, because to become an eminent [Windows] NT developer he would have had to use NT voluntarily, multiple times, and I couldn't imagine a great hacker doing that
Maybe first rate hackers don't play games. /s
If you look hard enough, you'll find exceptions of course. I think Ebay was built on Windows servers, but in the late 1990s, building your web startup on Windows was a sign that your technical leadership had bad taste and a competitor would run circles around you.
https://discord.com/blog/why-discord-is-switching-from-go-to...
A possible reason for this is that our current languages are way too similar to make a difference.
Even most of the ones we think of as radically different.
To some extent! There are also cases where any decision is better than no decision, and all the options are good enough that it's not worth the delay to argue about them.
Where does that fit, in a company where there's only one techie? Not sure I get your comment here.
Also "good enough" is a very, very tempting and treacherous mistress...
Having that said I agree that language/platform is a "non technical requirement" in 90% of real world cases. You pick what you know or in more industrious scenarios - what's available on the market or what's the most cost effective.
But people are indeed irrational about programming languages. There's tribalism, stereotypes and preconceptions. Most notable is probably PHP, language for human failures and shit projects. As if same exact project written in Java was suppose to be of higher philosophical value.
But it's something you can't deny. I've been asked in non ironic way by a (non technical) founder investor if I could recommend him Rails programmers, because he read about it and it's suppose to be great. I asked him about specifics of his new project and he said he doesn't have an idea yet, but it has to be in Rails. Go figure.
There are of course a few scenarios where changing the programming language is a more defensible, less "always wrong" kind of thing. An extreme case would be something like a COBOL system that needs maintenance and you have trouble finding people who can do it.
I would argue that the number of successful companies with Python code bases is the best evidence that language choice doesn't matter.
I think a lot of the problem of switching isn't so much the language, but relearning all the undocumented lessons that were learned the hard way the first time around.
When Java came along its garbage collection blew my mind. I agree with the 4x factor.
- Systems languages with manual memory management, like C or Zig, where real-time/low-latency performance is important.
- Rust and its borrow checker, as an alternative to manual memory management.
- A strongly FP influenced language such as a Lisp or Haskell. Especially Lisp macros. Exceptionally good for working with structured data in cases where purity is more important than performance.
- The BEAM/OTP architecture for distributed systems (Erlang/Elixir).
- Languages with good CUDA/PTX/Vulkan support, for programs that need the GPU.
- Assembly. This was a much bigger deal in the past before compilers got good. Today good mostly for educational purposes.
I've rewritten a very large and complex macro assembler program into C. The original developers were gone. Nobody would touch that assembler code. I volunteered. The result was a program that could be maintained and could be ported to multiple diverse platforms needed by the company.
I tried to port Optlink (a linker for Win32) from assembler into C. It failed because the market for Win32 programming died, and so I abandoned the project.
My Empire game started out in BASIC. Then it was converted to FORTRAN, then PDP-11 assembler, then C on the PC, then D.
With that pov, I don't see any contradiction by saying the language is an important decision, and rewriting your project in a new language is probably a bad idea.
Personally, my bias is towards the languages I'm most comfortable with. I recognize this and will make other suggestions and if I'm not responsible for the code, I'm more than flexible.
All the fad chasing and top down declarations that we're all going to use Cucumber, GraphQL, Microservices or anything else is often a bad move.
First learn the problem you are trying to solve and empathize on behalf of the user... Then empathize on behalf of support... on behalf of the maintenance developers... on behalf of yourself in a decade. Is there a boxed solution? Buy it.. Is something custom really needed, what can you outsource as part of it? integrate it. Do the simplest, easiest thing you can to get the job done.
I'm sorry but I disagree. Languages are tools, pick the best tool for the job. The idea that languages are all good at everything is not true. And when I see takes like this, I tend to think that that person just doesn't understand how to assess a language's strengths and weaknesses.
Want to write ML, probably best to use a language with functions as first class types (ie a FP language). You might say, most people doing ML use Python. This came to be because the language was picked based upon what people knew. But the big companies doing ML (successfully) don't use Python anymore and haven't for over a decade. ML researchers kept FP alive for several decades when nobody else cared because FP is the best tool for ML (or for writing a compiler). Where the FP folks get into trouble is trying to push FP where it doesn't make sense. I see this pattern repeating over and over again. Languages are pitched as silver bullets when they are just screwdrivers and hammers.
Right tool for the job, ignoring this leads to "religious wars" because that's how we describe disputes which are matters of taste.
Also, you are massively overvaluing expertise in a given language. A more talented engineer who doesn't know your favorite language after a couple of months will be better than you in your favorite language too despite your greater experience with that language.
> ... or there are underlying reasons for a language shift.
As to "best language" that is just as dogmatic as anything else... just look at the C/C++ vs. Rust divide in the Linux community.
I think you are overestimating the value of a best fit language for any given task, especially those where there are a half dozen popular languages that more people know well that can do the job good enough. Don't build for a sky scraper when all you need is a birdhouse.
Also, MOST engineers aren't particularly talented. If you're fortunate enough to be working for an organization where everyone is a rockstar, that's great... for those doing bog standard CRUD apps for business, you don't get rockstar money, and you aren't finding rockstar talent. You get what you get and make the pest of it.
In nearly three decades, I've once, only once worked a project where I didn't have to explain a relatively simple concept to someone, where everyone on the project delivers their pieces in time and all were talented. It was wonderful. Then new management gets stacked on top, all the job roles are reclassified to mid level developers and everyone rolls out of that group.
A lot of the actual experience is literally explaining public/private key usage to other developers who manage to (re)use the same keys from dev to all the production deployments. Or a pissing match with the "security expert" who doesn't understand that your app's use case is different than the in the box security script that is failing, because your /login route is a different app from / and the bogus query params don't matter.
I strongly disagree. The top AI companies are using a lot of Python (although sure they also use other languages too, but they're definitely using Python!). Even if by ML you mean old-school ML techniques, a ton of big companies also use Python for this (some might use MATLAB or R).
On the other hand I don't know of a single large company using an FP language for ML, unless you count something like Spark (which I would push back on: the Scala API of Spark is not really FP and almost all users of Spark that I know of program mainly in the more OO part of Scala rather than its FP part).
Even die-hard FP companies such as Jane Street use Python for their ML (see e.g. https://www.janestreet.com/join-jane-street/position/4276720... which notably mentions Python and does not mention OCaml).
Do you know of any company with a team of over 50 ML researchers (either old-school ML or modern AI) using an FP language as their primary workhorse for that team? Because I can't think of a single one.
More to the point, do you know of any ML researcher (and who is acknowledged as primarily an ML researcher by other ML researchers) who primarily programs in an FP language?
Even in the golden era of symbolic AI from the 70s and 80s they still weren't using FP languages (they basically didn't exist yet). The closest you could say is that they were using Lisps, but Lisps aren't FP languages by default. Some are (e.g. Clojure), but many aren't or at least aren't any more FP than any other multiparadigmatic language (e.g. Common Lisp). And again I don't know of any significant ML work being done in Lisps at this point (there's some scattered small teams and individuals doing work there, but nothing that I think could rise to the level of "big ML company").
I fully agree. The challenge is, some will want to use the latest languages and technologies because they want to learn it (personal development, meaning: the next job). Sometimes the "new thing" can be limited to (non-critical) testing and utilities. But having many languages and technologies just increases the friction, complicates things, and prevents refactoring. Even mixing just scripts with regular languages is a problem; calling one language from another is similar. The same with unnecessary remote APIs. Less technologies is often better, even if the technologies are not the best (eg. using PostgreSQL for features like fulltext search, event processing, etc.)
This is a bit related to external dependencies vs build yourself (AKA reinvent the wheel). Quite often the external library, long term, causes more issues than building it yourself (assuming you _can_ build a competent implementation).
I feel like this happens mostly because simpler is better, and most of these dependencies don’t follow a good “UNIX” philosophy of modularity, being generic etc. something that you’d notice the standard libraries try to achieve.
Most of these third party dependencies are just a very specific feature that starts to add more use cases until it becomes bloated to support multiple users with slightly different needs.
Python: Python is almost a hard-compiled language. Most of the dynamic stuff that's really hard to compile isn't all that useful. But Guido and his enablers love the dynamism, and the CPython implementation. So instead of PyPy taking over, we have CPython with hacks to call C.
Go: The "share by communicating" thing in Go works out about as well as it does in other languages, that is, it's useful but not central. Early on, there were tortured examples of implementing locks with queues. Nobody does that any more. People pretty much write Go like they do other languages, with shared state and locks. Queues are used when queues do something useful. The real strength of Go is that the libraries needed for webcrap are maintained and used by Google, so they're all well-tested and exercised. Also, goroutines/green threads eliminate the sync/async distinction. Garbage collection takes care of most ownership problems. Simple. (I recently wrote a web back end in Rust. Big mistake. Should have used Go.)
Rust: The "traits" system is an overreaction to Objects Are Bad. Rust probably would have been better off with single inheritance, which is well understood. (Multiple inheritance has too many dark corners.) People keep trying to do OOP with traits, which is like pounding a screw. Rust still doesn't have a good solution to the back reference problem, as I point out occasionally. The macro language sucks, but then almost all macro languages suck. "Async" is a nightmare but necessary to keep the Javascript crowd happy, since that's all they know. If you really need complex multiprocessor concurrency, Rust is currently the best game in town. Most people don't.
C++: They can't take anything out, and the cruft is too deep. "Modern C++" is not all that bad, but all of bad old C/C++ is still in there. So the safety situation remains awful. The cumulative complexity is now so high that even long-time language lawyers are giving up following it.
Javascript: Who thought that would rule the world? It's awful, but everywhere. Heroic efforts have made an inherently slow language go fast. It's kind of impressive, actually.
> Rust: The "traits" system is an overreaction to Objects Are Bad.
It's interesting. Interfaces, traits, and mixins are all OOP concepts.
> "Async" is a nightmare
In what way? I do agree though, that it's annoying to yet again have a new language not have concurrency baked into the language. In some ways, Rust has excuses, because they want their concurrency primitives to support microcontroller, real-time Linux, and general purpose programming.
Go: Greenthreading is a big deal in backends. Arguably the main reason for Kotlin is because Java didn't have that, but now there are vthreads. Rust has chipped away at the Go systems use cases, so it's mainly for backends and CLIs now, but I wish it had better error handling.
Rust: There's a very good preso from the Rust team about how they arrived at async/await, and also how every other language does concurrency. Greenthreading was considered, main problem being you need a runtime for that.
C++: Torvalds was right about it all along.
JS: Honestly the best high-level language, made better choices than Python, never made huge breaking changes, somehow had a decent answer to cooperative multitasking before most other langs, deserves its popularity.
I'd have to disagree, it really isn't, and I even think that's kind of the point the article makes.
Rewritting existing services from scratch in another language can often be a bad decision, because it assumes the choice of programming language is an important one. And any rewrite is costly, doesn't matter the reason why.
But starting a new project in a new language I don't think has much impact generally. And if it motivates the team, because they're excited about it, it can even help.
Deciding later to rewrite this once it's been built because it's not in the same language as what is common at the company, that's likely the mistake that will happen.
NOT because a good leader will save $$$$$$; because a bad leader can single-handedly sink a ship.
- Donella Meadows, Leverage Points: Places to Intervene in a System https://donellameadows.org/archives/leverage-points-places-t...
If you're a Python Programmer, and you've made that your identity, you've trapped yourself into the mindset that you are a Python Programmer. Same with any other identity you ascribe to yourself (or allow others to ascribe to you). Separating yourself from your tools allows you to evaluate the tools independently of your identity and you will find yourself unaffected (or at least less affected) by reaching a conclusion that the thing you know well may not be appropriate to the job. You may not be appropriate for the job, and that's fine too. Getting past paradigms (or identities with how I'm extending it) gives you much greater freedom to explore and participate in the world.
It's not me who has problems getting past paradigms, or anyone else I know for that matter, it's the recruiters and HR people who screen resumes by only ticking boxes on buzzword.
Generalizing briefly, the same phenomena of identity underlies a lot of our religious wars. Be it language, braces, indentation, or a variety of other programming choices. What's fundamentally going on is that programming expertise is fragile. (I think I first saw that idea in Code Complete?) A new language / style / technology / whatever very often will leave us less competent. Ideally we would respond with, "I guess I need to get back on that learning curve." But often it is easy to instead blame the external factor. "I'm a good programmer. I tried it. The result didn't work well. It must be bad."
Among the many attempts to try to fix the problem, I can recommend https://blog.codinghorror.com/the-ten-commandments-of-egoles.... To the extent that you manage to apply its advice, you really will do better.
Of course your improvement won't directly help those around you...
This may be a big factor in rejecting unfamiliar languages. Over time the brain trains itself to grok a specific syntax, and understanding becomes partly automatic: we look at a Java program and our brain injects meaning into our consciousness. If we then look at APL or Lisp, however, that training on Java doesn't apply and the automatic injection doesn't function. We're left having to read the symbols directly, and it's unsettling not having the auto-assist. It makes us feel we "can't" understand the language, when it would likely take a couple weeks of immersion to change that impression.
If you find yourself searching for a replacement language more frequently, you should stop, take a long look in the mirror and ask yourself:
1. Something has clearly gone wrong last time, since you're looking for a replacement so soon; are you confident of your language-picking ability?
2. Are you sure your goal is to do what's best for the software and its long-term maintenance, or is there some other consideration here?
Worse, this also falls into the trap of thinking you have to make a choice, and that an informed choice is better than chance. Reality is that you are often best trying to do both, and then having some sort of coin flip to disambiguate choices works far better than we'd like to admit. In large part for the simple reason that simply doing something is more than a lot of your competitors will be doing.
By far no. Now I don't know if I even should read beyond that.
If a company chose brainfuck as a main programming language, it's doubtful they'd come back from that choice.
Based on what? Economics define where you go, not language. If company using Perl or VB has steady cash flow and their bottleneck is language - they’ll just rewrite when it makes sense. No amount of writing in C# or Python from scratch will save you if your product is garbage.
Same thing with the fundamental architecture of programs, and especially the data model & database solution.
Still I don't entirely agree with the article. He makes it sound like there isn't any difference between programming languages and any preference is purely about developer identity. But that identity doesn't come from nowhere. Rust is popular because a load of C++ developers finally found something technically better. Not because they all woke up one morning and decided to be "a Rust developer".
Where the story falls apart is that Perl is arguably even worse than PHP. One deluded Perl programmer does not prove a principle.
On Rust vs Go, he's absolutely right that Go is has a slightly better "build & deploy" story (though not by a much). But reading between the lines I think he's misrepresenting that - it sounds like a) that was just one point for choosing Rust, and b) that was a point when comparing to typical alternatives, e.g. C++ or Java. It's not untrue that Rust is easy to build & deploy simply because Go is slightly easier.
What about them?
Extremely costly choice would be choosing .Net or Java and never releasing at all. Creating Hack advanced whole PHP ecosystem and cost nothing in the grand scheme of things.
> Dropbox had to rewrite their sync engine in Rust, and I think they also have rewritten backend stuff in Go.
Bah, idiots, should've just waited 4 years until Go and Rust came into existence.
I wrote a compiler/language, and I was expecting something different from the article after my experiences
Like many of the other commenters, I didn't like the article
The subject is right there in the title:
Engineers Can't Be Rational About Programming Languages
From the way so many commentators are refusing to engage with the point expressed in the title I think there is collective denial of the core point, so instead, people are discussing the symptoms rather than the disease that is the subject here.
As long as engineering salaries depend on tribal identity markers (i.e. language and tooling preferences) rather than ability to save money, people will entirely rationally choose tools that look good on their resume rather than save their companies money.
I was hired at a shop who had a large, complex Visual FoxPro application they'd developed in house. They brought me in to write a web interface for it, which I did in Python because there was, thankfully, no Visual FoxPro for Web Apps.
I contented that they needed to start a rewrite in something else ASAP. VFP wasn't going to get more supported as time went on. On the other hand, it was a large, working app, and the pushback was "yeah, someday, but for now we're using it to drive the company".
The discussions were always calm, measured, and conducted by adults. We didn't shout or scream at each other, or anything like that. Both sides had reasonable and compelling arguments to support their viewpoint. But from my POV, staying with the VFP app, with the looming deprecation warnings on the calendar, was insanity. Their point was that rewriting the entire, working app from scratch was also insanity.
It's not like a team wanted to rewrite it in PHP, which we can all at least agree would be madness.
That seems like the same argument that TFA is making, but in reverse. I think it's just as invalid. PHP is a perfectly fine language for many things. There is plenty of greenfield development being done in PHP these days + a substantial amount of PHP in production all over the place. Unilaterally writing it off as "madness" seems pretty disingenuous to me.
For some tech choices there was just one clear path from where we are to where we need to go. But for others, the overriding decision was more often than not that some team member had familiarity with the technology. Certainly not the only criteria but enough to tip the scales if it came to that.
That was an unusual scenario, however; the previous iteration of the product had major issues, buggy and costly to maintain, and the language change was in part for the purpose of repointing the hiring pipeline toward a different kind of profile. Such special circumstances aside, I'm with you on recommending against rewrites. Much as I'm often tempted.
If that produces many options, then choose based on the team's familiarity with the language, frameworks, and ecosystems.
A good software engineer should be pretty language agnostic when it comes to work, even if they have a personal aesthetic or technical preference.
> "the framework that best supports the business case and problem you are working on"
by asking the people who will be using it, then the answer will almost certainly be wrong.
The whole point here is working out which is the best tool is almost impossible because of natural human cognitive biases.
TL;DR summary:
No. No, that isn't the answer. It's not even an answer. It's the answer to a different question.
There's another almost-got-it in the article. He is suggesting people tie their identity to their programming language of choice. This seems odd to me, because we tend to think of identity as think like religion or ethnic group, not sub-professional groupings like neurosurgeon or devops, and certainly not specific to tooling ("I'm a DeWalt carpenter!")
The missing connection is back to economics. If I've spent a bunch of time coding .NET, it's going to cost me something to code Java or python. This is the actual economic conversation being had. People will have to learn a new toolset, while having deadlines over their heads.
The solution is actually this: You hire people who are language-agnostic.
I used to spend a LOT of time in VBA, and then .NET. I was daunted by making the jump to C, then c++ and python. Only over time did I overcome the nerves and move on with Java, Kotlin, Elixir, js, and Rust. It was an eye-opening experience that you could do this. Just make the leap, and you find it wasn't that far.
If you don't have people who have done this a few times, you will get resistance, and the real resistance is they don't think it's worth it to put in the hours to learn the new tools. They come up with all sorts of justifications for why things are fine with the old tools.
You'll also become a language lumper if you do this. I don't really think there's a lot of language features. The main ones are GC vs manual memory, strong vs weak type, imperative vs declarative. Once I had a few points on these axes, a lot of things eased when it came to new languages.
It sounds as if you did skip C++ and moved to Java instead. If so you serendipedously avoided the one language that's likely to cause problems. C++ doesn't work like the rest of the languages on your list, and it really is as full of land mines as people say - even though, with a good process, evidently it can be managed.
I'm not sure that's a correct analysis of their situation. Yes, if you hand a perfectly spherical DeWalt screwgun to a Nikita carpenter, they'll know how to use it, but in practice, everyone I know in the trades has picked a team and just committed to it (to the tune of many thousands of dollars in tools). Fortunately for them, the tools are basically the same, just a stupid bit of plastic (and there are 3d printable or buyable adapters) so it's easy enough to switch teams if they had to, it's just a pile of money, unlike sw tools and materials where I mean, yeah, eventually I got the hang of Ruby, as I did C++ and C and Python and Perl and now typescript and JavaScript oh god and also Swift. (Managed to avoid Obj-C tho!)
I think you have it totally backwards for >90% of people, and I counter your argument with a very famous quote:
"Dammit, Jim, I'm a doctor, not a bricklayer!"
I've seen this with React. You might have a great website working with some solid MVC framework + good'ol jQuery or equivalent, but no, the whole thing has to be rewritten from scratch to be an SPA in React, to "keep up".
The point of the article is that if you delegate this choice, the people to whom you delegate it will get it wrong.
If anything, maybe this says there is room for a Rust-like GC'd language.
[https://www.cnet.com/tech/services-and-software/takkle-com-s...]
then boom suddenly php, ruby & java are pragmatic choices in economic sense for business systems. same as c++ for hedge funds & video games etc.
Have they done in math in FTE salaries, times the estimated effort in days, mappend into money, and what is the expectation of ROI in that investment.
It got people to click the link to their think piece.
Management does need to care to ensure they hire people who will make the right choices (which is a careful balancing act of investment vs returns) if they don't trust themselves to do it.
PS I've seen PHP destroy billions in value in my career. It and Brainfuck are the only 2 languages you should never use to make software for other people. Every other language has a core use it is good at. JS in the browser, Python for scripting, Java when you need good observability and 3rd party libraries, etc...
Ummm... No, I think it was about forcing Perl on engineers who had been using PHP.
And similarly, "Python for scripting" (Instagram?)... It's the effectiveness you achieve with it that is perfect for many new projects, based on language itself (dynamism, introspectiveness, readability...) and rich standard library and ecosystem.
I'd say you are not looking objectively at language choices despite the evidence being there in plain sight to counter the claims you are making.
We do pick tools in good part based on how we feel about them (especially engineers who believe themselves beings of pure rationality, tbh), and I think that's in fact okay; how we feel doesn't come from nowhere, it's informed by decades of experience often acquired the hard way. But it's still a squishy metric that can only be trusted to a point, and being aware of how this sort of decision making occurs, in yourself and in others, is highly valuable IMO.
Sure, it’s good to be aware of future business needs so that we, as technical people, can be asking the right questions to prepare for what that future may look like, but that almost never means a decision about language x over language y. It’s much deeper than that.
That's an example. It is not the subject under discussion.
I also don't think it's necessarily bad that people do this. The input to any decision a person makes includes their entire life experience up to that point[1]. How could an executive encode all that in some kind of pat logical explanation, and how could the also-human engineers at the company possibly digest such an explanation, and what could make it more compelling to them than their own life experiences? People need to get through life, though, so they need to make decisions. They can't fully rationalize every single one, but they want to feel at least OK about the decisions they're making, so they tell themselves and each other these incomplete little stories and get on with it. That managers scaffold this process with their own stories is a little manipulative, but how else could people cooperate enough to have companies? The whole process just seems intrinsically human to me.
The most important part of being an executive is understanding all of this and choosing to hire people who will ultimately make good strategic decisions for you. Don't hire a well-known Perl contributor as your CTO unless you like the idea of rewriting your product in Perl. If your company is dying because this has happened, my condolences but at least you're not alone.
Edit: I hadn't read this far when I wrote the comment but the author also literally says, "The moment you hire a Rust developer to evaluate languages, you’ve already chosen Rust." I guess I just disagree that it could work differently. Each of us possesses bounded knowledge and bounded rationality, and "which language is best", is probably too complicated for an individual to figure out, especially when you don't even know what the roadmap will be in a year—you'd have to build the company several times in several languages and compare the results (and the best engineers I've met do write code multiple times, but rarely more than twice IME). Each of us can only really know how we would solve the company's problems. Executives' job is to try and guess, and make decisions that are ideally optimal but at least internally consistent.
[^1] My favorite example of this, actually: even in the highly-rational field of scientific research, scientists have to decide whether a given body of evidence is dispositive of a particular theory, and the standards they apply likewise depend on who they are and what their life experience is. So, as Max Planck put it, science advances one funeral at a time.
However, we continue to write new code in C++ due to libraries and because people know it. These are all factors that the OP considers to be negligible.
The conventional wisdom was "Where will we get C++ programmers?", "We don't have experience with C++", "C++ is too bleeding edge", and so on. The same excuses people give today to not use Rust, or your favorite hyped language.
If we follow the logic of OP, we will almost never develop new languages, because there are already multiple established languages good enough for any task at hand.
- Look at people using identity-focused reasoning with politics
- It’s research!
- That’s the same as those <programming language tropes>
- Because I have some anecdotes about that
But the person making the argument couldn’t be falling for the same thing? Nope.[1]
Maybe it would be nice to have more objective metrics. So what’s that?
> We need a framework that makes the invisible costs visible. One that lets us have the economic conversation instead of the identity conversation. One that works whether you’re choosing your first language or evaluating a migration.
> Our industry has never really had that framework… Until now.
I better Stay Tuned.
[1] Those irrational people: making arguments. Me, the rational one: also making arguments, but mine are correct.
Hardly, no.
The entire point of the article is pointing out an entirely different cognitive fallacy.
At the very least the title is clear Bulverism.
Admittedly, I did enjoy the article, even if I was initially a bit predisposed not to from the title. And I think the quote "if you hire a rust evangelist to choose your programming language, you've already chosen rust" is very insightful. But the author still goes into zero depth to asses to what extent rationality of language choice is a widespread problem, as opposed to isolated cases they've seen during their career.
Which is ironic given that their thesis is that people go into topics emotionally without weighing them factually against the ground truth.
AFAICS you don't seem to have even noticed the initial example.
Engineers often aren't rational because engineers can still be stupid. Dogmatism/black-and-white thinking is often a sign of low emotional intelligence (and can also be a defense mechanism called "splitting").
The Dunning-Kruger effect also applies to "smart" people. You don't stop when you are estimating your ability correctly. As you learn more, you gain more awareness of your ignorance and continue being conservative with your self estimates.
I mean, its largely a statistical artifact around which a pop science myth has accumulated, but on its own terms it applies smoothly and continuously across the entire range of ability in a domain, not in any special way just one one side of binary knowledge dividing line (the finding was basically that people across the whole range of ability to tend to rate their own relative ability closer to the 70th percentile than it actually is, but have monotonically increasing with actual relative ability.)
We understand this when we talk about the irrationality of the language flame wars. Back in the Slashdot days we joked about it explicitly. So in that tradition: I don't want to start a holy war but...
You've got all the ingredients. First and foremost the BDFL. He's the cult leader, coming in with a new "way of thinking" which, if you follow his rules (the language) will lead to a better life (better code, better career, personal enlightenment, righteousness, purity, becoming closer to the machine (god)). Often times this is expressed in terms that even he cannot quantify -- the language is "efficient", "advanced", "secure", or "modern". But the point of the language is, if you only adhere to these dogmas (everything is an object/table/function, everything must be immutable), then your life will be filled with fast/efficient/secure code (again, can't really quantify it).
Some of them even sell you a "purist" cult -- "we don't sling dirty imperative code here, our code is pure functional, which is closer to the ideal (god/the word of the BDFL) and therefore good". Lispers figuratively go there saying things like Lisp is the language of God. HolyC (TempleOS) literally goes there with claims the project is a word from God. Now, there's mental illness mixed in with that (but would you believe me if I told you many language designers suffer from mental illness? At least every one I've known. If anything, there's a good deal of narcissism and delusions of grandeur in this community, which incidentally you'll also find in cult leaders).
Of course like any good cult, aesthetics are enforced. You can't just program in the language, you have to program the way everyone else programs. Take the Zen of Python, which preaches a sort of asceticism. Some cults are about "beautiful" code, that looks "clean" and is free of anything impure like certain syntax or even specific characters that are thought to be a nuisance. The key to all of this is it has nothing really to do with the essence of the language. Just having these elements floating around is enough for people to glom on and stick around. You don't even need to have an actual language to form on of these cults, look at Jai. There the BDFL has held back his language for about a decade, promising to release it but never actually doing so, which paradoxically keeps people holding out to receive the forbidden knowledge he is keeping from them.
We've got our holy books of course. The language spec holds all the keys to understanding the word of the BDFL, although the number of people who have actually read it compared to use the language rounds to 0. Not only because these specs and docs can be voluminous, but also sometimes they are written purposefully to be inscrutable. See "Nock" and "Hoon". Half of understanding those things is just getting your head wrapped around the jargon, which itself is meant to be exclusionary -- if only the authors can read it, it puts them in a special place of translating the holy texts so that mere mortals might understand their wisdom (it also means that if anyone disagrees with them, they can just insist the texts are being translated wrong). Other times there are style guides which tell you the one true way to read and write the language. Then there are the various blogs and tutorials which are the equivalent of sermons, spreading the good word to the masses.
Speaking of the masses, that brings us to the community! That's us, we are the cultists! The community will defend the BDFL to the death, and attack anyone who questions the dogma. You see this all the time in language communities, where any criticism of the language is met with vitriol and ad hominem attacks. Usually the BDFL treats people this way, and his community sees it a sign of strength, so they emulate it. Although The community is often more zealous than the BDFL themselves, going out of their way to convert others to the faith. We see that from time to time on HN, when people show up here from some lang community and they get way too into it, to the point their lang is flagged here every time it's posted (not even gonna name them because they'll be here to say something). They will also police the community, ensuring that everyone adheres to the dogma and punishing those who stray from the path. This is often done through social pressure, other times it's through explicit excommunication (banhammer).
Which leads to sects! I mean forks! When a language community gets big enough, you start to see splinter groups forming. Tensions rise in the community around aesthetics, features, or very often how the community is being (mis)managed, how the BDFL is leading or ignoring users. Eventually you get a Martin Luther type who strikes out and forks the cult intending to create their own cult. We saw this with projects like Elm stagnating as the BDFL continued to bless certain packages but refused this access to the rest of the community, so forks starting to appear. Or look at the recent controversy with Nim and the Nimony fork.
And let me tell you, as someone who was a dev for a PL and saw how these things started off, you really attract... interesting folks when it comes to a brand new language. For the thing I was involved with, it was just one person after another who had some out-there idea thy wanted to glom onto the language. Stuff like "vorpal math"... whatever that is. So, yeah that's the kind of people you start out with in the early days if you're lucky, because no one else is paying attention to the niche PL scene except the vorpal math guys, who are primed to receive any kind of special knowledge from a cult leader. You collect a couple of these followers, they become true believers, and then you grow from there. It's not too hard, but it's very unstable, so most communities don't get too big as the BDFL loses interest himself, or he can't keep the hype going. Once in a while you get guys like Yarvin who really run a grift over a looooong period of time -- with Urbit they were literally selling virtual land plots to their community. So just like actual cults, there's also a grift to run when it comes to languages (languages are not profitable at scale, but they can be profitable if you know how to grift).
Now I'm not saying all languages are cults. But I think the fastest way to build a language from nothing into something somewhat active is to lean into the cult-like aspects to entice people in. Because otherwise the language has to stand on its merits, and small languages can't. So they have to stand on a promise of heaven that may or may not materialize (it probably won't), which is the basis of the cult. Because to keep the movement going, the cult leader either has to deliver nirvana, or keep promising it's just around the corner. Maybe it will be, but it's very easy to waste a decade of one's time hanging on to some lang dev waiting for him to drop the next update which will fix everything, but it never comes.
[0] you sweet summer child.
False. Next.
One day, many millennia ago, Caveman Ug carved himself a really nice club, and it became his favorite. He carried it everywhere and used it for everything.