My story on “worse is better” (2018)
sigbus.info
sigbus.info
Code is a tool for exploring and understanding problems as much as it is about solving them. Sophisticated solutions can't be designed before they are validated.
Part of the problem is that it's next to impossible to separate the facts from the religion in software domain knowledge. Software artifacts are simultaneously art, technology. math and religion.
It's also hard to separate the abstract knowledge that is broadly applicable from its original context where it was hard won: the particular operating systems, toolchains and tech stacks.
Even that knowledge which is separable is mired in the jargon of those systems, so that it looks like it is specific to them. If those systems are long obsolete, then it's easy to dismiss anything that is robed in their jargon as being obsolescent by association.
If civil or electronic engineering were like software, then every five years, someone would be reinventing a class B push-pull emitter follower amplifier output stage, or Pratt truss, under different names.
I've got some abstractions out there I wrote early on when I didn't think I coded well and they have lived for years and saved tons of time.
Others don't live long :(
So many times I've taken code that was a mess because someone tried blindly DRY code for the sake of being DRY and rub abstractions on top of it and I've reverted the code back to being simple copypasta and its become so much more clearer and robust. Then you can look at the result and a simple abstraction may pop out which can reduce enough code duplication that the result is satisfactory (it may require a bit more boilerplate in the subclasses or whatever, but nothing likely to be brittle under future fixes).
But an underly-DRY codebase... good luck. Copypasta tends to accumulate minor differences and figuring out if those minor differences matters can become very time consuming.
The DRY version of this is functions/constructors/methods that take a stupid number of parameters, many with unclear purposes.
I'd rather debug code spaghetti by rearranging copypasta to sort out the simple deltas than try to unwind LSP-violating inheritance spaghetti.
Good intentions at every step, but a Frankenstein's monster of a function (or entire class, sometimes) in the end
It's the DRY equivalent of having 20 slightly-different copypastes all over a codebase.
The Django monolith I worked in for some time was like this - 30, 30, 40 parameters on the methods to handle completing a purchase.
It is the aggregation of a dozen different developers each adding a knob to close a ticket/bug over the course of maybe 10 years.
It is better to under-abstract and keep some duplication than over-complicate and miss. Complexity snicks up on you (it is a time bomb).
Welcome to the field of genetics.
A person who possesses music theory can explain why they are composing something they way they are, and suggest better notes to someone who is trying to embellish the music with a harmonization or whatever. Just like the people who held the "theory" of the compiler were able to spot better ways of implementing the ideas of the second group without spoiling the design.
https://www.dreamsongs.com/WorseIsBetter.html
Over time, I have come to believe that the problem is overly-aggressive abstraction. It is very tempting for most developers, especially good developers, to reach for abstraction as soon as things get complicated. And this can pay off very, very well in some cases. However, too much abstraction in a system leads to a very, well, abstract code base that becomes hard to get a handle on: there's no there there. You see this in the java world with AbstractFactoryBuilderLookupServiceBuilders and so forth, and with very elaborate type setups in functional programming languages.
Concretizing the crucial bits of your system, even if that means a few large, complex and gronky methods or classes, ends up often making things more understandable and maintainable and, certainly, debuggable.
John Ousterhout wrote a book that makes this point as well, advocating for "deep" rather than "shallow" classes and methods:
https://www.goodreads.com/en/book/show/39996759-a-philosophy...
Sometimes, yes, overly-aggressive abstractions are a problem. But the author describes policies of v1 and v2 of the linker. And I would say the most critical difference between them is, that the authors of v2 had a better understanding of the requirements that actually mattered. Therefore they were in a better situation to evaluate the architecture and they were able to make better trade-offs.
Deciding to trust object files as inputs might raise a red flag for some people. In principle it could allow attacks/exploits of the linker. But in reality for most threat models this does not matter. Because the compiler generating object files has the same level of trust as the linker. Companies that want to provide an elf linker as a service product are out of scope. Deciding what is in scope and what is out of scope is probably one of the hardest decisions in product engineering. Because many of us software engineers lean towards perfectionism or, especially inexperienced developers are searching for the silver bullet, that set of rules that enables them to develop every product successfully.
[edited typos]
The correct lesson is that the real world is not obliged to conform to your personal model of "better". You have a personal obligation to continuously adapt your model to match the real world. This is the model of science, and is opposed to Platonism. After you have adjusted your model, it is certain to still not be right, and need further adjustment.
Usually "the world" you are obliged to adjust to has its own problems. There are powerful forces making us favor accommodating a Microsoft execution environment, even though that execution environment has always been a cesspit. It represents its own poor abstraction. Posix file system semantics are another example. Von Neumann architecture and the C abstract machine are not the only, or best way to organize computational resources. It is important to recognize when somebody else's pragmatic failure threatens to taint your own models.
Lisp, RG's hobbyhorse, didn't get sidelined because of Philistines. Lisp turned out not to be better, despite how strongly RG felt about it. Instead of figuring out what about Lisp was not right, he called things that were, along axes that matter, more right "worse", preserving his personal model and lessening readers' ability to reason about merit.
All of this is to say: I don't agree with you, but I also agree with you and I suspect he would as well, with qualifications. And he would probably also disagree with you.
Lisp machines were clearly abandoned because of their price. Every time I find some history about someone that actually made that decision, the reasoning was exactly alike, those machines costed more to keep than the Unix ones to install, and were less capable due to outdated hardware.
Yet the essay goes all over the place, citing time to market (that was completely irrelevant, UNIX was the newcomer, Lisp machines were there already), university-based prejudice (yet every single one decided the same at around the same time), and blaming the user. The essay doesn't even talk about money.
You seem to think he was advocating Lisp machines, scoffing at his essay based on that misconception. But if there was a single person in history who worked hardest to destroy Lisp machines, it was probably RPG.
...or worked hardest to turn ALL machines into Lisp machines! ;)
Then he tried to develop and market similar technology in the form of a databased-backed incremental C++ development system for UNIX (called Energize). https://www.youtube.com/watch?v=pQQTScuApWk He hoped that the market for C++ environments was larger than the shrinking Lisp market and that C++ developers would want an interactive system with integrated code management in an object store. That did not went well.
Lisp Machines anyway raised the standard of quality in CRT monitors. Manufacturing their own monitors has to have contributed substantially to their downfall, but we all benefited in the end. Well, all but them.
But there is no absolute requirement to conform oneself to a social milieu. A social milieu changes. It can be altered. It supports a vast number of models. And milieus overlap so densely that one can just go play somewhere else.
Disclosure: I used to be a Lisp bigot, but I got better.
What makes a "better" Lisp program differs from what makes a "better" C++ or Rust program. Besides fitness for purpose, there is maintainability, energy cost to execute, and results per unit time. What did you learn coding it? Can it be the basis for something more ambitious? Is it secure, deadlock-proof? Are its results accurate, aesthetically pleasing, generative of insight?
We can get mired in detail, obscuring important truths. We talk about performance a lot, not because we are obsessive, but because it is a measure that is hard to fake. A faster program says something fundamental about how your computational resources are being directed to the target results.
We can be misled by details of realizations of computation. What is fast on a PDP-11 is not necessarily fast on a Ryzen 5. But submitting to the rigors of performance for the best machines we have is a discipline that connects us, howsoever imperfectly, to the physical principles of computation. It destroys illusion: if a variation seems like it ought to be faster, but is actually slower, there is no sugar-coating the fact. Hewing to physical performance enforces a kind of honesty we don't get any other way.
(Annoying as fuck, that. ;-)
And that seems to indicate that a(n initially) half-assed, but improvable, model of it that is then iterated upon is a good way to build one's model. Feels to me like that is exactly what "Worse is Better" is about -- shouldn't your beef here rather be with "the MIT model"?
> You have a personal obligation to continuously adapt your model to match the real world.
A bit hard to get a consensus model out of that, since everyone's perception of the real world -- heck, everyone's actual "real world" -- varies.
> This is the model of science, and is opposed to Platonism.
Again, what feels like setting up a Platonic ideal to me is more "the MIT model", rather than "Worse is Better".
> After you have adjusted your model, it is certain to still not be right, and need further adjustment.
Ah, dangit, at some point you just gotta say "Screw it, good enough!" (See, for instance, "The saddest 'Just ship it!' story" the other day.)
[Edit: Reduced repetitive weasel wording.]
The top ten languages in https://www.tiobe.com/tiobe-index/ right now are Python, C, Java, C++, C#, Visual Basic, JS, assembly, SQL, and PHP. Of these, the "dialects of Lisp" include Python, Java, C#, VB, JS, and PHP.
Remember that in 01991 all "serious" modern software was either C, C++, Pascal, or assembly. BASIC, whose only data structures were the string and the array, was for amateurs or the MIS department, which mostly used COBOL, assembly, and JCL. Fortran was established but was considered antiquated (except by supercomputer people) and didn't have pointers.
If we compare these languages on the points pg lists in http://www.paulgraham.com/diff.html, the earliest versions of Lisp score 9/9, Python is 7/9, Java is 7/9, C# and VB are Java with different syntax, JS is 7/9, and PHP is 5/9. By contrast, C is 2/9, C++ is 3/9, Pascal is 3/9, assembly is 0/9, COBOL is 0/9, Fortran 77 is 1/9. You can quibble a bit about these numbers (do Pascal procedure parameters qualify as "a function type" even though you can't store them in variables? Does the Python expression "intern(s) is foo" qualify as "a symbol type"?) but the broad division is very clear.
I think it was in the conversation where he originally wrote that essay that Guy Steele said of his own work on Java that they had managed to drag all the C++ programmers kicking and screaming about halfway to Common Lisp, so we should be happy. I've lost my archives from that time, so I can't be sure.
In terms of syntax, Python or Java have nothing in common with Lisp. But in terms of the issues you raise — accommodating a Microsoft execution environment, POSIX filesystem semantics, the Von Neumann architecture, the C abstract machine, or just the tools they give you to analyze problems — they're just dialects of Lisp with slightly different syntax (and more hair on eval, and sort of broken symbols or no symbols).
In terms of "worse is better" of course Python and C# lean just as hard on "worse" as C does.
If we restrict the sense of "Lisp" to languages with S-expression syntax like Emacs Lisp, Common Lisp, Scheme, and Arc, then Lisp did fail to (at least) become popular — but it's plain to see that when people abandoned Common Lisp and Scheme, they were mostly moving to languages like Python and JS which adopted Lisp's most appealing ideas, not to C++.
I also think r-bryan's point in https://news.ycombinator.com/item?id=31346478 is true, beautiful, deep, and merits quoting:
> Your notion "the real world" conflates the physical world with a social milieu. The physical world is (at this scale) immutable, so of course we must conform to it…
> But there is no absolute requirement to conform oneself to a social milieu. A social milieu changes. It can be altered. It supports a vast number of models. And milieus overlap so densely that one can just go play somewhere else.
In that vein, it's worth noting that Linux got pretty far before it ever had to accommodate a Microsoft execution environment, though I did have coworkers in 01997 who thought I was hopelessly unhip for preferring Unix (which they thought of as antiquated) to Microsoft Windows.
It is true that some good ideas originate from Lisp. However, by saying that 'Lisp dominates software development today', you are giving the false impression that the Lisp heritage is the only one that matters, as if Lisp were some sort of linguistic asymptote toward which all other programming languages must inevitably converge. I take issue with that. Creating a new programming language involves copying the good ideas from previous languages and rejecting the bad ones. The notion that this or that modern programming language is a 'dialect of Lisp with slightly different syntax' completely ignores all the bad ideas from Lisp that were not copied and all the good ideas from other languages that were.
But I do think it's a sort of "linguistic extreme" that defines a dimension in programming-language space, and new popular languages tend to be pretty far in the Lisp direction. In TIOBE's top ten languages, the four I identified as "not Lisps" are pretty old: C is from, say, 01973; C++ is from 01982; SQL is from 01974 (and wasn't Turing-complete until a recent unfortunate accident, therefore not a programming language); and assembly language is from somewhere between 01948 and 01968. The other six popular languages I listed are all from 01989 or later; C# and arguably VB are from 02000.
There are several other linguistic extremes that things could have moved towards instead. For example, FORTH, Self (the essence of Smalltalk), miniKANREN (or maybe Mozart/Oz/Prolog), Coq (or at least some kind of cleaned-up ML), the π-calculus, m6, Inform 7, some kind of lazy language (maybe Haskell), and now something interesting is emerging in Rust. You could imagine a history in which all the new languages adopted key controversial ideas from FORTH or ML instead of from Lisp, but that mostly isn't the history we're in.
Java/C#/VB in particular takes pervasive ad-hoc polymorphism from Smalltalk, parametric polymorphism from ML, and type declarations from, I guess, Fortran, though detecting errors at compile time wasn't a motivation for having them in Fortran. And Python takes pervasive ad-hoc polymorphism from Smalltalk and gets its parametric polymorphism for free with the dynamic typing it takes from Lisp. (Python's predecessor ABC was a teaching language derived from ML, which is why Python spells null as None, but Python dropped the static typing.)
Here are some candidates for Lisp's abandoned or not-adopted bad ideas that come to my mind:
1. Dynamic scoping as default. PostScript and TeX do do this, but nothing else does. Even Common Lisp and Emacs Lisp have abandoned it. (And Scheme never had it to begin with.) Nevertheless, it was viewed as a defining attribute of Lisp.
2. The β-reduction model of program execution, in which the output of the program is a transformed version of the program; and, more generally, there is no clear distinction between the program and the data it processes. This was sharply abandoned in Scheme, but without abandoning metaprogramming. Mathematica is the only current language I know of that works this way. (Maybe Q?)
3. Textual serialization (READ, PRINT) as the standard and indeed only serialization. (The original Lisp and many of its successors also have pervasive serializability, which pg inexplicably omitted from his original essay: any data object can be written to a byte stream, and the same or equivalent data object can later be read back from it.) Variants of this are pretty popular; JSON.stringify, Python repr(), Firefox .toSource(), Python pickle, etc., but in most cases modern systems relieve the tensions between the different uses of READ and PRINT by providing several different serializations.
4. The general idea that the important part of software development is getting a prototype working, after which making it robust is comparatively trivial. This idea probably does not originate with Lisp, and it is popular in startup business literature, but not in current programming language design. But people use Python in Jupyter a lot to interactively explore computational ideas.
5. Symbols as the only string type, which is to say, hash-consing all your strings. This is sometimes a worthwhile thing to do in a particular library but not a good tradeoff to impose on all users of the language. Lisps stopped doing this about 50 years ago too.
I'm interested to hear which bad ideas from Lisp you're thinking of!
Actually when the AI winter end 80s / early 90s set in, a bunch of Lisp projects&applications moved to C++.
> Pascal is for building pyramids—imposing, breathtaking, static structures built by armies pushing heavy blocks into place. Lisp is for building organisms—imposing, breathtaking, dynamic structures built by squads fitting fluctuating myriads of simpler organisms into place. The organizing principles used are the same in both cases, except for one extraordinarily important difference: The discretionary exportable functionality entrusted to the individual Lisp programmer is more than an order of magnitude greater than that to be found within Pascal enterprises. ... To illustrate this difference, compare the treatment of material and exercises within this book with that in any first-course text using Pascal.
You can easily substitute "C++" for "Pascal" here and "Python" or "JS" for "Lisp", and indeed a JS translation of SICP has been completed, including a column-by-column comparison: https://sicp.sourceacademy.org/chapters/5.2.3.html
Common Lisp was not developed as a scripting language like Python or JavaScript. It was developed for complex applications (100k to 10M lines of code were not rare - for example the PTC CAD system written in C and Lisp had 7 MLOC Lisp code already over a decade ago) and derived from a language with was actually a systems programming language: Lisp Machine Lisp.
Thus there were a bunch of ambitious projects initially written in Lisp and then continued in C++ (and also in C) for better performance on smaller machines, smaller memory footprints and less need to mix C and Lisp.
In particular, Lisp in general (including Common Lisp) is designed as far as possible to maximize flexibility, and I think there's an unavoidable tradeoff between flexibility and correctness. Oversimplified, programs are flexible when they can do things their original author did not anticipate; they are correct when they cannot do things their original author did not anticipate.
Of course, in reality, it's not that simple. But this obviously false formulation contains important bits of truth. Testing a flexible program is more difficult, and more bugs are likely to slip through your testing, and each one usually takes more effort to diagnose and fix. And of course in general any two virtues trade off against one another in the limit, because if you take every possible measure to increase the first, some of them will happen to decrease the second. (By the same token there are some measures you can take that will increase both.)
______
¹ Though GCC still routinely generates much better code than SBCL, much less the new Elisp compiler, that's not the primary consideration. I'm not sure why you're mentioning it.
It's complex as hell code-wise, but it simplifies the amount of cross-team/cross-company alignment and synchronization that has to happen in order to cut a new version of the application containing hotfix 8495 for the part of the app maintained by Steve's team.
There's actually a decent parallel between that and Microservices. Microservices make maintenance of the whole more complicated by introducing the network between pieces, but allow each piece more flexibility in how it's developed.
The folks at Lucid were starting to get a little worried because I would bring them review drafts of papers arguing for worse is better, and later I would bring them rebuttals against myself. One fellow was seriously nervous that I might have a mental disease.
after over a decade of thinking and speaking about it . . . [I wrote] "Back to the Future: Is Worse (Still) Better?" In this short paper, I came out against worse is better. But a month or so later, I wrote a second one, called "Back to the Future: Worse (Still) is Better!" which was in favor of it.
When those maxima are things like operating systems and programming language ecosystems, those are very very big hops to make.
Abstraction always costs. This implies any abstraction you put in that doesn't deliver commensurate benefits makes your code fundamentally worse.
If you have an abstraction with a name that seems to say it does X, but to be sure I have to trace through four other source files plus an unknown number of dead ends just to see if it really does exactly just X, and in the end it could have just been coded in place, it has already cost way more than any benefit it could yield.
Abstraction is an engineering tool, not a moral imperative. You can always add abstraction later.
So, "Worse is Better" is at best misleading. Better is better. But the measure of "better" you have been using is likely to be way off. People who think of themselves as smart tend to undervalue simplicity. It is a personal failing.
If you have to verify that "it does X" by following source files and tracing execution then you're talking about indirection.
An abstraction has mathematically sound laws that can be proven and introduce precise, new semantic layers to a program. By definition one doesn't have to think about the layers underneath the abstraction and can instead think entirely in the abstraction.
The difficulty with abstractions is that few practising programmers know how to think even informally about abstractions. The mistake of substituting abstraction for indirection leads to the misguided notion that virtual methods and classes are "abstractions." However if you try to formalize these abstractions with relations and properties I think you will find most of these proofs difficult, if not downright impossible write. That's a good sign you don't have an abstraction.
The central tenet of worse-is-better is to prioritize implementation simplicity over user experience. But if you simplify the implementation since some features are never used and not needed then you haven’t even had to make a choice on that spectrum—you have just cut out unneeded cruft.
In fact the user experience has improved since it is faster...
This post is stating that they tried to generalize too much instead of building to narrow use cases first. There is no need to bring "worse is better" into it at all, IMO
In that sense, every more or less radical simplification that sacrifices some of those fine (theoretical) advantages for simplicity can be said to be an example of "Worse is Better". Because what "Worse is Better" means is just that "theoretically 'worse', but much simpler code is actually better than theoretically 'better' but much more complex code". It's "worse" vs "better" on all those fine more or less theoretical dimensions against "worse" vs "better" on the single dimension of code complexity / simplicity.
So I'd say this article, which was all about how a radical simplification of the code -- sacrificing the (unnecessary) generalization and flexibility they'd first tried to build into it -- turned out to bring a lot of other practical advantages, is a prime example of "Worse is Better". As I understand it.
My take is that one should program in function of what the code should do, and not in function of what it's comfortable to me (as a developer).
Yes, it's a great feeling when your code fits like a jigsaw puzzle, but also more complexity = more code being executed. Behind that RAII, behind that "operator=" and that "p = new Struct", etc. there might be extra complexity for the sake of developer's readability and comfortability. There is little or no added value for the end user or the purpose of the program itself.
Also the code should be written "for the now", not for "that future feature it would be awesome to have someday like making it compatible with every other library X, etc.".
At the end of the day, even without realizing it, your program is slow.
I remember a developer where I work did a C# implementation of an AT command parser, in which every AT command was a separate DLL. It was very complex, and super slow. But the developer argued "if I need a new AT command, I'll just add a new DLL". It might have been better for him as a developer, but it was worse for the end user and the system in general. The code died the day that guy left the job.
The original article called 'worse is better' was pretty successful and widely read, and part of it was the title grabbed people's attention. You could argue it's a rephrasing of "less is more" or "do one thing well" concepts for the click bait zeitgeist. That's not a bad thing - an old concept wrapped up in a new way of expressing can keep the idea alive - anyway it's funny and a little confusing, which might trigger someone to engage with it differently. There's many concepts that I've heard n times expressed in different ways only to have it click on the n + 1 way of phrasing it.
'Yes, it's a great feeling when your code fits like a jigsaw puzzle, but also more complexity = more code being executed. Behind that function call, behind that nested expression and that "p = malloc(sizeof(*p))", etc. there might be extra complexity for the sake of developer's readability and comfortability. There is little or no added value for the end user or the purpose of the program itself.'
See? The same argument can be used against C in favor of assembly. Abstractions allow us to write safer, more correct code in less time, which actually does offer a lot of added value for the end user. C makes many useful abstractions harder or downright impossible, which is one reason why software written in C is generally such a shitshow from a security perspective.
There is no way to avoid most of things you listed. It's not comfort; it's a necessity.
> software written in C is generally such a shitshow
Whatever language you use to code, all your calls will sooner or later pass through a function call, a nested expression or a malloc()/free() call, somewhere in the huge stack of C code your language needs, depends and relies on. There is no escape. Basically, today your preferred language exists and might be able to something just because C code exists.
Oh yes, there definitely is: There are no function calls, nested expressions, or malloc in assembly – those are abstractions provided by the language and translated by the compiler, for comfort, not necessity. In fact, you can simulate all that in C itself: use gotos and an explicitly managed memory area as stack, and manually transform nested expressions into SSA form.
The folksy software adage for this is YAGNI. (You Ain’t Gonna Need It)
The software industry might be better off if we start making explicit that we have a real trade off of time and effort between designing a system that can handle all contingencies, and one that needs adjudication occasionally, i.e. throws an error or crashes, needs patching, etc. [2]
Notice that Rui’s solution here was to basically allow lld to just crash under rare, extraordinary cases. This seems reasonable when written here, but in the real world suggesting that a system be allowed to crash, ever, often gets very hard pushback.
[1] “Smart Contracts and DApps: Clarity versus Obscurity” https://youtu.be/JPkgJwJHYSc?t=3512
[2] “Hard-assed bug fixin’” Joel on Software (2001)
You could crash, but you can also throw. Somebody somewhere sometime might be able to do a better job with the situation than you can right there and then. Checking status codes all up and down the call graph does not increase the likelihood that something could be done, but costs in the meantime. Hiding the checking behind abstractions does not reduce those costs.
This is a very big assumption.
Edit: The assumption is that this day can ever come. The author decided that in his system that day would likely never arrive.
There's no need for that person to decide, then and there, when they can just as well leave the choice to somebody else, at exactly zero cost.
This is where the confusion lies. “Zero cost” here means “it’s a few lines of code, a few cpu cycles to handle it. What’s the big deal?”
But I’m not talking about a few lines of code or cpu cycles, I’m talking about developer time. Leaving the decision to someone else may take little no time to you, today, but might incur a steep cost on someone else’s time tomorrow. Even worse is when you, with deep domain knowledge, tempt a far less knowledgeable dev into “handling” an error at a time and in a place that will be difficult or impossible to handle, wasting their time.
If somebody decides they want to catch the exception and try something else, that is totally their choice, and they can devote as much or as little time to getting their idea working as they like. If they get it working, good. If they give up first, that is fine too.
OTOH, it could be that you're just not getting this: Those exceptions that can't ultimately be handled will perhaps throw a (more or less informative) error message onto the screen and into a log file, but then they'll have to do a (more or less controlled) shutdown of the app.
One form of shorthand for this that I could see myself using is... "just crash". So perhaps that was what was originally meant: Maybe your "throw an exception for godssake!" is "just crash".
I understand perfectly well what you’ve said, and I don’t disagree with any of it. The point I’m trying to make (badly, apparently) is not that.
My point (and I believe the point of the linked article, and, I mistakenly thought, my original comment) is that decisions by a few can have enormous impacts on the productivity of many, including design decisions like “should we bother polishing the errors from these corner cases at the expense of delaying new feature work, or just let people report a bug if they run into a corner case and get on with new features?” Rui chose to group a whole swath of errors in lld into the “not worth polishing” category so they could move on with more important work.
My apologies that my point somehow came across as “always crash your program, never throw exceptions!” Thanks for your patience and taking the time to read generously and deeply and to really understand my point.
Producing faulty output is a more likely response to unchecked faulty input than crashing. It would have been better for the author to mention that, if true.
Interestingly I have encountered crashes in Ninja (not lld), caused by corrupted on-disk state I had to delete: https://github.com/ninja-build/ninja/issues/1978. I think I traced it down to a memory indexing or null pointer error, which would've been caught by asserts but they were disabled in release builds.
The real world is messy. Things that come into contact with the real world will acquire a little bit of wear and mess. If it's all still brilliantly clean and tidy, you can't have done anything useful yet!
One of the most useful ideas in developing reasonably complex systems that I encountered is treating different types of errors differently. There's a place for both panics/exceptions on one hand result monads/error messages on another.
The problem is that this does't go far enough - decades ago machines were running on a single CPU and OSes were focused on the scheduling of processes. Syscalls were all blocking, so for each individual process there could only ever be one syscall ("request") in flight at a time. Now, we're seeing a change (for example with io_uring) towards fully asynchronous I/O exposed to the userland, which allows submitting multiple requests to various I/O devices simultaneously, which has the potential to improve throughput a lot.
Likewise many programmers don’t seem to get that the incentives of a programmer are quite different than the incentives of a manager. Programmers think “aw man my stupid manager is making me push out features instead of refactoring, if I were in charge I’d focus on code quality and performance”, not understanding that maybe, just maybe their manager might have different incentives and a different perspective.
Worse is better is essentially a shorthand for understanding product management and scoping.
I find this an odd take on that paper. To me it has always been about how simplicity is more important than correctness. And while the authors take doesn't conflict with WiB, I do think they miss the point.
> "Simplicity of implementation is very, very important, and you can sometimes sacrifice consistency and completeness for it."
They almost get it in the conclusion. They are starting to see the important of simplicity, but still are fixed on correctness. Simplicity is not something you sacrifice correctness and completeness for "sometimes", it always wins if there is a contest. It is always more important (at least in line with WiB).
Not quite: The absolutely simplest code is an empty code file. But that doesn't do anything when compiled, so I doubt it would become very popular; people want their software to do something. And if that "something" is equivalent to "rm -rf /", they'd prefer it to do something correct in stead. So code simplicity doesn't always win; having at least some darn somewhat-correct functionality easily beats it.
This is one of the reasons sum types are so critical in my opinion. They let you write code in that style, where OOP forces you to make everything look kinda the same.
Also, it's not OK to aim to increase the amount of code. //TODO include both statements in the creed.
Yes; people who unit test.
In fact, neither design is the "better".
In not-so-modern-anymore POSIX, you can choose whether a system call will be restarted after a signal is handled, or whether it will terminate with an error. Both requirements are needed.
It is signals themselves that are "worse". But they let you have asynchronous behaviors without using threads.
Sometimes you want a signal handler to just set some flag. This is because you have to be careful what you do in a signal handler, as well as how much you do. And then if you want the program to react to that flag, it behooves you to have it wake up from the interrupted system call and not go back to sleep for another 27 seconds until some network data arrives or whatever.
In addition to sigaction, you can also abort a system call by jumping out of a signal handler; in POSIX you have sigsetjmp and siglongjmp which save and restore the signal mask. So that would be an alternative to setting a flag and checking. If you use siglongjmp, the signal itself can be set up in such a way that the system call is restarted. The signal handler can then choose to return (syscall is restarted) or bail via siglongjmp (syscall is abandoned). I wouldn't necessarily want to be forced to use siglongjmp as the only way to get around system calls being always restartable.
Anyway, the Unix design showed to be capable of being "worse for now", and have space to work toward "better eventually".
In the present story, the monolithic linker design isn't "worse". Let's just look at one aspect: crashing on corrupt inputs. Is that a bad requirement not to require robustness? No; the requirement is justifiable, because a linker isn't required to handle untrusted inputs. It's a tool-chain back-end. The only way it gets a bad input is if the middle parts of the toolchain violate its contract; the assembler puts out a bad object file and such. It can be a wasteful requirement to have careful contract checking between internal components.
Gabriel naturally makes references to Lisp in the Rise of Worse is Better, claiming that Common Lisp is an example of better. But not everything is robust in Common Lisp. For instance the way type declarations work is "worse is better": you make promises to the compiler, and then if you violate them, you have undefined behavior. Modifying a literal object is undefined behavior in Common Lisp, pretty much exactly like in ISO C. The Loop macro's clause symbols being compared as strings is worse-is-better; the "correct requirement" would have been to use keywords, or else symbols in the CL package that have to be properly made visible to be used.
I don't think that Gabriel had a well reasoned and organized point in the essay and himself admitted that it was probably flawed (and on top of that, misunderstood).
The essays is about requirements; of course the assumption is that everyone is implementing the requirements right: "worse" doesn't refer to bugs (which would be a strawman interpretation) but to a sort of "taste" in the selection of requirements.
Requirements have so many dimensions that it's very hard to know which directions in that space point toward "better". There are tradeoffs at every corner. Adopt this "better" requirement here, but then you have to concede toward "worse" there. If we look at one single requirement at a time, it's not difficult to acquire a sense of which direction is better or worse, but the combinations of thousands of requirements are daunting.
If we look for what is the truth, the insight in Gabriel's essay it is that adherence to principled absolutes is often easily defeated by flexible reasoning that takes into account the context.
3.1415926 is undeniably a better approximation of pi than 3.14. But if you had to use pencil-and-paper calculations to estimate how many tiles you need for a circular room, it would be worse to be using 3.1415926. You would just do a lot of extra work, for no benefit; the estimate wouldn't be any better. Using the worse 3.14 is better than using 3.1415926; that may be the essence of "worse is better". On the other hand, if you have a calculator with a pi button, it would be worse to be punching in 3.14 than just using the button, and the fact that the button gives you pi to 17 digits is moot. A small bit of context like that can change the way in which the worse-is-better reasoning is applied.