I've heard this about Lisp, but I don't really get how this doesn't apply to other languages. How is writing special operators in Lisp different from writing special functions in any other language?
It can be, and he even says as much:
>> Bottom-up design is possible to a certain degree in languages other than Lisp. Whenever you see library functions, bottom-up design is happening.
I'll also point to kazinator's quote of Ken Thompson in the previous discussion linked by dang:
>> It is the way I think. I am a very bottom-up thinker. If you give me the right kind of Tinker Toys, I can imagine the building. I can sit there and see primitives and recognize their power to build structures a half mile high, if only I had just one more to make it functionally complete. I can see those kinds of things. The converse is true, too, I think. I can’t— from the building—imagine the Tinker Toys. When I see a top-down description of a system or language that has infinite libraries described by layers and layers, all I just see is a morass. I can’t get a feel for it. I can’t understand how the pieces fit; I can’t understand something presented to me that’s very complex. Maybe I do what I do because if I built anything more complicated, I couldn’t understand it. I really must break it down into little pieces. - Ken Thompson, Unix and Beyond: An Interview with Ken Thompson, 1999
And if you use a TDD style of development (in particular, but test-heavy in general with frequent use of unit and integration tests below the full end-to-end-test level) you'll also likely stumble onto a similar bottom-up style of development.
I think, circa 1993, his emphasis on Lisp and bottom-up development made a lot more sense than it does today with the increasing availability of interactive development environments (essentially every dynamically typed language, pretty much; but even many others like Java and C#) and increased emphasis on test-heavy development methodologies.
There are a number of reasons that bottom up programming in the style PG seems to be advocating for is not popular in industry. Industrial programming favors the median developer while bottom up is a technique that works best with small teams of skilled programmers. Big cos obviously prefer "safe" languages, but even startups are generally conservative due to the vc induced pressure towards hypergrowth, which also favors the median developer since hypergrowth requires hiring.
Writing good programs from the bottom up requires more skill[1], but can lead to substantially better programs by certain criteria. But it may also be considered inscrutable by the median developer. The median developer can also misuse the power afforded by the bottom up language and write something much worse than they would have been able to in a more conservative language. This isn't a knock on the median developer, btw, just my perspective on why this style is rarely found in industry.
[1] neither LLMs nor stack overflow are of much help when you have essentially written a new language to solve your problem.
I think another reason is that it’s just more work for another programmer to grok a bottom-up written program written with its own DSL then a program that’s bent to fit the existing (more rigid) language since they already know the existing language.
> You mentioned that a lot of Forth programs you’ve seen look like C programs. How do you design a better Forth program?
> Chuck: Bottom-up.
> First, you presumably have some I/O signals that you have to generate, so you generate them. Then you write some code that controls the generation of those signals. Then you work your way up until finally you have the highest-level word, and you call it go and you type go and everything happens.
> I have very little faith in systems analysts who work top-down. They decide what the problem is and then they factor it in such a way that it can be very difficult to implement.
https://www.oreilly.com/library/view/masterminds-of-programm...
And that's true, lisp macros are way more powerful than what most languages support. The newer ones (and some "progressive" old ones) got template systems that are almost as powerful, but the most used languages simply don't have anything comparable.
That said, nowadays using a lot of that kind of feature is normally frowned upon.
In these cases the languages used are generally uniform so your skills easily transfer and you aren't suffering as many 200% problems by learning both the language and the authors peculiar add-ons.
Of course it still happens just not as easy with rigid languages such as Go.
It's all tradeoffs.
That is, unless you treat them as another language, with tooling, documentation, and stability. Then they scale again, but it requires quite a lot of work.
My no. 1 suspect is OOP or, more specifically, languages like Java that won't let you write a function. You can bypass that limitation creating a class whose only reason is to hold stand-alone functions. I don't do that, lately I don't much Java at all, but I've seen huge Util.java classes.
In sane languages I used to start exploratory programs with two files: one for the main flow, the other for utility functions. Later the former would become different windows, the latter different libraries. Classes grew slowly, one functionality aspect at a time.
Forced OOP nudge you into a priori design the class structure before understanding the domain. Bottom-up advantage is that, having the primitives earlier, you can test with real data from the start and show the results to the domain experts.