In defense of complicated programming languages

By the way, where can I read a D tutorial from the bottom up?

Edit: I just realized from the sister comment who I was replying to. Old school charm for sure. Now I love it even more.

As an engineer in the forties it is somewhat encouraging to read that you felt the same way even if it was about different topics.

For me, these days it is about frontend generally, Gradle on backend and devops. So much to learn, so little documentation that makes sense for me. (I'm considered unusually useful in all projects I touch it seems but for me it is an uphill struggle each week.)

I always win in the end even if it means picking apart files, debugging huge stacks and a whole lot of reading docs and searching for docs, but why oh why can't even amazingly well funded projects make good documentation..?)

I just never quite understood where this "don't worry about the implementation" is coming from, as well as the tendency to explain abstractions in general terms, with analogies that make little sense, etc. The "don't worry about the implementation" did so much harm to humanity by producing bloated, wasteful software.

In fact I think a good abstraction is the one (1) whose implementation can be explained in clear terms, and (2) whose benefits are as clear once you know how it's implemented.

We understood functions as "mappings" between objects for hundreds of years, and when programming came along it gave us new ways to think about functions, but being able to "make" a function in hardware/software doesn't actually change what a function is at its core.

There's a reason why computer science professors explain concepts at a high or abstract level and don't jump into implementation to help students understand them. It's because the concepts ARE the high-level meanings, and if you need to see an implementation then you're not really understanding the idea for what it is.

If an idea stays abstract in your mind, it gives you more flexibility with how you apply it and more ways to find use out of it. But it does take a mindset shift, and is an actual learned skill, to be able to see something as purely abstract and accept that how it's made doesn't matter.

-- Edit - just realized who I was replying to. So take this comment as not meant to be lecturing, but just a 2c to offer :).

The GNAT Ada compiler has an option to output a much-simplified desugared code. Not compilable Ada, but very inspectable unrolled, expanded code. Makes for a great teaching tool. 'Aaaaaaah this generic mechanism does that!'

Link https://docs.adacore.com/gnat_ugn-docs/html/gnat_ugn/gnat_ug... look up -gnatG[=nn]... Good stuff.

http://clhs.lisp.se/Body/f_disass.htm

I'd started working on manually writing the same code is both C++ and C, but your approach of using something automated is an even better idea. Showing the implicit this pointer isn't hard to do manually, but polymorphism is a bit more of a pain. But I think the best part about using a tool is that he can change the C++ code and see how it affects the emitted C. Being able to tinker with inputs and see how they affect the output is huge when it comes to learning how something works.

When learning Git, I enjoyed reading a tutorial that explained it from the bottom up, but, in the end, having been shown, early on, what is merely the implementation detail, created a cognitive noise that is now hard to get rid of.

I will resist replying to this.

- Principles for how things fit together. This is similar to your comment about digging into the assembly. Understanding how something is built is one way of determining the principles.

- Understanding of why something is needed. I still remember back to first learning to program and not understanding pointers. I pushed through in reading the book I was using and eventually they talked about how to use it and it finally clicked.

I went to the university bookstore and found the textbook section for the university's undergraduate business degree programs, and bought the textbook for an accounting class.

Seeing the coverage of those tax areas from the accounting point of view cleared up what was going on, and then I understood what was going on in the code and regulations.

If I come across a feature I like in a programming language, I usually find myself trying to figure out how to implement or emulate it in a language I already know (ideally one that won't hide too much from me). Implementing coroutines in C using switch statements, macros, and a little bit of state for example.

Funny, that's similar to how I learned assembly! I wrote some small program, and then used lab equipment to monitor the bits flipping in the PC circuits...

I remember two lessons about networks at school, the first one was "top to bottom" (layer 7 to layer 1), I understood nothing, then there was another one bottom up, and I finally understood networks..

And what I think it's also important: don't ask for the key if the door doesn't need to be opened.

The examples on the text about Python and OOO are on point. Yes, people would reinvent classes if they needed, but the great thing about classes (and type annotations) are that they're optional

Compare with Java where you need a 'static void main' method inside a class just to begin. Why? Not even C++ requires that.

Do help them keep in their lanes but keep simple things simple.

This is a hard problem. We have, at least, structs, classes, objects, traits, modules, and namespaces. There still isn't consensus on how best to do that.

At a higher level of grouping, we have more trouble. This is the level of DLLs, microservices, and remote APIs. We barely have language for talking about that. We have no word for that level of structure. I sometimes refer to that as the "big object" level. Attempts to formalize that level have led to nightmares such as CORBA, XML schemas, and JSON schemas, and a long history of mostly forgotten interface languages.

It's interesting to read that the author likes Python's new typing system. It seems kind of half-baked to have unenforced type declarations. Optional type declarations, sure, but unenforced ones?

Later in life I've discovered a trick that gets me into this state of mind more easily. For any feature, I ask myself: What problem does it solve?

I totally agree there and I think it's due to the fact that you've got a mental model of the problem space and your brain can see the empty hole. Once you have a tool of that shape, your brain knows where it goes and how it solves your problem.

It kinda put me in a mood of going over the basic principles of mathematics again, but in this new light.

Any recommendation of content that follows the advice given in the article?

Thanks!

Except that words for concepts nobody's ever bothered to consider don't just appear in the language. That can only happen when somebody tries to think about the topic and is forced into making up some new words.

Given that most animals can reason about other animals as agents with intentions, I'm not sure what people have been unable to answer. This also sounds a bit longer the discredited setting version of the Sappir-Whorf hypothesis (linguistic determinism).

I am yet to find whether they really dislike or just find it unfamiliar.

I am convinced that unfamiliar gets conflated with unintuitive and hard all the time.

The problem here is that most 'people' have already learned the non-simple, non-consistend and bent logic of non-mathematical (aka imperative) programming languages. Because (almost) everybody has learned the basics of mathematics before programming, so for _everybody_ a term like `x = x + 1` had been a violation of all three.

Depends on what you're writing; Gerald Jay Sussman claims that edge cases in mathematical notation (and corner-cutting of sorts in what is written) makes physics quite hard to grasp in "The role of programming" [1]; hence the Structure and Interpretation of Classical Mechanics book, which includes many Scheme programs which explicate everything.

[1] https://www.youtube.com/watch?v=arMH5GjBwUQ

The only reason why humans seem to prefer infix notation, is because it is what's being taught in school. The first mathematical expressions most people get to see is

    1 + 1

    + 1 1

But every new user is totally confused about this (I'm not even sure if my explanation above will be understood), because everyone expects modules to be a special case of imperative file loading operations, rather than declarative syntax creating named namespaced items, like everything else in the language.

The semantics of mutable references and other interface constructs must be independent from actual implementation, to allow for changes in the latter. So Rust is behaving as expected here. If the changes to X and Y are truly independent, you can have a function that returns separate mutable references to both ("splits" the struct). But if your library function takes an S, it might rely on the whole struct and then the changes are no longer truly independent.

Why does that require a macro?

``` struct S { x:X, y:Y }

fn do_with_y (y:&mut Y) {} fn do_with_s (y:&mut S) {}

fn main_program (mut s: S) { do_with_s(&mut s); do_with_y(&s.y); /* ... */ } ```

I prefer seeing code like this rather than wrapping it in abstractions and macros/helpers because it's not particularly verbose and very clear where the borrowing happens ; you'd see the same in C++ or C, to be honest.

You could even argue it makes writing programs more complicated. But they definitely also make writing programs much much easier.

Kind of like how algebra is more complicated than basic arithmetic, but good luck proving Pythagoras's theorem without algebra.

Beware of the Turing tarpit where everything is simple but nothing is easy! Python is by no means perfect, but using different syntax when doing different things can make a lot of sense. "Everything is a X" can be a nice mental model, but on the other hand, as we've seen with classes, mixing data structures, structs and namespaces all in one concept is not always the best solution.

W.r.t whether types makes Python more complex - my point was that a type system makes the _language_ more complex, but it makes the _code_ using it easier to reason about. That's my main view of the blog post: Be careful trying to simplify a programming language if it leads to the code itself being more complex.

And sure, I have no particular attachment to OOP "classes" as such, and I don't think they are necessary. In fact I much prefer Julia's approach. What I meant with classes being inevitable is that 1) structs, or something like structs, are inevitable, and 2) methods, in the sense of functions that are semantically tried to particular structs, are inevitable. In Python, these two things are provided by classes, so in Python, there really is no escaping classes.

Good languages with good IDE support?

I'm old enough that at some point assembly and C were my favourite languages and PHP was what I was most productive in and Python was also something I enjoy.

These days no way I go back for new projects.

After Java "clicked" for me and the introduction of Java generics shortly afterwards it became my favourite language and later TypeScript and C# had been added to that list.

I'm not to dumb, but I prefer using my smarts to develop programs, to work together or even pair program with the language to mold something, not to babysit languages that cannot even help me with trivial mistakes.

In another language you might introduce structs first (C and Go only have structs, C++ has both classes and structs but they are the same thing) but in Python you don't have that option - you can use data classes or named tuples but those already need most of the ceremony you introduce for dealing with classes.

It can be managed by modules/namespaces, but without even that (a la C), I really don’t see it managing complexity well. The important point of OOP is the visibility modifiers, not “methods on structs”, that would provide no added value whatsoever.

What OOP allows is - as mentioned - not having to worry about the underlying details at call site. Eg, you have a class with some strict invariant that must be upheld. In case of structs you have to be careful not to manually change anything/only do it through the appropriate function. And the bad thing is that you can make sure that your modification is correct as per the implementation, but what classes allow for is that this enforced invariant will remain so even when the class is modified - while now your call site usage may not fulfill the needed constraints.

Very quotable!

That's why I like it when languages allow you to start writing simple code and gradually make it more complex. Haskell for example is not like that. Python does better here. I think Scala is one of the best languages in that regard.

And that's why we call it a curve instead of "learning time" or "learning content", which are two different dimensions.

I know that many developers in earlier times, before the internet, when programming languages were more limited and access to library repositories was not possible most experienced programmers built up their own libraries of functions that they would use and add to as required.

These days, we still have these extensions to the languages, but they are shared in repos and are called shared libraries or frameworks.

Ruby is fairly easy to learn. Does that mean it has a nearly empty toolbox? No it doesn't.

A language with a GC is easier to learn than one without a GC. Does the one with a GC have less in the toolbox? What if it also allows opting out of GC?

ObjC is considered a hard language to learn, esp for people used to C++ and Java. Does this come from it having more tools in the toolbox?

"A language with a short learning curve is like a toolbox that’s nearly empty" is a nice quote, but it also objectively seems to be wrong.

These days, programming is being thought to more pupils and students than ever, because languages like Python has made it very accessible and easy to learn.

The fact that a non-CS/IT person who has never written a code in their lives, can learn the basics with languages like Python, and make tools which increases productivity in just mere weeks, is amazing. I know this, because I've witnessed it multiple times at work.

That would almost certainly have never been the case, had we only had languages with steep learning curves. Imagine if someone wanted to make a web-scraper in C, and some program that act on the scraped data. In python, that's basically under 20 lines of code...in C? Most likely hundreds.

Further more, its more consistent if signed and unsigned behaving identical than having to look for the type with integer promotion rules of the number.

If you prefer the current C/C++ way of less performance and inconsistency that is fine.

    unsigned short a = 0xFFFF;
    unsigned short b = a;
    unsigned short c = a * b;

Anecdotally, as I’ve gained experience, I and most people I know have only grown in valuing simplicity. It’s one reason the the average Clojure developer is older.

You also seem to suggest that a “high level low level” language is not really possible (the trap c++ and rust falls into) due to low level details always seeping through, making refactors much slower. So my question is: how would the ideal low and high level language look like in your opinion? Regarding the high level languages, do you find java’s Valhalla plans good? I personally really like the semantic-only distinction of basically loss-of-identity, and loss-of-tearing, giving ample grounds for optimizations behind the scenes, and over time (in part thanks to some of your comments), I grown to really like the simplicity a language with simple classes and methods can bring.

> As someone who strongly dislikes complicated languages

while

> Working on OpenJDK at Oracle

as if Java was uncomplicated…

People have figured this out about C++, but they’re still in denial about Rust. Notice how in the article the borrow checker is the best kind of feature which emerged to handle an existing problem.

In reality the borrow checker is a hammer which forces all code to look like something the borrow checker can nail down. Rust is similar to C++ and it looks like it will be used outside low-level programming or systems programming and in those cases one can gladly give up e.g. manual memory management, obsessively avoiding copying, having to specify how wide an integer is and so on at least in parts of the code.

If you are writing small scripts, it really doesn't matter. If have a million lines of code and thousands upon thousands of source files, any structure you can impose on it makes a big difference.

That said, OOP is just one paradigm of imposing structure.

a class is also a type (if you have a typed language),

a class is also a module, you group together values that makes sense together (the x and the y of a point) with the functions (methods) that interact with it.

When you have a lot of codes flying around, you can add encapsulation (private/public thingy) so the user of a code does not see the implementation which helps to create libraries that can evolve independently from the applications using them.

Also compared to an associative array, a class is more compact in memory (granted JavaScript runtimes see dictionaries as hidden classes).

In itself every element is easy, but together they are hard.

You asked for an example where a class is more suitable than a function. I can think of no better example than the abstract concept of files on a modern operating system. In fact it's so useful you will probably have difficulty coming up with ways to perform the same operations without using an object-oriented strategy.

I/O or resources.

Any time you're interacting with a stateful system like an API or a database, there's inevitable setup and maintenance to keep access to that resource sound. Additionally, you rarely care how that resource is kept coherent, you just need to access it at parts of your program. Without a class or some system that keeps some bookkeeping data around with methods, every function has to have that bookkeeping state passed in as a handle, and all of a sudden all functions need to know something about how setup works, as opposed to what it does. This is a rare instance of where namespaces + globals relative to those namespaces reduce complexity; even languages like Clojure have DI "objects" for resource access, and lots of languages that have IO monads are really a data structure over some state + functions.

This setup of objects as things you can't interact with directly--aka not data--is the original one in the literature. Once upon a time you needed to interact with a device driver or a mouse or a screen or a hard drive, and logically encapsulating that device was useful. You'd then interact via "messages" dispatched to the object which are effectively what methods are these days (the term "dispatch" is still used in the literature). It also makes mocking out that physical real-world thing easier.

Notably, objects never were meant as a way to represent data. I'm one of those functional programming weirdos that likes most things to be immutable, and I have simple stateless functions in an enclosing namespace. I pass simple structs (or case classes) of data around in those functions and it works well for me. But the I/O or externally interfacing parts of my program are always in specific places in my program, and they always use dependency injection, and it's always obvious what they are. They look like a sandwich where those objects are at the edges of the program (APIs/ingress and database/egress), but the meat of the program is in the inside and is boring pure functions.

1) A class is basically a contract known at compile time. If there's a mistake (for example, you're trying to reference a non-existing property), it can be caught before the program is even run, saving time.

2) A class often acts as a namespace/facade, i.e. you encapsulate a piece of logic inside a class, and its users only see a simple-to-use interface, they don't have to know about implementation details (such as what dictionary must have what keys).

3) A class can be made self-validating. For example, there can be a rule "price can't be negative". If we have a proper "Price" object, it can validate this rule itself, whenever it's constructed/updated. Thanks to data encapsulation, we can prevent users from updating the price value directly, only by asking the object to update itself (and it can refuse to do so!) -- making sure it never can be below zero. If you use free-standing functions on associative arrays, you can forget to validate certain rules in some places, leading to data corruption.

3) Classes allow polymorphic behavior. I.e. implementation can be changed dynamically based on the current context.

4) If you follow domain-driven design, it's easier to discuss and model new requirements with "domain experts" (not necessarily programmers) using the concept of "objects", with well established properties and behavior attached to them, rather than some ad hoc associative arrays and functions.

Not all of this is unique to OOP, of course. OOP is just one of the tools.

Now you want to make 100s of different dogs in your program and you want make them to bark at each other.

You want to loop over all the dogs and feed them.

You can make infinite loop that will check all the dogs and if one is hungry call Feed and if one is not then not.

Learn what is the difference between Object and Class. You can create objects and let them live in your program.

You have probably noticed in your "fairly simple scripts" that your language has some notion of types. Your variables are one type or another type, in some languages they can change type and in others they mustn't, but likely the language has the notion that, for example, while dividing 9 by 3 is a reasonable thing to do, dividing your array of business departments by the name of a CSV file is not.

Now, these types aren't like the elements (whether classical or chemical) they were chosen by the language authors. There are probably a handful of obvious ones, probably strings like "Clown" and numbers like 19.03 are different types, your dictionary isn't a number, and if your scripts are allowed to access files probably a file "Handle" or equivalent isn't the same kind of thing as a dictionary.

So, in this sense a class is merely a type that you can define in your program.

This has any number of useful properties, for example suppose you deal with a lot of Social Security Numbers, if you had an SSN class it'd be clearer that, while SSNs might well be "numbers" in some sense, you can't go around summing them up, any place that seems to be happening is a bug waiting to happen, as a class there's no reason it should be possible to do arithmetic on SSNs. Your chosen language might provide easy ways to say e.g. "This dictionary is for numbers, don't put anything else in it" and chances are if that's available you can likewise say "Only for SSNs" and now there's no risk you accidentally got the total number of people processed as a Social Security Number.

Does that justify classes as an idea to you?

The specific word "class" is associated with a particular approach to programming called Object Oriented Programming, which is much less popular today than it was at the end of last century, but this general idea of being able to make types for your application is more general.

To be honest, this does not require classes per se - Golang manages to have interfaces without classes, Rust has traits.

My take is that for simple stuff I don't need classes. An example is I'm writing a simple GUI calculator for my brother who's a stone mason. It's a stone calculator that accepts quality, height, width, length measurements in mm and output a tables with volume costs in either metres or feet cubed.

I could model it as a job classes that accepts a stone list class, and customer name, cubed cost, tax, parameters. And methods to get total costs, taxes etc.

The stone list class is a list of stone classes. Each stone class has quality, height, width, length parameters and methods to get cost, get taxes, etc.

But all I do is store the measurements in a list, each list (stone) is stored in another list, and I iterate. Calculations are done on adding a stone, or when you recalculate the whole list. I know it's not big, complicated, or impressive example but for me it was easier to implement it classless.

I'd say that feature cohesion is important: in Java, the new(er) features blended into the language without creating incongruities and footguns. In C++, it was not as smooth, though.

Another, and maybe more important factor, is the prevailing tastes in the community. PHP 8 is more complex than PHP 4, but the community attitude changed, many aspects of the stdlib and key frameworks evolved, and now PHP mostly shook off the stigma of being a cluster of antipatterns.

But yes, the entire OOP-classes with namspaced methods and "self", that's not inevitable

So which languages are left without object systems?

What "Classes" are is a tragedy of things.

Classes are closures, with a syntactic form that allows for nested composition. This is a particularly brittle form and these qualities are found in all languages with Classes. Language maintainers, as they currently exist, have failed to concede that this is a bad idea, because "it works good enough for lots of cases". This has hurt the industry by creating a culture that can only support the view that "all code is ugly". Given the choices, the variability in how code evolves randomly to deal with choices made under these conditions, is unsurprising.

Why? Because Classes are a good feature that has been misrepresented as a compositional tool instead of a syntactical shortcut and nobody is dealing with the gravity of that...given the amount of history, industry, effort, et al based around supporting their faux necessity, who could be blamed for not raising a stink? And why is there a "new" keyword in 2022? Python gets this right.

Almost every language ends up heaping additional syntax for behavior that is either compiler added (eg php/rust traits, ruby aspects) a meta-language (eg Java aspects, php attributes, various C++ impls, ES6 javascript classes), or a straight mixin syntax (Python decorator). This alone, is obvious that the support for composition is a major problem that has both been neglected, then mishandled by heaping on the same mistakes. Interfaces, were a particularly laughable solution, which has made the problem worse by tying that limited form of composition with reflection.

> Languages that mostly avoid mutable state don't tend to have object systems, for instance.

They do, in the form of closures. How infrequently composition-by-nesting is used in functional languages is telling (basically the inverse of nonFP languages).

May I ask you why do you think Java is an example of adding features it was not designed to have? In my opinion, even with the original warts of the language, it is one of the few long-running languages that managed to withstand adding every feature and the kitchen sink, and additions are thoughtful and deliberately slow (late-movers advantage, it let’s other languages make mistakes with a feature first, avoiding bad ideas like async).

Generics was added very cleanly to the language seeing the constraints of the time, lambdas are also more than good in my opinion (with the only wart being around checked exceptions, unfortunately). The new direction with algebraic datatypes (record, sealed classes) and pattern matching is also an example of a feature that was known forever (ML), but the general developer community got to like it only recently.

it is only bad in the context of managed languages that can do better. It is a “necessary evil” in case of rust for example

If you read just a little farther, you get that extra apparatus has a purpose that is opaque to beginners, that just adds confusion.

Sounds like premature abstraction. Instead of declaring a class, you have declared an interface.

More specifically, I think structs, or something like structs, are inevitable. And methods are also inevitable. Whether it be by OOP-style classes that contain both fields and methods, or in the Julia style where you have structs, and global-scoped functions with struct-specific methods, or in the Rust style you linked to with structs that conforms to a trait, which implements the method is not important for my specific point.

All three ways add complexity to the language, but such kind of complexity is unavoidable, because it will show up in the code anyway. Even if you use C, which does not have an explicit connection between structs and functions (i.e. methods), you will just write functions where you document that such and such argument should be a pointer to such and such type.

> I think it was literally just meant to be the bare minimum to try to solve the most important parts of the problem, and at that it succeeds.

In other words, it's the cost effectiveness of a language feature. You want features to provide as much value for as little cost as possible. This is what Go's going for; to paraphrase Russ Cox - "If a feature is not clearly above the bar, it's below it." [1].

> Just as reducing programming language complexity doesn’t remove it, but only moves it around, removing runtime cost doesn’t remove it either. And the problem here isn’t that compile time isn’t a better time than runtime; it’s the hidden costs that suck. Sure, maybe the compile times in Rust will improve, and maybe it’s not even that big of a deal. But every little feature the language has puts some additional stress on the ecosystem. The language server has to bear this load, for example.

The worst hidden cost that's often missed is the impact on the programmer.

Take Python's controversial walrus operator [2]. If you look at the reasoning and the examples, the change seems reasonable. But consider that once the feature gets implemented, every single Python programmer will have to learn what it does in order to be able to read others' code. Does this feature provide enough to justify this weight? Undoubtedly not. Of course, it got implemented anyway, because the impact of making the code slightly more readable in some cases is clearly visible, but the cost incurred on every single programmer's mind is not.

This is how languages reach untenable levels of complexity. By the time you realize your language is getting complex, you're 20 features in and it's too late to turn back. "A frog dropped in a pot of boiling water will jump out of the pot to save his own life. If the frog is put into cool water and slowly brought to a boil, he will remain there until he is cooked through."

[1]: https://github.com/golang/go/discussions/47203#discussioncom...

[2]: https://www.python.org/dev/peps/pep-0572/#abstract

I would like to know more about this, would be really useful. So you have any recommended docs to read?