Hacker News Reader

> The key problem has been that the resulting programs are fantastically hard to understand. One of the things I think Dijkstra got right in "Goto Considered Harmful" is that humans can reason about programs better when there's a strong correspondence between the textual structure of the code and its execution structure at runtime.

When I briefly worked on a Ruby on Rails app in 2015, my first ticket was to investigate a crash that had been reported. I quickly reproduced the issue and got a stacktrace that ended in a function call to "is_readable", so I said: "No problem, I'm just going to grep for that identifier to find the definition." Cue two hours of utter frustration until I learn that you can use `.source_location` on a function to get a file and line where it is defined, which pointed me to a pile of metaprogramming held together by strings and class_eval. To no one's surprise, I fully support the statement quoted above.

Yes, look at literally any complex Python library and there's generally going to be an unreadable mess of dunderscore properties and ClassFactory stuff. I've seen Python libraries that call a SQL database query and then build Python objects and code from that! What a nightmare.

Namedtuples were (are?) constructed with eval under the hood

Many years ago I worked on a Rails project where the two previous developers were particularly creative with metaprogramming. That cost a lot of time and extra money to my customer because it was really hard to figure out all the tricks they used. Instead I write Rails projects in the simplest possible way I can figure out. It works and everybody else and I will be able to understand it quickly next year. If it makes me look dull, I don't care.

In the 80s there was an April Fools issue of "Communications of the ACM" positing a "COME FROM" statement (not too long after "GOTO Considered Harmful"):

https://dl.acm.org/doi/10.1145/358027.358043

It was a joke because of course it was obviously worse than GOTO. But much of modern programming seems to consist precisely of COME FROM style constructs. A common term is "inversion of control", which can also mean "You don't have control".

As with metaprogramming, it's powerful in that it can take control of your system without your direction, but dangerous in that it's hard to predict when it will. I find that the hardest thing to debug is when things don't happen, when it's out of your control to make them happen. There's no place to put a debugger on "never happened".

Dynamic types, similarly, are hard to debug because they fail by not happening. Errors aren't caught, but there's no instant at which it wasn't caught because it was "any time".

> In the 80s there was an April Fools issue of "Communications of the ACM" positing a "COME FROM" statement (not too long after "GOTO Considered Harmful"):

1978 actually: http://www.modell.com/Magery/SPharmful.html

But that was a followup from a much older paper, from 1973: https://web.archive.org/web/20180716171336/http://www.fortra...

And to make you feel really old, "Go To Statement Considered Harmful" dates back to 1968. The context was the debate on structured programming which mostly happened in the 60s and early 70s (though it has flareups unending, they tend to be limited in scope).

"COME FROM" also made its way into INTERCAL, although I don't know the timeline.

I actually discovered it from INTERCAL, which I know to be from the 70s, but the wiki told me COME FROM was added by INTERCAL-C which was created in the early 90s, so it’s a much later addition to the COME FROM universe.

> A common term is "inversion of control", which can also mean "You don't have control".

Back in the Slashdot days I told people inversion of control means "in Soviet Russia, API calls you!"

The app based gig economy?

> The key problem has been that the resulting programs are fantastically hard to understand. One of the things I think Dijkstra got right in "Goto Considered Harmful" is that humans can reason about programs better when there's a strong correspondence between the textual structure of the code and its execution structure at runtime.

This resonates very strongly with me. I've been doing a whole bunch of side/toy projects in Rust for months, and I can attest that even compile-time metaprogramming can make programs "fantastically hard to understand". Runtime metaprogramming makes that problem an order of magnitude worse.

Don't get me wrong, I'm not against metaprogramming, either compile-time or runtime. But it's way too easy to get overly enthusiastic when doing it.

Honestly, I think runtime metaprogramming can be a lot easier to understand then compile-time metaprogramming, because it's easier to build runtime debugging features for runtime metaprogramming, compared to building runtime debugging features for metaprogramming that's already been erased at compile-time. For example, a sibling comment mentions Ruby—well, pry already supports "show-source [expr]" for basically any possible Ruby expression you might want to call, no matter how meta-programmed it might be. Navigating and debugging Ruby code can get very fluent once you have the right set of tools.

Compile metaprogramming can have compile-time debuggers. If it uses the same language for meta and regular, it should be the same debugger, ideally.

The IDEs haven't quite embraced this concept properly yet, though: they usually have distinct "editing", and "running" states, but what's really needed is "editing", "building", and "running", with debugging available in the latter two.

Completely agree. I spent a lot of time thinking about what programming language features are important for understandable code when I was designing Darklang. One of the major negative influences was from Rails, where I had often found it hard just to understand where a particular piece of code came from. Clojure is similar - with "why is there a nil here" being the challenging question. With Python, I made my own mess making things far too polymorphic.

But I had previously had experience with OO languages - Java and C++, and they often had similar challenges, though not quite as pronounced, along with problems of their own.

Eventually, I came to realize statically typed functional languages were the panacea. Sum types offered better composition than OO, and they had a good trade off between power/succintness (coming from higher order functions) and program understanding. In particular, the problems from other languages: what code is this, what type is this, where did this nil come from, were solved.

Usually though, macros in lispy languages are evaluated not at runtime, but before that, at read time or macro expansion time.

There are perfectly valid use cases for these things, which do not make it harder to understand the code. Properly done, they might even make the code easier to understand. One example is writing a "timing function" (a macro). WHy a macro? Well, because you cannot call (time expr ...) when time is a normal function, because of order of evaluation. You could put the expressions into lambda and reduce readability that way, or you could make time a macro and get rid of the lambda wrapping of the expressions.

What if we want or need code that has to run for centuries? (Just a thought experiment.) What struck me in your post was your mentioning the fact about debugging being difficult. What if we don't debug? What if we treat them as evolving systems. You can debug mechanisms, but you can mainly treat organisms. Possibly this is the actual line between mechanical and biological systems, and that a meta-programming language (the DNA) is involved is hardly surprsing.

You might like this blog post I wrote a long time ago:

http://journal.stuffwithstuff.com/2010/11/26/the-biology-of-...

Thanks, I did. (something to think about: parasitic organisms.)

Another thing to think about: software monocultures and resilience against adversarial attacks.

Hah. Interesting. But I was going the other way: parasites as groomers of long running software. (So symbiotic would be more correct.)

>What if we treat them as evolving systems.

Professor Sussman mentions this in this talk: https://youtu.be/HB5TrK7A4pI

Sort of related, I always wanted to ask someone who only have extensive experience with dynamic languages - what is your mental model when you write a parameter, variable? Don’t you inherently link some vague type to each variable? I honestly can’t think back to when I started programming (even though my first lang was php).

>I've yet to see an optionally or gradually typed language pull that off well.

Opinions on Coalton in Common Lisp?

https://github.com/coalton-lang/coalton

Never heard of it, but there are lots of other SML-with-Lisp-syntax languages floating around out there.

> The boundary between them just gets too weird

What's weird about ts with js files allowed?

TypeScript is definitely the most successful (in terms of popularity and in terms of I think they did a good job) language to try to tackle this.

But it took possibly the best living language designer on Earth to do it, the type system is (deliberately) unsound and will never be able to be used for optimization, and the type system is just incredibly complex. Literally a Turing-complete language running in the compiler.

And that's the least bad that anyone's been able to come up with. I applaud the TypeScript folks for doing it, but I think it's a worse user experience than most other modern statically typed languages unless you happen to be sitting on a pile of JS code already.

It's like plasticine with wires through it. Sure, the wires help but it would have been better to not start with plasticine in the first place.