Comment by flavio81 | Hacker News Reader

Comment by flavio81 | Hacker News Reader

Very good slide here:

https://github.com/matthiasn/talk-transcripts/raw/master/Hic...

Caption: "The problems of programming"

And you can call them problems, and I'm going to call them the problems of programming. And I've ordered them here -- I hope you can read that. Can you read it? Yeah, ok. I've ordered them here in terms of severity. And severity manifests itself in a couple of ways. Most important, cost. What's the cost of getting this wrong? At the very top you have the domain complexity, about which you could do nothing. This is just the world. It's as complex as it is.

But the very next level is the where we start programming, right? We look at the world and say, "I've got an idea about how this is and how it's supposed to be and how, you know, my program can be effective about addressing it". And the problem is, if you don't have a good idea about how the world is, or you can't map that well to a solution, everything downstream from that is going to fail. There's no surviving this misconception problem. And the cost of dealing with misconceptions is incredibly high.

I'd suggest to consider the diagram cited above, it is a good representation of where the real difficulties lie in programming.

I have some doubts about this one, despite my love and ongoing professional work with Clojure. For overall productivity, I find the issue of typos more significant than this slide leads us to believe, especially in a language like Clojure where the compiler does not catch nearly 90% of my typos. It's very easy to have a nil running through your program in Clojure because you either mistyped a keyword or you just picked the wrong one when pulling something out of a map, for example. But that is just one example, I find that I spend a fair amount of my work day tracking down silly typos at run time, which frankly disappoints me tremendously. I really like Clojure but tackling this problem has been non-trivial.

I think the mistake he's making is in that while his hierarchy is correct in that one bug from a category further up is far worse than any bug in a category below it, if you have many more bugs from lower categories they can start to become problems on the scale of a few bugs higher up the chain. Constantly dealing with typos is a great example because one or two would not be a big deal, but they can add up to significant mental effort and time loss.

I agree: one of the things I do appreciate in Java is that the compiler does catch those silly things and I don't really have to think about them any more. As a side effect, refactoring is very safe and you can depend on it. Clojure - that I like quite a bit - does not help me much along these lines; and this is an annoyance.

I agree with this sentiment, but I find that doing small checkins to git is helpful, with a good git ui.

Common Lisp OTOH uses defined arglists with keyword parameters, where the compiler can check mistyped/wrong keywords.

IntelliJ + Cursive makes keyword typos and most other typos a non-problem (or very easily spottable problem) for me.

How does Cursive know if you're pulling out a valid keyword in a map?

I doesn't know if :resource_id is in the map, but that's not a typo if that's the case.

What OP meant is if you type :resuorce_id it won't autocomplete correctly so you're more likely to notice.

Ah, well I get that same behavior in Emacs. Doesn't always help though.

Also if you type resuorce_id when referencing it will tell you that it cannot resolve the local. Of course if you BOTH destructure resuorce_id AND reference resuorce_id then you're out of luck :). That happens rarely though. One recent IntelliJ addition which is sometimes annoying but often very helpful in many cases like that is notifying you of grammatical typos: as long as you're not using obscure one off abbreviations, etc it will most likely catch "resuorce_id", "widht" and similar typos but might also complain about valid words or abbreviations in some niche domains.

Hum, the repl catches most of those for me almost as instantly as Eclipse would highlight it if it were in Java. And the onese that are left are annoying, but really not a massive amount of wasted time. Like possibly a lot less time then the time I would spend wrestling with complex types, or having to build generic structures so I can group disparate types together.

I'm with you here, they usually come out in the repl. There's always spec too, if it really turns into a problem.

Seems like we could have some tooling that raises a flag if keywords with a short Levenshtein distance were detected.

Ya, I'm not trying to make excuses. I welcome better tooling, or even language features that help. The problem is there, but I agree with Rich Hickey in that its not the highest in my list.

Ugh...

And people complain about Java's NullPointerException, but this is way worse.

But we still have teams spend endless amounts of time debating on which flavor of Javascript, where the braces go, or which NPM module is the most effective.

...and which programming language to use. One of the things Rich says more than once in this talk is that he doubts even that programming languages (including his own) are the most important aspect of doing our programming work well. And yet we talk about PLs as if they are everything.

There's a term for this: bike-shedding[1].

[1] https://en.wikipedia.org/wiki/Law_of_triviality

Yeah I heard him say that and to be perfectly honest I don't know if I believe he truly thinks that. This is a man who devoted years of his life to create a new programming language because he found it difficult and frustrating to use existing languages. In this talk and many others he talks about how he believes Clojure's features (like dynamic typing and dynamic dispatch and the whole keyword system and others) are really key to writing good software and how many other features (like static typing) are irrelevant or harmful... and now here he's saying programming languages don't matter much?

He mentioned he liked Common Lisp but they didn't let him use it in production, so he created a Lisp that runs on an industrial runtime.

Yeah, I understand that. But if he really didn't care much about programming languages, why wouldn't he just use C++, which he was allowed to use in production?

He talks about that quite a few times in the video.

Yes, and the things he says about it indicate he does think programming languages are important.

I think there may be a bit of people talking past each other here. I think one can reasonably say that which language one uses isn't as important as which features that language has. Also, there comment you originally responded says "that programming languages (including his own) are the most important aspect of doing our programming work well." That doesn't mean that the language or its features aren't important. They're saying that it's not the most important thing.

The hierarchy Rich refers to[0] puts domain complexity and misconception as 10× more important than anything that's language specific. I believe this is the basis of the quote I pulled. I think this is compatible with thinking that language features are important, but domain complexity and misconception are more important issues when tackling a problem with code.

[0]: https://github.com/matthiasn/talk-transcripts/blob/master/Hi...

Rich was a professor of an advanced C++ class (at NYU, I think) in the late 90s, and wrote a very popular paper on how to get some killer performance out of that language. He's been around the block and knows a thing or two about these things.

I don't understand how this is a reply to my comment.

Which is a little weird, because (as I mention elsewhere), there's Armed Bear Common Lisp, which is a Lisp which runs on the JVM. Seems to me that he could have saved a lot of time an effort just sticking with Common Lisp.

He'll argue, of course, that the time & effort are worth it for immutable data structures. I'm unconvinced, but that's the great thing: we don't all have to agree! I'm glad he's having fun.

Immutable data structures were part of it, but not the whole story. From the article:

"But Lisp had a bunch of things that needed to be fixed in my opinion. It was built on concretions, you know, a lot of the design of more abstractions and CLOS and stuff like that came after the underpinnings. The underpinnings didn't take advantage of them so if you want polymorphism at the bottom, you have to retrofit it. If you want immutability at the core, you just need to, you need something different to, you know, from the ground up. And that's why Clojure was worth doing as opposed to trying to do Clojure as a library for Common Lisp. The Lisps were functional kind of, mostly by convention. But the other data structures were not, you had to switch gears to go from, you know, assoc with lists to, you know, a proper hash table. And lists are crappy data structures, sorry, they just are. They're very weak, and there's no reason to use them as a fundamental primitive for programming. Also packages and interning were very complex there."

But notice that his arguments are always focused on the tool/feature.

You can look at Clojure as a "reference implementation" for many of Hickey's ideas. (A production-ready reference implementation, of course.)

But all of Rich's ideas/"features" can be applied in your language of choice. (Immutability, functional programming, avoid-ADTs, how one should do versioing, etc.) I've employed many of these and other of Rich's ideas effectively in Java. The ergonomics would be better in Clojure, of course, but for (probably good) political reasons I have to use Java at work.

Just notice that Rich's rhetoric is nearly always focused on specific problems and solutions that transcend Clojure.

> But all of Rich's ideas/"features" can be applied in your language of choice.

Not just can, but have been and are. Clojure is one of those languages (like SmallTalk) where, even if it's not the most "successful" or "popular" language in the long run, it has had an oversized impact on how we program in general.

Genuinely curious...what are these over-sized impacts?

Immutable data structures?

Like... Singly linked lists?

I even remember using VLists (work by Philip Bagwell, that probably inspired the data structures in clojure) in 2006, and I am not even a programmer.

No: the idea that immutable is a sensible default, and active enforcing of immutability. In Java, e.g., a linked list's cell might be immutable, but its contents are most likely mutable - so back to page one. It is definitely possible to write code for structures that are immutable by interface or convention, but the environment does not help you doing so.

Yea, not VList exactly, but similar and also pioneered by Phil Bagwell. They're implemented as a trie under the hood. And provide a list or set or stack or queue or map like interface. You can not mutate the value at any node, instead edits are diffs in the tree, a bit like git commits.

So they end up being almost as fast as their normal mutable counterparts, and with similar and sometimes better memory consumption, because nothing is ever copied under the hood.

This allows them to be good enough to be made the default datastructures. Clojure was the first to make this gamble, and show that the state of the art immutable datastructures mostly pioneered by Bagwell were practical and could replace immutable ones for the 99% use cases.

Clojure did not invent immutable data structures!!

I think Clojure contributed to the functional and meta programming resurgence. It didn't innovate on any one feature, but it brought a whole lot of old forgotten ideas back together, and showed their value.

Somehow a language is only an attempt at incarnating principles. It also evolves to go closer to that.

They may not be the most important, but they have a big impact on our ability to map the domain/problem into a solution, and it has an impact on the ability to maintain the solution when the requirements change (either the domain has changed, the users need a solution to a slightly different problem, etc.).

Being able to understand the domain and map it into a solution for a particular problem has to be taken on a case-by-case basis. That's up to the individual company/team/developer to handle.

Programming languages are useful in the sense that once a solution has been decided on, they can help to express it in a clear way.

In fact, look at Clojure as an example. Even if you have a fantastic understanding of the problem, domain, and (high-level) solution, implementing it in Java can still be problematic, as Rich discovered. Getting concurrency right without having problems with race conditions and deadlocks can consume enormous amounts of time. Having a language that makes concurrency easier (Clojure) means that you can spend more time focusing on the actual problem to be solved, than with low-level implementation details.

Also, another thing to keep in mind is that often times a good way to understand a problem and its domain is to try to write a solution, and then see where it falls short, and re-write it, etc. This prototyping phase can be helped by good languages.

So maybe programming languages aren't the most important aspect in programming, but it's definitely important enough that it shouldn't be considered bike-shedding.

Reply on news.ycombinator.com