> Programming language designers should avoid adding powerful, higher-order abstractions because they are hard to understand?
In general? Absolutely![1] Making algorithms easier for humans to understand is the whole purpose of abstractions.
Programming is based two things: algorithms and abstractions, with algorithms being fundamental to computation and abstractions are usability features designed to help people write code -- that is their one and only purpose (computers and even theoretical computational models don't care about abstractions). Unlike algorithms, abstractions are not useful in isolation, and their utility is not a function of mathematical "power" but of psychological benefit. Their utility is measured by how much they help the human programmer write and read an algorithm[2]. Another way to look at it is that algorithms tackle essential complexity and abstractions tackle accidental complexity[3].
More specifically, abstractions help human programmers in two ways: they increase code reuse and improve code readability.
But here's the thing. There are things other than powerful abstractions that help humans program -- for example, a clear execution model (what happens when), debuggability etc.. This means that the more powerful (i.e. abstract) abstractions become they do not necessarily perform their function -- namely, assisting developers -- better. So every abstraction must be carefully weighed: how much wasted code does it save, how much more readable it makes code, vs. how much does it hurt understanding or debuggability.
I'd say transducers are just about the point where the abstraction starts hurting more than it helps. It's a borderline case. Now, I am not saying Clojure shouldn't have transducers (again, borderline), but that they most certainly shouldn't be emphasized.
> if you don't understand it you don't have to use it.
That doesn't quite work. I said, don't use them right away. Once a programming language and its libraries use an abstraction, you must learn it sooner or later. After all, most code you'll read isn't your own. This is why every language feature has a cost, and why good language developers are hesitant about introducing new features (transducers aren't a language feature, but they do have a prominent place in the standard library).
---
[1]: For example, Java and Go are both languages whose designers intentionally and radically reduced the use of many of the abstractions available in other popular languages of their time. Java in particular drastically removed abstractions possible by the most popular language at the time it was introduced, and attained tremendous success, partly because of that (well, at the time it was already apparent that C++'s power -- in addition to its lack of safety -- was greatly detrimental to code maintenance in most parts of the industry).
[2]: Abstractions are secondary to algorithms. Also note how almost all sub-disciplines of computer science deal with algorithms, and just one, rather small, discipline -- PL research -- is concerned with abstractions.
[3]: Even algorithms are often not judged in isolation; there are lots and lots of useful algorithms that aren't used because they are too hard to implement and maintain correctly -- regardless of abstractions used.