My Increasing Frustration with Clojure
ashtonkemerling.com
ashtonkemerling.com
The underlying problem Clojure/Core has is their communication. If they would come out and explain their philosophy then people wouldn't get frustrated and confused when they expect the language development to work like other open source projects. Clojure's development is functioning exactly as planned. It's not a mistake.
A better way to treat Clojure is closer to a project at Apple (except for Swift). You can submit bugs, and make suggestions for future improvements, and if you really want to provide a patch. But go into it realising that it's not a community project, and you're much less likely to get frustrated.
With all that said, the proof of the pudding is in the eating, and for the most part Clojure has developed pretty well from Rich's tight control. I'd love it if there was a Snow Leopard year for Clojure where the focus was fixing longstanding bugs and feature requests, and more focus on those in general, but the language is developing well.
The moment you get over expecting Clojure to behave like a typical "open source" project and understand it is Rich Hickey's personal project (+ Cognitect's), which happens to also be available to you, if you want to use it, then the whole thing makes more sense. Some frustration disappears. You know where you stand, and what to expect.
Hickey and Cognitect have every right to run it the way they like. IMO, the only thing they don't do right is to communicate their intentions. There's a pretense maintained around it being an "open source" project, when it really isn't, not in the way that python is open source.
Use it if it works for you. Otherwise don't. Don't expect to have a voice, and don't expect you'll be able to contribute much, and don't expect that something will be fixed unless it suits Hickey's Cognitect's agenda.
I use ClojureScript, knowing and accepting these things (with some sadness), because I think the language gives me a competitive edge.
Then people's code got way slower. Why? Well this was a often used function and that single if blew the JVM inline budget causing that function to never be inlined. And that improvement had to he yanked out.
When we talk about patches for a language, we have to realize that if we don't want constantly breaking APIs we have to test that a patch doesn't break existing code, that it also doesn't slow down existing code (or at least not enough that users would care), that in all sorts of cases the JVM won't freak out and do something bad. Then we also have to make sure that the change doesn't restrict the language in the future. Does making some interface public mean it will always be public? Are we happy to keep it that way for all time?
Taking all that into consideration, I have to sit back and say , yeah, maybe I'll just remember to only hand sets to the functions in `clojure.set`. Or maybe I'll help out by adding clojure.spec annotations for these functions so I can turn them on during testing.
In the end, I'd much rather have a rough-around-the-edges language that takes a garbage-in-garbage-out approach, than one that breaks the API at every turn. Perhaps they aren't mutually exclusive, but it certainly takes a heroic effort, and a lot of prioritization.
For some reason the blog's author dismisses spec as irrelevant greenfield development, when it would quite neatly solve many of his complaints
Also, it was ironic to read elsewhere in the article, "Even more obnoxious, it was closed as wontfix. Apparently a single sentence in the docs is good enough for the Clojure team..." Then in the next paragraph: "I was treated pretty shabbily by..."
(One can go on. Take the headline, "Ignorance or Apathy of Underlying Principles")
Calling hardworking people (who do free work for one's own company) "obnoxious" and ignorant/apathetic seems shabby too. While demanding "fixes" from them, and putting down their "big highlights from the past year or so" (Transit, Transducers, Spec) with a backhanded "These are okay, I guess." I'd never tolerate a client/manager with that attitude about my hard work, and that's someone paying me. No wonder free software maintainers burn out.
Me being a pretty average typer (~95 WPM) I could type 2-3 paragraphs detailing why I am against fixing anything, in no more than 6-8 minutes.
So for 7 years such bugs to never even get a proper explanation is showing the maintainers as douchebags, not victims.
I am not gonna teach anyone how to organize information. For the time being I am writing absolutely everything that comes to my mind (or is raised by customers and teammates) in a private Trello board. Even if I close everything in it 3 years from now, I would have some peace of mind knowing that I tried my damnest to never miss anything. And that's it's written down somewhere and that I am aware that it's waiting my input or work.
I was somewhat offensive simply because bugs 7 years old marked as "wontfix" without a tangible explanation is absolutely not OK for language maintaners. The jobs of the senior devs are usually very hairy because we're good in fixing messes and repairing things. I stand behind my opinion that you have to go back and fix your past mistakes every once in a while. You can't pretend you're 18-20 years old and only sprint ahead and be blinded by the glamour of the new and shiny features. Stuff periodically needs repairing, it's how it is.
Seriously though, I don't understand why is this being downvoted on the grounds of a subjective opinion. I stand behind my philosophy and I am returning periodically to fix things I've done in my past, both in my profession and real life.
You're free to disagree.
In any prolific, underfunded work, there's always things to pick at.
Please edit out the "douchebags" comment. If not for ethical reasons, then for the fact that some of them post on HN, read such words, and probably be momentarily demotivated/distracted. Not to mention HN's guidelines on civility: https://news.ycombinator.com/newsguidelines.html
My apologies. It definitely won't ever happen again.
If you have a chance, could you please take care of updating the docs? Thanks in advance! Let's make Clojure great together.
I'd love to help but as it is, the original article convinced me that I shouldn't. As a commercial project maintainer myself I am all too well familiar with how many things one can miss if he doesn't write everything down and constantly manage information -- that's why I have a huge private Trello board.
I am not gonna scorn fellow extremely busy devs. But I will point out that as language maintainers your responsibility is bigger than mine. You should do your best.
If I ever decide to become an active Clojure user I definitely won't be only criticizing. I'll do my best to help!
It's my opinion that I don't have to be a Clojure expert with 5 years of experience in order to be able to recognize patterns in the language implementation and the way the ecosystem is governed, and eventually deduct I wouldn't want to associate with them for the time being.
That's over the line. Please edit out name-calling from your comments here.
I apologize. I'll definitely remember the guidelines for the future. Thanks for pointing them out for me.
(BTW, I already can't edit the post. Ouch.)
(Since people sometimes misunderstand this, I'll clarify that it isn't foul language in the sense of profanity that's the issue here. It's treating each other with respect.)
Not impossible, but one of the heavily advertised virtues of Clojure is that it can be deployed alongside Java code without anyone having to be the wiser.
In the former, any deviation, including bugfixes, from the existing implementation is a breaking change. Indeed, without a specification, it's debatable whether any behavior is actually a bug or not.
If you're giving sequences to the set functions, you either (A) don't know what you're doing or (B) have yet to discover that clojure.set is not what you want.
Set union can't be done efficiently on arbitrarily sized sequences. At least one of the two arguments needs to offer fast set membership; ideally the larger one, hence the requirement that all arguments be actual sets. If you have sequences, you need to make an explicit decision to pay the cost of indexing (eg. via set or into functions) or you need custom code for your use case.
Sure, static or dynamic enforcement could help catch silly errors here, but then the author goes and poo poos on "greenfield" development of things like spec, despite the fact that would "fix the bug" exactly as requested by throwing when you gave non-sets to the set functions.
I've followed a lot of the discussion around the supposedly horrible way the Clojure community is run. So far, the absolutely only complaint that I can sympathize with is that communication could be faster, clearer, and softer. Alex Miller has made dramatic strides in improving this, but he's just one man. Cut these guys some slack, they work crazy hard to build and shepherd something awesome for free.
I'm not so sure about that. I'd say that Clojure now exists and thrives to help Cognitect make money. This is no way a bad thing, only a reminder that there is no such thing as a free lunch.
In a similar vein, Go exists to help make Google money. The original purpose of open sourcing it was different for Google (make the language more robust/better) than it was for Hickey (grow the language/community).
Clojure is _free_. If you don't want Cognitect stuff, don't use it. If you don't like how Rich/Cognitect et al are running the project, take your EPL licensed ball and go home. Similarly, in the case of Go: Substitute Rob/Google and BSD-ish license.
Just because these languages serve the interests of their respective maintainers does not make them any less free to you. It may make the community less useful to _you_, but hey, they created these things to be useful to them. There's an implicitly mutually beneficial social contract around contributions and community. If you don't like that contract, don't use the language.
In the very unlikely case that Cognitect or Google puts some company-specific stuff in the language in a way that actively destroys the good will of this social contract, then and only then do you have any right to complain.
Of course not, but it does make them unencumbered. And I'm not complaining. I'm just saying that's cost (i.e. not free) of doing business with a sponsored open source language.
I never said "nothing is free", nor implied that, but merely that users need to cognizant of the goals of "free" language maintainers (especially when there is a corporation rather than a person as BDFL) that do not necessarily align with the goals of the median or average user.
Now that we are on this topic, I'm starting to think maybe it's worth paying for a programming language so that the goals of the maintainers are better aligned with the goals of the users. That being said, there is real value to the ability inspect the source. As such, it's very difficult to do open source as a paid product.
1. https://web.archive.org/web/20100220100027/http://clojure.or...
2. https://web.archive.org/web/20140704050611/http://clojure.or...
My own interpretation: programmer culture has such problems that someone actually considered corporate funding healthier than user donations.
I'm not so sure about that. I like the 'baseball cap' argument more. The quote is about React, but it may be relevant for Go as well
> Facebook doesn't care if you use the Web, it only cares that you use Facebook.
> So why release an open source Web framework at all? Because Facebook is battling Google for engineers. So you've got a big fight between two companies over which company is the coolest place to work, and both of them are companies that your grandparents love. How are you going to win this fight? One way is to have the hippest Web framework.
> Basically, both Google and Facebook are desperate to find a baseball cap that they can put on backwards. Angular is Google's baseball cap. React is Facebook's.
That can also be said for C, assembly and almost any language, even Brainfuck.
And when there's not a speed/memory tradeoff for not having the language do the right thing (like e.g. is the case for some decisions in C), this just amounts to having the programmer do "busywork" -- manually do the work that the compiler/interpreter should have been doing instead.
Blaming the programmer for such errors is thus a kind of "blaming the victim".
The approach "The language X is totally fine, it's the programmer's problem who doesn't know what they're doing" for such cases, that is, cases where:
1) the right thing/warning could be inferred automatically
2) there's no (or not significant) speed/memory tradeoff to do so
is a bad idea and those that propagate it should feel bad.
Regarding misusing the set functions: There is no busy work. Instead of (set/intersect some-set some-list) you just need to do (set/intersect some-set (set some-list)) to explicitly opt in to the speed tradeoff of performing set operations on lists.
> is a bad idea and those that propagate it should feel bad.
I don't feel bad about it at all.
If Clojure hadn't followed the garbage-in/garbage-out strategy, it would have been too slow for production use. If they had added strong types early or Racket-style (read: slow) contracts, then we'd never have had the years of experience that led to the design of spec, which is new take on the problem space.
In the case of the set functions, the right thing _can not_ be inferred automatically without a speed tradeoff.
I noticed that as well.
This made me very curious about efficient set union. Here's a pretty straightforward set union in python:
def set_union(coll1, coll2):
result = set()
for item in coll1:
result.add(item)
for item in coll2:
result.add(item)
return result
Regardless, the article strongly suggests that one of the two arguments to clojure.set's union does offer fast set membership, since only the second argument is permitted to not be a set.
cljs.user=> (source clojure.set/union)
(defn union
"Return a set that is the union of the input sets"
([] #{})
([s1] s1)
([s1 s2]
(if (< (count s1) (count s2))
(reduce conj s2 s1)
(reduce conj s1 s2)))
([s1 s2 & sets]
(let [bubbled-sets (bubble-max-key count (conj sets s2 s1))]
(reduce into (first bubbled-sets) (rest bubbled-sets)))))
nil
cljs.user=> (source clojure.set/intersection)
(defn intersection
"Return a set that is the intersection of the input sets"
([s1] s1)
([s1 s2]
(if (< (count s2) (count s1))
(recur s2 s1)
(reduce (fn [result item]
(if (contains? s2 item)
result
(disj result item)))
s1 s1)))
([s1 s2 & sets]
(let [bubbled-sets (bubble-max-key #(- (count %)) (conj sets s2 s1))]
(reduce intersection (first bubbled-sets) (rest bubbled-sets)))))
nilThe third branch ([s1 s2]) adds the elements of the smaller collection one by one onto the larger collection. Conceptually, it does this by constructing a new copy of the result for every addition, since conj is not destructive. It is buggy and will return a result of whatever type the larger collection is, with data being duplicated if that type allows for it.
My question was, why does efficient set union of two collections require one of the two collections to support fast membership tests? As an answer to that question, you've posted code that doesn't involve membership tests at all and supplied no discussion of why it is or isn't efficient.
https://github.com/clojure/clojure/blob/master/src/clj/clojure/core.clj#L75
https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/RT.java#L661
https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/PersistentTreeSet.java#L51
https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/RT.java#L667
Sorry, I misspoke / used bad shorthand. Union should have efficient set addition for the larger argument and set _intersection_ requires efficient membership tests.
That's not how persistent data structures work
http://hypirion.com/musings/understanding-persistent-vector-...
> why does efficient set union of two collections require one of the two collections to support fast membership tests
I haven't thought much about this but it seems intuitively true to me. You want to take the smaller set, and add each item to the larger set if it doesn't already exist. That last part is why you need a fast membership test.
If you're still unconvinced, try loading up two similarly structured database tables with a million rows and get the union of them (with and without indices). The indexed one will be orders of magnitude quicker.
Telling people that they're not doing a good job when you don't even know them only pisses people off, and eliminates any chance that they'll agree with you on principle.
I was hoping highlighting that would make you realize that all you were saying to me was that I'm not doing a good job, but you didn't seem to get it. Each time I summarized what you said before, you changed your story to make it sound less bad.
Ultimately I don't really care that much, this is the Internet and I don't have to interact with you after this. But I'd recommend not doing that to people IRL, they'll resent you.
Funny, that's exactly what the original poster did to the Clojure core team.
> Each time I summarized
You call it summarizing, I call it mischaracterizing. I stand by the position that if the original poster thinks that clojure.set is bugged: they don't know what they are talking about. That doesn't mean they suck at their job, that means they haven't actually stopped to think about the particularities of those functions.
Have we all forgotten the principle "make the minimum amount of WTFs per minute" already?
Why is everybody so content with their technology and so quick to look down upon people who aren't familiar with the traps in it? Is it really so wrong for your technology to be easy and painless not only at the entry level, but all the way to middle-high level of professionalism?
No, that would be _wonderful_. Unfortunately, "easy" is only one dimension along which we must make tradeoffs. Clojure has chosen one point in the design space for input validation vs performance. You're free to have a different preference.
As a relatively mature man and a programmer for me the way a project is maintained is just as important as the ability of the technology itself.
This post helped me make a conscious decision to not work with Clojure. If I am gonna learn a new language and technologies I prefer the least possible amount of surprises, because when I inevitably stumble upon them I'll be shamed as "not active enough to research beforehand".
Yes. In fact, this is partly what clojure.spec is about; which the original author calls out as being wrong to prioritize for some unknown reason.
I have zero idea if that's the case, though. I'd be glad to work with a technology if I had a polite and constructive person such as yourself to help me out when I am really stuck.
just a heads up.
For context, these arguments made by the original poster have been around for _years_ in the Clojure community and, frankly, those of us who have also been around for years are tired of hearing the same old complaints. It's particularly annoying because: the core team has made multiple announcements in past days/weeks about improvements to exactly what he's complaining about!
Oh, goodie. I'm part of an elite club with high barriers to entry.
There are several ways to solve it all with tradeoffs. Types can solve it with tradeoffs. Checking the types in the function can solve it with tradeoffs. Stating in the docs what arguments may be passed is another way to solve it.
Steve Yegge was making similar criticisms years back, saying Clojure was "user-hostile". That's probably not fair and largely a matter of opinion (see the link for a back and forth), but it is another piece of evidence that says the Clojure community does things their own way. Not shocking, it's a lisp!
https://www.reddit.com/r/Clojure/comments/3jjit5/what_does_s...
Clojure development is opinionated, for good and bad.
> The bugs that will be fixed and the features that will be added are the ones Hickey cares about
Those two statements do not compute. A community driven language, or any technology that wishes to attract and retain users, really, should care about the things that its users care about. I can understand that the significance of these bugs might not be well understood by Clojure's maintainers, but that is the problem here. It is the task of technical maintainers to care about the problems of their users (speaking as the maintainer of several projects).
It's also worth answering that argument that Clojure is opinionated and therefore the maintainers don't need to fix certain things, because... opinions. This is bogus. Opinions are what lead you to build certain features, like transducers. Caring about quality is what should lead you to make sure they work in a way that is expected (based on how others parts of the technology work, what is least surprising to users, what is documented, etc.)
The only example given for a bug is not even a bug but undefined behavior triggered by undocumented API usage.
If the author means clojure.spec by "greenfield development" (I don't see what else he could mean) he should really reconsider his complaint since clojure.spec is going to do what he is asking for (throwing a runtime error during development if you use clojure.set with the wrong datatypes).
My impression over the last few years is that the core team became truly excellent in reacting to users needs and reports often instantaneously, especially Alex Miller and David Nolen on the CLJS side.
C gets a lot of grief about that sort of thing. It's interesting to see current languages taking the same approach.
* standard-defined. You can rely on this; if the code you write is limited to this, you're good to go.
* erroneous. The compiler is supposed to give you an error message when you write code like this.
* implementation-defined. A particular compiler can do something reasonable with the code. If you write code depending on implementation-defined behavior, you're ok, but your code is not portable. A compiler that doesn't do something reasonable in an I-D situation isn't required to generate an error, IIRC. On the other hand, whatever it does is supposed to be consistent.
* undefined. There are explicitly no requirements on the compiler. Whatever it feels like doing is fine with the committee. There's no requirement for a compiler warning or error (and by default, there probably won't be one).
The reason for the different categories is twofold:
1. C is a systems language. It's frequently used to do things that depend entirely on the specific hardware the code is running on. The result isn't portable, but in this case, it isn't supposed to be.
2. Performance. The idea is that the compiler can take portable code and produce a program that runs very quickly on the specific hardware. As long as the code doesn't do anything non-standard, the results are guaranteed to be good.
An example of the latter is signed integer overflow. (I can't remember whether it's I-D or undefined, but...) If you do MAXINT + 1, the resulting value can differ on different hardware; it could be the most-negative integer, it could be zero, it could cause a hardware trap. Now, the C standard could specify what the behavior should be, say generating a signal that kills the program with an error. But that would be very expensive on hardware that doesn't trap signed integer overflow---it would require a check after every arithmetic operation. So the standard doesn't say what should happen.
Another example is poking at memory that isn't allocated by your program (on the stack, or the result of malloc, say). You need to be able to express that for systems programming, but a general standard can't say anything reasonable about it.
As a result, C has a lot of unsafe corners where you can unintentionally invoke I-D or undefined behavior. In the first situation, your program will break if you try to port it. In the second, your program might work, most of the time. Or it might toss a runtime error. Or it could generate mangled results.
Or it could have a really nasty and embarrassing security hole.
As a result, C is regarded as a difficult and (unnecessarily?) dangerous language. And most languages that aren't C and aren't aimed at systems programming try very hard to avoid undefined behavior. Even if you manage to avoid security holes, the bugs resulting from mangled results are a royal pain in the ass to track down.
Clojure tries to put as many user faults into "erroneous" as possible but in the OPs complaint it doesn't have the required information at compile time because it is typed dynamically. This is the fundamental problem with dynamic typing that more problems keep ending up in runtime. Behavioral concerns like security etc. are then often addressed by runtime checks.
Clojure avoids those runtime checks everywhere for performance reasons - It is a clear design choice. So your observation seems correct that the "C approach" is used in the dynamic parts of Clojure.
Since we don't have a standard, the official documentation is the contract we should rely on and very soon clojure.spec, which is intended as a "standard as data", is going to be.
I would absolutely disagree with this. Clojure has long had a Garbage-In-Garbage-Out philosophy, which puts it clearly into "undefined" territory. The examples with clojure.set are pretty clearly GIGO in my opinion, but "erroneous" would mean a runtime check and an exception clearly stating which invariant was violated, not returning something totally unexpected.
You could argue that it's under "implementation-defined", where the number of implementations is one, but I don't see how you could consider the set operations accepting and returning non-set things as consistent.
I also disagree about macros, too - I spoke at the conj last year about that, and that talk seems to have been at least part of the motivation for developing spec. I hope that things will improve as spec becomes more widely used, but right now the situation for macro error messages is pretty bad.
This taxonomy is much more comprehensive, and you've laid it out well. Thanks.
You only saw one example bug? He linked to 5 tickets in the section "show stopping bugs remain untouched".
EDN, Transit, reducers, transducers, clojure.spec
(I'm not saying any of these things are bad or anything, just pointing out that there was a lot of green field development done in the past few releases)
Clojure is beautiful, and backed by beautiful theories, but its implementation needs to be clean or it will eventually develop enough special cases that its beauty will be ruined. In some sense, you could say this is what happened to Ruby, though there the allowance of "special cases" was more deliberate. I recall, back in 2003, reading the interview between Matsumoto and Bill Venners: http://www.artima.com/intv/ruby.html -- and I thought that Ruby had a beautiful philosophy. But having worked with Ruby for many years, I now see how much it is undermined by the many special cases it allows, and the wild ambushes that its most imaginative features allow (monkey patching).
I don't want to see Clojure go down that path. I want Clojure to remain beautiful, and that means the implementations must also be beautiful. There is a limit on how much the core team can say "That bug is only an implementation detail". If you take away all the implementation details, then Clojure is only a theory, not a real technology that can be used by real people to do real things.
In a truly ideal world, Clojure could be the practical Lisp that pushes itself to bring in as much of the experimental Lisp's beauty as is practical. I would love to see the best ideas from Racket and what Fogus refers to as the Fluchtpunkt Lisps:
http://blog.fogus.me/2011/05/03/the-german-school-of-lisp-2/
These are ideas that can make the world better. I have high hopes that Clojure will be the project that makes some of these beautiful ideas practical. I will be very sad if Clojure ends up like Ruby or Groovy or Scala, becoming a bit too muddy to reach its full potential.
For quite some time golang specifically set self-hosting as a non-goal. The rationale was that C beat go in many areas that were important to compilers (raw speed, I would guess, initially). It's only recently that they started self-hosting.
So just because you can self-host doesn't mean you should. The benefits have to outweigh the cons. Like any project, choose the best language for the job. That said, I have no idea whether the Clojure system would be better off using more of its own features or not.
(0) Lisp, Scheme, Forth: “A small extensible core language beats a fixed large language.” (While Common Lisp is actually pretty large, there's a rather small subset of it from which the rest can be built.)
(1) ML, Haskell: “Let's see how much we can do using types as our primary abstraction enforcement mechanism.”
(2) C++, Rust: “Abstraction isn't at odds with efficiency and fine-grained control.” (Runtime enforcement is, though.)
What does Ruby have to offer? Matsumoto claims to prioritize “developer happiness”, but allowing arbitrary modifications to arbitrary parts of any program doesn't make everyone happy. And, even if it did, “developer happiness” isn't a technical criterion anyway.
It's a guiding principle, it doesn't have to be a technical one.
For what it's worth, I'd say Elm also has this philosophy, possibly Elixir too (though I'm not as familiar with Elixir).
Ruby has a beautiful and consistent object / module system, a beautiful integration of functional and object oriented programming paradigms, and extreme powerful metaprogramming. All of those things are pretty wrapped up in its design philosophy which contains a whole bunch more than developer happiness.
Yes, Ruby has warts. So do all the other languages you listed. It is helpful to be able to see both the good and bad in languages and not get hung up battling them against each other.
I don't program much with objects, so I don't know what to expect from them, but after programming in a language that treats modules seriously (Standard ML), I can't accept anything less than the following from anything that calls itself a “module system”:
(0) Implementor-side abstraction: Fine-grained control over which implementation details are exposed to different parts of a program.
(1) Client-side abstraction: The ability to define a module as a function of another, yet to be supplied, module.
Ruby's “module system” doesn't quite cut it.
> a beautiful integration of functional and object oriented programming paradigms
Functional programming in Ruby is possible, but a gigantic pain:
(0) Partially applied functions are mildly inconvenient.
(1) Higher-order functions that recursively pass themselves a modified version of their function argument(s) are not so mildly inconvenient.
(2) Now imagine actually debugging such beasts when you make a mistake. Unit tests don't help because they overemphasize the concrete cases, when higher-order functions are all about abstracting as much as possible.
At a more fundamental level, functional programming is value-oriented, and programming with compound values (not the same thing as compound objects) in Ruby isn't the path of least resistance, to put it mildly.
> Yes, Ruby has warts. So do all the other languages you listed.
I wasn't commenting on the languages themselves, but rather the philosophy of their designers. C++ has more than “warts” - the language as a whole is a monstrous abomination, but the philosophy of not accepting unneeded performance overheads is aesthetically pleasing IMO.
Other beautiful philosophies include:
- Logic programming. Although Prolog strayed from the path in the name of efficiency, approaches like minikanren seem more "philosophically pure".
- Term rewriting. I've only used it in the form of Maude, but Pure looks like a more practical implementation.
- Proof assistants. The (tongue in cheek) philosophy is something like "once it compiles, we don't even need to run it". Agda is probably the most "pure" example which is still maintained. Less elegant examples are Coq, which uses imperative metaprogramming, and Idris which has no qualms with switching off checks in the name of pragmatism.
I know neither Maude nor Pure, so I can't comment.
Coq is ridiculously powerful, but hardly beautiful.
I agree, hence my qualification that it relies on an imperative layer (Ltac) to do anything useful, like dependent pattern-matching. I've edited my potentially ambiguous phrasing.
A bit like communism.
Whenever I watch one of Rich Hickey's signature talks, I'm enthused, but actually looking and using the concepts leaves a rather stale taste compared to the promise.
The thing i love most about clojure is that all the pieces seem to fit together and be working towards a grander architectural vision much larger than just a language or just a database. I use Clojure not because it lets me code my java/javascript stuff a little better, but because this grander vision makes impossible things possible. The thing I am trying to build is not possible in any other ecosystem today.
So I am willing to eat the many little pains described in the article, pains I feel every day, because the grand vision is so powerful, and hope that one day the little stuff will be better.
Even one of the most prolific Clojure users with a tremendous track record of extremely well engineered and practically useful libraries, Zach Tellman, has been bitten by Rich Hickey's / the core team's NIH syndrome[0]. That's no way to treat the people who love your language.
http://dev.clojure.org/jira/browse/CLJ-1517?focusedCommentId...
Author of Clojure fork (cursive is mine)
> Clojure is Rich's personal project, which he is so kind as to share with the rest of us and in in its refinement Rich chooses to accept some amount of input from us as users. I'm somewhat ashamed to admit that it took me two years to realize this
https://www.arrdem.com/2016/02/22/clojarr_-_a_friendly_cloju...
Edit: You really don't need an IDE for clojure. A repl and a text editor are your best friends IMO
Have a friend who as learning to program. He got Cursive going in less than 30min
I'm not saying that to be snarky, but I'm just pointing out how quick it actually is.
brew install leiningen
lein repl
<begin learning clojure>
The thing is that most Clojure users are/were heavy Java users and they feel awkward without their IDEs and also some have a big java project that hey mix with clojure.
I do think that Cognitects interaction with the community could be friendlier in terms of bug reports and community patches. They have a bit of a history of not communicating what they're doing (for example, the Zach Tellman thing someone else mentioned). I don't think they need to change much to fix this, they just need to communicate what they plan to do sooner (in the Zach case, they could have told him sooner "hey stop working on that, we'll think about it but aren't sure, it might take some time" so that he knows not to spend a year on it before it gets rejected/NIH-implemented). A tiny change in behaviour would go a long way. Having said all of that, I still <3 Cognitect and Rich and think they're doing great work.
I also think that Clojure has a bit of a stability/quality image problem that could be easily fixed: They should put a feature freeze on the language for ONE release and then spend 6 months or so resolving JIRA issues (bugs, stability, quality; not feature or performance tickets). By spending one release cycle on a "bug bash", I think would show the world that they care and that yes, Clojure IS solid.
I've never personally been bitten by any Clojure bugs and don't think it actually has of a quality problem at all. In my experience, Clojure is of excellent quality. Its just a perception thing, really. Sure there of course are bugs, but so does every other language.
After the 1.9 release is out would actually be an ideal time: spend 6 months on quality and call it Clojure 2.0 ;-)
After that, I think people would be much happier to receive new features.
I don't think Cognitect will do that though, at least, going by how they've worked in the past and honestly I won't lose sleep if they don't - I'll still love Clojure and will still have buckets of respect for Rich and his team. However, I do think it would benefit the community (and by extension the ecosystem and language).
I actually have encountered one; the cl-format function is documented as "implements the common lisp format function", but does not comply with that function's spec. (Unless they've fixed it since I ran into the problem in 1.4, which is sounding unlikely.)
class Set<A> { public Set<A> union(Collection<A> c) { ... } }
class SortedSet<A> extends Set<A> {
@Override public SortedSet<A> union(Collection<A> c) { ... }
}
e.g., in your example, because your returned type is Set, you presumably would not be relying on the result to be a sorted set! -- so there would be no problem. if you WERE relying on it being a sorted set, you would have to explicitly downcast -- i.e. would pointing a gun and at your foot and pulling the trigger. maybe you know it's unloaded, but the danger is clear from the type system. ;)
(0) We don't need no stinkin' abstractions! Obviously not befitting a high-level language like Clojure, move along.
(1) We enforce abstractions in our heads. Running into an undocumented scenario is undefined behavior. (Yes, in the C sense.)
(2) We enforce abstractions via compile-time checks. Clojure won't have this anytime soon, move along.
(3) We enforce abstractions via sensible coercions (whenever possible, if desired) or runtime exceptions (anywhere else).
Like the author, I find (3) the most sensible choice, and I'm surprised that the Clojure team would rather choose (1), which is known to be a major productivity killer.
Design bugs, not implementation ones. In other words, the author of the post acknowledges that the existing implementation does what Hickey wants, but the author would prefer that it did something else.
(0) Concatenation, which acts on sequences, and is neither commutative nor idempotent.
(1) Merging, which acts on priority queues and multisets, and is commutative but not idempotent.
(2) Union, which acts on sets, and is both commutative and idempotent.
There are some sensible coercions:
(0) Every set can be coerced into a multiset in which every element occurs exactly once.
(1) Every priority queue can be coerced into a sequence by lazily removing the minimum element until the heap is empty.
(2) Every sorted multiset can be coerced into a sequence by traversing its elements in order.
(3) Compatible coercions, like (0) and (2), may be composed.
But there's no sensible way to coerce a sequence into a set. The elements of a sequence might not even have decidable equality testing, which the set abstraction needs to ensure that no element is duplicated. The union of two things that are neither sets nor coercible to sets should be treated as an error, to be caught either at compile-time (which Clojure won't do, and that's fine) or at runtime (which any sensible dynamic language should do). Protecting abstractions is fundamental if you want to avoid bugs.
I really was expecting belt and braces defensive error checking and never ever giving a dud result.
Because the bottom-level operations won't let you do things that are completely idiotic, everything is "safe", so maintaining the sanctity[1] of higher level abstractions sometimes gets lost in the shuffle.
I once wrote a blog post about related issues.[2]
[1] Ok, so there are lots of terms with just slightly the wrong meanings. I came up with a new one. Sue me.
[2] http://maniagnosis.crsr.net/2011/06/some-lisp-suggestions.ht...
My frustration came from the lack of a type system, and that there's no "myThing." (myThing-dot) to give you intellisense. Even many dynamic language editors have some variant of this now. So coding in Clojure involves too much looking stuff up and memorizing stuff, and is way slower than it should be in 2016.
In Scala, Java, C#, F#, C++, I can do `myArray.` and it shows all methods available to arrays. No Clojure editor has anything like this. JetBrains editors take it a step farther and even include "code template" options that apply. So typing "myArray." will also include options for "foreach, for, for-reverse" templates. This, in addition to automatically importing any required library for the method you select.
With Clojure, there's no "dot". You're stuck hunting through documentation, trying to figure out first which namespace your function belongs to, then requiring it, then figuring out which function you want. Cursive only helps out with that last step, and even then, it displays every function in the namespace, when generally you only want to see the functions that apply to your variable.
While I'm all about immutability and structured data types, I've grown slightly away from functional-first languages in general for that reason. Even in F#, you've got to do `myList |> List.map ... |> List.filter ...` etc, requiring more lookup than C#'s `myList.Select(...).Where(...)`. The "dot" is an underappreciated aspect of OOP.
Clojure does have a lot of amazing people though, and I've been very welcome reaching out to people for help as a whole.
This exact HN discussion (and the original article) actually made me stay away from Clojure for good. I was curious, I wanted to learn the LISP way of doing things but I'm not gonna pretend that I am OK with people dismissing the possible irrelevance of their language by a death of 1000 paper cuts -- of which several were outlined the by the original article's author.
Same thing like the game Mortal Kombat X. Bought it for both myself and my girlfriend and we love it but it turns out Warner Brothers & Nether Realm Studios figured they'll support it only on the console almost immediately after it was released on PC.
I have enough money to buy 5 PS4 machines if I want to, and to buy one MKX copy for each. But I won't. I won't support people who treat their users/customers like that.
Same goes to Clojure. Same goes for Rails in the recent months when DHH actually has to write articles defending his strong opinions.
> If you think you know how I ought to spend my time, come over and mow my lawn while I expound on the problems of dev entitlement culture.
All due respect, but I don't think Rich gets it.
Other people are spending _their own_ time as well, researching, writing and reporting issues, and in many cases, would be perfectly happy to submit fixes for those issues (and in some cases, they have). It doesn't take any more work for Rich or someone else to look at an issue and say, yes, this is a problem and thank you for fixing it, then it does to say screw you, WontFix. And that is where Clojure has a problem.
Rich's argument is really a strawman. Nobody is telling him what to fix, but neither can anyone force him to recognize the importance of fixing long outstanding bugs _in his own work_, even if, thanks to a great community, fixing those bugs wouldn't require him to lift a finger.
it seems all peopme are really crying out for is clear and accurate communication. i mentioned this in another comment that this attitude is a stark contrast to the f# and racket communities.
Clojure is far from a perfect language, but what language is? I also love C++ and you'd be hard-pressed to find anyone who thinks it is perfect.
There are flaws and unique quirks with any programming language. A good language is like a person: it has a personality with all the good and bad that come with its unique bundle of traits.
I left Clojure because its inherit both Java and Lisp flaws: memory hungry, difficult setup, enterprise frameworks and hard to read, impure and slow.
I believe that there still chance for Lisp in Web Assembly.
He maintains projects by trying to triage issues in an efficient and emotionless manner. I wouldn't take it as rude though.
As someone who has been using it professionally for more than three years, there is a ton I have to gripe about in Clojure. Personally I find the hodge-podge type system and inconsistency with how certain core functions handle associative data structures infuriating, and schema/clojure.spec on some level I see as a hacky half-measure, and yes parts of clojure.test are semantically inconsistent and awkward to use in ways that bug me. But: as a language allowing me to get things done in a professional context, as a community of developers with actual adult leadership who are generally welcoming and share a practical, thoughtful philosophy, I think it holds up just fine against other language ecosystems.
It would have been quite reasonable to write a blog post about the deficiencies in clojure.set and clojure.test. Write a post about how Protocols could be better used throughout the Clojure codebase itself, and maybe even push some patches up via Jira and see what responses you get. And sure, I've had some prickly interactions with David Nolen myself; he can be that way--he also addresses bugs quickly, is constantly providing answers to questions on IRC and Slack, and is pushing ClojureScript in new and interesting directions while acting as the primary maintainer of the CLJS codebase (I believe?).
Point being, some of the issues raised in the piece may be real issues. But they do not represent some kind of overarching deficiency in the team as the author seems to suggest, nor are they necessarily relevant to the majority of developers using Clojure professionally.
If you think you know how I ought to spend my time, come over and mow my lawn while I expound on the problems of dev entitlement culture.
The thing is communicating takes time and busy people are entitled to prioritise "doing" over "talking".
As a community, Clojure is defined by the promises made and the processes established. I think "Brand Clojure" is defined by some truly wonderful attributes but it could be improved.
Perhaps this could this be fixed with process? For example
* Document architectural decisions [1] [2]
* Close tickets WONTFIX with a link to the relevant architectural decisions
Make that a process/promise so that it becomes a consistently applied approach and it would give the community confidence they are being treated well, heard and respected. "Brand Clojure" becomes more lovable.
But these processes also take time and we don't want to burden Clojure core devs. (More realistically, I doubt they'd allow us to impose on their precious time.)
So, can this happen without adding overhead for the core developers? I don't know. It's an interesting question though. Can we learn from others online communities on this topic? Are those who do this well based on dev leads who are naturally inclined to communicate? Are any using community funded positions to deliver on communication" outcomes?
Sounds like a community advocate/support role:
* Triaging tickets ensures inbound communication is effective and helpful. That helps by saving core dev effort. That helps avoid confusion on both sides.
* Generate architectural decision docs which can be linked from tickets
* Ensure WONTFIX doesn't feel like a dead end. Link to ADs, blog topical workarounds.
* Keep architectural decision docs fresh as consequences become more apparent over time.
Few ideas in there. Just a thought exercise.
--
[1] See Documenting Architecture Decisions http://thinkrelevance.com/blog/2011/11/15/documenting-archit...
[2] Is the 'garbage in/garbage out' architectural decision documented?
There's just an unstated precondition: the arguments are sets.
and these undefined-behavior bugs are hard to fix because someone somewhere already depends on that on undefined behavior, and it's just a snowball which understandably leaves library designers unresponsive
to be clear I assume he is taking about this:
``` try-spec.core=> (clojure.set/union [1,2] [1,2]) [1 2 1 2] ```
Though im not sure what "depending on their length means".
> for the above bugs there are two possible fixes: raise an IllegalArgumentException if anything other than sets are provided
Isn't this an argument for types? Wont this type of problem be prolific in any dynamic language? What is the union of anything besides sets?
> coerce lists and vectors to sets before continuing
This only works for very similar data. How do you coerce a map into a set? Is it based on keys or values? What about a string? is it based on the whole string or the characters. I loaded up python3 to see how it was handled...
``` In [7]: set(["hello"]).union(["hello"]) Out[7]: {'hello'}```
is that what you would expect? Maybe. But honestly it should have thrown an error imo. Python is just skirting the issue. A set needs a hashable type. A list isn't one because its mutable. So python knew you "meant" a set of the items in the list... It throws up its hands if you force the issue further:
``` In [8]: x = [1,2,3]
In [9]: set(x) Out[9]: {1, 2, 3}
In [10]: set([x]) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-10-0b079714f02c> in <module>() ----> 1 set([x])
TypeError: unhashable type: 'list'```
The problem is that we have handed the union function the wrong type of arguments. That it fails in further down stream is the trade off we make with dynamic languages. The community seems to admit the trade off and alleviate it with things like clojure.spec. Im not mental equipped to argue type theory here. But i suppose i feel this is an _ancient_ argument and saying its a "bug" seems simplistic.
> How you define getting the wrong type with nonsense values counts as “working” is beyond me. Is it just because it doesn’t throw an Exception? Anyone here prefer bad data instead of exceptions when dealing with functions like this? I doubt it.
This is maybe the deeper problem. You would expect union to fail given the wrong types. Sure the failure wont be a type error but it will be contained inside the function. Leaking out the wrong data means its harder to debug. I'm generally curious how and why set would work on lists at all.
As to the rest of the arguments made here. I don't know enough to hazard a guess. But i would appreciate any feedback on my interoperation of the above situation.
Python on the other hand at least attempts to adhere to a principle of least surprise and enforces strong type requirements at run-time. It does this because consistency is generally more important than performance considerations in Python, one of the reasons that Python is much slower than Clojure.
The API for the set constructor in Python accepts any iterable as an argument. Furthermore the elements of a set must be objects that are hashable.
Consequentially, Python sets rule out mutable members, such as Lists, Maps, and other Sets. Sets of Set Objects aren't allowed since Set Objects can be modified. Strings are immutable so Sets of strings are allowed and even frozensets (an immutable kind of set object) and tuples (which happen to be immutable in Python) can be members of sets in Python. I believe that this is all spelled out in the documentation pretty clearly, and the language's built ins and standard library enforce this simple model at the cost of run-time type checking.
Thus,
set([1,2,3,2]) == {1,2,3}
# the tuple (10,20) is immutable and hashable
set([1,2,"abc",(10,20)]) == {1,2,"abc",(10,20)}
# strings are iterables
set("abc") == {"a", "b", "c"}
# lists are iterable and strings immutable and hashable
set(["abc"]) == {"abc"}
# comprehensions and generators are iterable
set(i*2 for i in range(5)) == {0,2,4,6,8}
# sets are iterable
set({2,4,6}) == {2,4,6}
# int 14 is not an iterable
set(14) <--- error
# and [2,3] is a list and so mutable and not hashable
set([1,[2,3]]) <--- error
set([1,2,3]).union(["a", "bc"]) == {1,2,3,"bc","a"}
set({1,2,3}).union({3,4,5}) == {1,2,3,4,5}I think pythons resolutes only seem less surprising because someone would be used to them? Maybe that coercion is easier to reason about. Im honestly not sure. Like this example:
In [19]: s = set([1, 2, 3])
In [20]: s.union({"x", "y"})
Out[20]: {1, 2, 3, 'y', 'x'}
Why the key and the value? The meaning of the key and value have been drastically change now. Maybe this is what all languages do?
anyway, I think i gained a lot thinking this through. Thanks everyone :)
*One error Is I mistakenly said that strings weren't hashable.
{"a", "b", "c"}
{"name": "Fido", "age": 8}
The point I was making is that Python doesn't make performance trade offs with these set APIs. Set union is in no way surprising. It either gives the correct intuitive result or raises an exception because of incorrect argument types.
>>> s = set([1, 2])
>>> d = dict(x="hi")
>>> s.union(d)
set([1, 2, 'x'])
So this works:
(clojure.set/union #{1 2 3} [2 3])
But this does not.
(clojure.set/union [1 2 3] #{2 3})
In these two cases, the ordering of arguments does not matter.
So "nonsense values" returned by `intersection` means it returns a seq when you pass in a seq? So `(:key_1)` rather than `#{:key_1}`?
Umm... what? I'm afraid I don't understand how the "grand vision" behind Clojure adds pixie dust that makes it capable of something another Turing complete language isn't. Care to elaborate?
To build it outside of clojure ecosystem, you would need to first build all the pieces of the clojure ecosystem
You're basically describing, for example, a Ember/Android/iOS + Rails + Postgres app. Logic lives in the client (Ember/Android/iOS). Data + Authorization lives in Rails + Postgres. And the Rails part of that is really not doing much. It's deciding if a request is allowed, then serializing or deserializing data from the database. I guess you can do that in the dbms, but you still need to do that somewhere.
That's simply untrue, and even the author's conclusions contradict your point. You can already do all of these things with just javascript. And if you're arguing that javascript has rebuilt the entire clojure ecosystem, then what is the non-aesthetic case for clojure?
I like clojure a lot and I've made a few toy projects with it, but I see its primary value as helping me write better code in all languages I use.
Another important consideration is your familiarity with both. If you have a deep knowledge of javascript and its ecosystem, many of the architectural wins you get from clojure out of the box are fairly trivial to port over. I suspect the reverse is not as true, because a clojure expert dipping their toes into javascript doesn't have as clear a picture of what the limitations are and how to overcome them.
As we can already see in this thread, there's a tendency for people who use clojure to believe that some architectural concepts are literally impossible for all other languages, rather than merely less common or requiring some pre-configuration.
[1] http://my-codeworks.com/blog/2015/css3-proven-to-be-turing-c...
We detached this subthread from https://news.ycombinator.com/item?id=11883875 and marked it off-topic.
Also, s/he started it ;)
Thanks for the friendly responses.
Haha, what a fantastic way to word it. Duly noted.
We detached this comment from https://news.ycombinator.com/item?id=11884356 and marked it off-topic.
Opportunity here to create a company, fix the bugs, improve the broken bits, build an editor and release a better clojure and charge accordingly.
union and intersect not doing the same thing as in common lisp (accepting lists and vectors), and returning buggy values instead is just a bug, even if the developer didn't have that in mind originally, and doesn't like to support that.
Clojure is a different language.
Also, are you sure vectors are supported in CL? Looking over Common Lisp the Language, the adjoin/member/intersection functions only mention lists.
Finally, it's not even clear CL won't return buggy values, either. From the documentation on CL's intersection function: "If either list has duplicate entries, the redundant entries may or may not appear in the result." So it's just as GIGO as Clojure.
"To say that I was treated pretty shabbily by David Nolen on this issue almost goes without saying."
One of the cool things about clojure and other languages like this is that these things can often be fixed at runtime - so even if your fix doesn't get accepted you can have a "my-better-fixes" lib you pull in for yourself. A "consistent-set-opts" lib you add to your project.cls for example.
I shouldn't need to pull in more libs for baseline language expectations, either. Core libraries are what languages really are, much more important than just syntax.
His other examples sound prima facie troubling (I don't know enough clojure to have a real opinion), but he's miss-characterizing that interaction.
We eventually narrowed it down to students working on their DB implementation projects, and further to the use of mmap. If you mmap'ed a region larger than a physical file and wrote to the outside region, that NFS filesystem went belly-up. Yes, that's explicitly mentioned as a thing not to do in the mmap documentation. (On the other hand, it worked fine on a local AIX filesystem and in fact was how local file systems extended files; I'd spent the previous couple of years working with AIX filesystem developers.)
Anyway, after getting the professor to tell the students not to do that, I notified IBM tech support. The response I received was exactly, "the docs say data must meet requirement X, and that's good enough".
In a sudden fit of ethics (honestly, I have no idea what came over me), I completely failed to advertise the nifty denial-of-service attack widely. It would have been spectacularly amusing to embarrass my former employer on BUGTRAQ, say. Not to mention imagining the face of the tech support guys when they tried reading their "good enough" spiel to an IBM customer who had more than our handful of machines.
On the other hand, AIX 4 was a complete re-write and fixed the problem with the NFS filesystems. Yay.
Anyway, the bottom line is that using documentation to cover your ass for implementation infelicities is not a very good idea.