Hacker News Reader

>Clojure is a language for people who know what they are doing by people who know what they are doing.

That can also be said for C, assembly and almost any language, even Brainfuck.

And when there's not a speed/memory tradeoff for not having the language do the right thing (like e.g. is the case for some decisions in C), this just amounts to having the programmer do "busywork" -- manually do the work that the compiler/interpreter should have been doing instead.

Blaming the programmer for such errors is thus a kind of "blaming the victim".

The approach "The language X is totally fine, it's the programmer's problem who doesn't know what they're doing" for such cases, that is, cases where:

1) the right thing/warning could be inferred automatically

2) there's no (or not significant) speed/memory tradeoff to do so

is a bad idea and those that propagate it should feel bad.

C absolutely is a language by/for people who know what they are doing. The fact that so many people are using it for so many things where modern safer languages now exist is not a failing of C or its designers...

Regarding misusing the set functions: There is no busy work. Instead of (set/intersect some-set some-list) you just need to do (set/intersect some-set (set some-list)) to explicitly opt in to the speed tradeoff of performing set operations on lists.

> is a bad idea and those that propagate it should feel bad.

I don't feel bad about it at all.

If Clojure hadn't followed the garbage-in/garbage-out strategy, it would have been too slow for production use. If they had added strong types early or Racket-style (read: slow) contracts, then we'd never have had the years of experience that led to the design of spec, which is new take on the problem space.

In the case of the set functions, the right thing _can not_ be inferred automatically without a speed tradeoff.

> Set union can't be done efficiently on arbitrarily sized sequences. At least one of the two arguments needs to offer fast set membership; ideally the larger one, hence the requirement that all arguments be actual sets.

This made me very curious about efficient set union. Here's a pretty straightforward set union in python:

    def set_union(coll1, coll2):
        result = set()
        for item in coll1:
            result.add(item)
        for item in coll2:
            result.add(item)
        return result

Apparently that isn't the efficient way to do it -- what is?

Regardless, the article strongly suggests that one of the two arguments to clojure.set's union does offer fast set membership, since only the second argument is permitted to not be a set.

    cljs.user=> (source clojure.set/union)
    (defn union
      "Return a set that is the union of the input sets"
      ([] #{})
      ([s1] s1)
      ([s1 s2]
         (if (< (count s1) (count s2))
           (reduce conj s2 s1)
           (reduce conj s1 s2)))
      ([s1 s2 & sets]
         (let [bubbled-sets (bubble-max-key count (conj sets s2 s1))]
           (reduce into (first bubbled-sets) (rest bubbled-sets)))))
    nil
    cljs.user=> (source clojure.set/intersection)
    (defn intersection
      "Return a set that is the intersection of the input sets"
      ([s1] s1)
      ([s1 s2]
         (if (< (count s2) (count s1))
           (recur s2 s1)
           (reduce (fn [result item]
                       (if (contains? s2 item)
             result
                         (disj result item)))
             s1 s1)))
      ([s1 s2 & sets] 
         (let [bubbled-sets (bubble-max-key #(- (count %)) (conj sets s2 s1))]
           (reduce intersection (first bubbled-sets) (rest bubbled-sets)))))
    nil

I don't feel that I know any more now than I did before you posted this "reply".

The third branch ([s1 s2]) adds the elements of the smaller collection one by one onto the larger collection. Conceptually, it does this by constructing a new copy of the result for every addition, since conj is not destructive. It is buggy and will return a result of whatever type the larger collection is, with data being duplicated if that type allows for it.

My question was, why does efficient set union of two collections require one of the two collections to support fast membership tests? As an answer to that question, you've posted code that doesn't involve membership tests at all and supplied no discussion of why it is or isn't efficient.

I wanted to give you an answer and started following down the call trail:

    https://github.com/clojure/clojure/blob/master/src/clj/clojure/core.clj#L75
    https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/RT.java#L661
    https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/PersistentTreeSet.java#L51
    https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/RT.java#L667

Then I gave up. I am curious too. Where I left off, we had a Cons cell. And I am hoping it will get converted to a set later (via empty).

> why does efficient set union of two collections require one of the two collections to support fast membership tests

Sorry, I misspoke / used bad shorthand. Union should have efficient set addition for the larger argument and set _intersection_ requires efficient membership tests.

> Conceptually, it does this by constructing a new copy of the result for every addition, since conj is not destructive

That's not how persistent data structures work

http://hypirion.com/musings/understanding-persistent-vector-...

> why does efficient set union of two collections require one of the two collections to support fast membership tests

I haven't thought much about this but it seems intuitively true to me. You want to take the smaller set, and add each item to the larger set if it doesn't already exist. That last part is why you need a fast membership test.

If you're still unconvinced, try loading up two similarly structured database tables with a million rows and get the union of them (with and without indices). The indexed one will be orders of magnitude quicker.

> they work crazy hard to build and shepherd something awesome for free.

I'm not so sure about that. I'd say that Clojure now exists and thrives to help Cognitect make money. This is no way a bad thing, only a reminder that there is no such thing as a free lunch.

In a similar vein, Go exists to help make Google money. The original purpose of open sourcing it was different for Google (make the language more robust/better) than it was for Hickey (grow the language/community).

I anticipated this response to the word "free", but I stand by my statement.

Clojure is _free_. If you don't want Cognitect stuff, don't use it. If you don't like how Rich/Cognitect et al are running the project, take your EPL licensed ball and go home. Similarly, in the case of Go: Substitute Rob/Google and BSD-ish license.

Just because these languages serve the interests of their respective maintainers does not make them any less free to you. It may make the community less useful to _you_, but hey, they created these things to be useful to them. There's an implicitly mutually beneficial social contract around contributions and community. If you don't like that contract, don't use the language.

In the very unlikely case that Cognitect or Google puts some company-specific stuff in the language in a way that actively destroys the good will of this social contract, then and only then do you have any right to complain.

> does not make them any less free to you

Of course not, but it does make them unencumbered. And I'm not complaining. I'm just saying that's cost (i.e. not free) of doing business with a sponsored open source language.

If you take the stance of "nothing is free", then the word "free" isn't particularly useful.

A+ extreme reductionism.

I never said "nothing is free", nor implied that, but merely that users need to cognizant of the goals of "free" language maintainers (especially when there is a corporation rather than a person as BDFL) that do not necessarily align with the goals of the median or average user.

Now that we are on this topic, I'm starting to think maybe it's worth paying for a programming language so that the goals of the maintainers are better aligned with the goals of the users. That being said, there is real value to the ability inspect the source. As such, it's very difficult to do open source as a paid product.

There is a history here. Please read:

1. https://web.archive.org/web/20100220100027/http://clojure.or...

2. https://web.archive.org/web/20140704050611/http://clojure.or...

My own interpretation: programmer culture has such problems that someone actually considered corporate funding healthier than user donations.

There are many definitions of the term 'healthier'. But the links are pretty amusing. And now we have a first fork https://github.com/jaunt-lang/jaunt

There is a distinction between "Clojure is free" (talking about the burden on the consumer) and "Hickey et al work on Clojure for free" (talking about the rewards they receive for their effort). The latter is less clearly the case.

> The original purpose of open sourcing Go was different for Google (make the language more robust/better)

I'm not so sure about that. I like the 'baseball cap' argument more. The quote is about React, but it may be relevant for Go as well

> Facebook doesn't care if you use the Web, it only cares that you use Facebook.

> So why release an open source Web framework at all? Because Facebook is battling Google for engineers. So you've got a big fight between two companies over which company is the coolest place to work, and both of them are companies that your grandparents love. How are you going to win this fight? One way is to have the hippest Web framework.

> Basically, both Google and Facebook are desperate to find a baseball cap that they can put on backwards. Angular is Google's baseball cap. React is Facebook's.

https://www.pandastrike.com/posts/20150311-react-bad-idea

So you're saying because I've run into these bugs in my own code, I clearly don't know what I'm doing and I should be using a language for someone of my intelligence?

No, I'm saying you should stop and think about what you're doing.

Cool. So you're saying because I found bugs I'm not doing my job right? That I'm rushing and not thinking about what I'm doing?

You're ignoring the fact that I don't believe clojure.set to be buggy.

You're not getting what I'm trying to communicate.

Telling people that they're not doing a good job when you don't even know them only pisses people off, and eliminates any chance that they'll agree with you on principle.

I was hoping highlighting that would make you realize that all you were saying to me was that I'm not doing a good job, but you didn't seem to get it. Each time I summarized what you said before, you changed your story to make it sound less bad.

Ultimately I don't really care that much, this is the Internet and I don't have to interact with you after this. But I'd recommend not doing that to people IRL, they'll resent you.

> Telling people that they're not doing a good job when you don't even know them only pisses people off, and eliminates any chance that they'll agree with you on principle.

Funny, that's exactly what the original poster did to the Clojure core team.

> Each time I summarized

You call it summarizing, I call it mischaracterizing. I stand by the position that if the original poster thinks that clojure.set is bugged: they don't know what they are talking about. That doesn't mean they suck at their job, that means they haven't actually stopped to think about the particularities of those functions.

I always found such way of thinking very offensive to your users (programmers in this case).

Have we all forgotten the principle "make the minimum amount of WTFs per minute" already?

Why is everybody so content with their technology and so quick to look down upon people who aren't familiar with the traps in it? Is it really so wrong for your technology to be easy and painless not only at the entry level, but all the way to middle-high level of professionalism?

> Is it really so wrong ...

No, that would be _wonderful_. Unfortunately, "easy" is only one dimension along which we must make tradeoffs. Clojure has chosen one point in the design space for input validation vs performance. You're free to have a different preference.

Thank you. The only problem here would probaby only be the lack of clear and strong statements about the Clojure philosophy. The C standard for example clearly shows the areas where there is an undefined behaviour and when you can expect the "garbage in, garbage out" behaviour. Is Clojure doing the same?

As a relatively mature man and a programmer for me the way a project is maintained is just as important as the ability of the technology itself.

This post helped me make a conscious decision to not work with Clojure. If I am gonna learn a new language and technologies I prefer the least possible amount of surprises, because when I inevitably stumble upon them I'll be shamed as "not active enough to research beforehand".

> The C standard for example clearly shows the areas where there is an undefined behaviour and when you can expect the "garbage in, garbage out" behaviour. Is Clojure doing the same?

Yes. In fact, this is partly what clojure.spec is about; which the original author calls out as being wrong to prioritize for some unknown reason.

You'll only be shamed (by me) if you go write a long blog post complaining about it. I don't represent the entire community. Otherwise, you'll be politely corrected and have the tradeoffs explained to you in IRC or Slack. My apologies if I've given you any other impression.

Thank you again. No, you didn't give me that impression. The author gave me the impression that this is the predominant answer he was given -- or he assumed so.

I have zero idea if that's the case, though. I'd be glad to work with a technology if I had a polite and constructive person such as yourself to help me out when I am really stuck.

i have recently been excited about clojure, but your behavior and responses of other clojure people in this topic has honestly put me off of it. i understand that things are difficult and cljure people apparently work very hard, but the tone and attitude being used is in stark contrast to what i have seen from communities like those of F# and racket. whether you're right or not, it doesn't seem to be a great approach. for people like me, this is a first taste of clojure.

just a heads up.

My apologies. The community is actually quite welcoming to beginners.

For context, these arguments made by the original poster have been around for _years_ in the Clojure community and, frankly, those of us who have also been around for years are tired of hearing the same old complaints. It's particularly annoying because: the core team has made multiple announcements in past days/weeks about improvements to exactly what he's complaining about!

not trying to be dramatic. haha. i am mainly just excited about clojure but this has been a bit of a contentious debate which gives one pause when first getting their feet wet in a new ecosystem.

> Clojure is a language for people who know what they are doing by people who know what they are doing.

Oh, goodie. I'm part of an elite club with high barriers to entry.

Experience actually matters in software development. I'm not sure why you deride it as elitism.

Tone matters too.

> the author goes and poo poos [...] spec, despite the fact that would "fix the bug" exactly as requested by throwing when you gave non-sets to the set functions.

I noticed that as well.

As someone who is unfamiliar with clojure -- is there any particular reason a function which doesn't actually work with sets allows non-sets to be passed to it in the first place? Or this a deficiency in the type-checking? I mean, truly, this kind of thing seems like a solved problem...

This is a tradeoff made with dynamically-typed languages. Open up your dev console and try some JS for other examples: `[] + []` or `[] + {}` or `{} + []` or `{} + {}`.

There are several ways to solve it all with tradeoffs. Types can solve it with tradeoffs. Checking the types in the function can solve it with tradeoffs. Stating in the docs what arguments may be passed is another way to solve it.