Hacker News Reader

Comment by platz | Hacker News Reader

Is it still so easy to maintain Clojure code given it's lack of types?

(I know about Typed Clojure; it doesn't seem to have won the hearts of all Clojurians)

(Or, do you just test the bejezus out of the thing?)

I add :pre and :post assertions to most of my functions, so I know the type of everything coming in, and also what is returned. An example of a function that saves a document to MongoDB:

(defn persist-this-item [item]

  {:pre [

         (map? item)

         (= (type (:item-name item)) java.lang.String)

         (= (type (:item-type item)) java.lang.String)

         (if (:created-at item)

           (= (type (:created-at item))

org.joda.time.DateTime)

           true)

         ]}

  (let [item (if (nil? (:created-at item))

               (assoc item :created-at (tyme/current-time-as-datetime))

               item)

        item (assoc item :updated-at (tyme/current-time-as-datetime))

        item (assoc item :_id (ObjectId.))]    

    (mc/insert "tma" item)))

So this function needs to be given some kind of map, and it needs to have 2 keys, :item-name and :item-type, and both of these need to be strings, and if :created-at is already set, then it needs to be a Joda DateTime.

It's not accurate to say that Clojure has no types. It just gives you some flexibility about how strict you want to be.

I find that Clojure strikes a perfect balance: its flexibility with types allows me to easily integrate a lot of 3rd party tools without much work, but when I need to I can be as strict as a I like with types.

Edit to add: wow, the code is ugly on Hacker News. Funny that this site provides no tools for posting code snippets, given the subject.

Maybe I'm misunderstanding, but it looks like every time these functions get called, they're type checking their inputs and outputs? That seems like a waste of resources. Why not just write unit tests that ensure that they will always be called with the types you expect (or fail at an earlier step if not)? Identify the places where there might be ambiguity in something's type (e.g. when a JSON blob arrives, it might map a key to a list instead of a dictionary, or if a function might return multiple types, etc), and use unit testing to ensure that the type you expect will emerge from those places, or else a failure will occur.

I believe :pre and :post conditions only get run when assertions are enabled. There are other languages that have this feature (Eiffel touts it pretty heavily), but I haven't seen the AHA! example where this is the long lost feature I've been missing.

But that said, :pre/:post w/ unit tests seems pretty powerful. You can assign the invariants to the functions themselves and have a better / more robust set of assertions in your unit tests.

If it only will get run in some sort of testing mode, that's fine I suppose. Although the other drawback to this sort of thing is that it seems to really decrease readability. Most of the time when you're reading code, you just want to see what's actually happening, and having a big messy set of type assertions would be noise 90% of the time (especially because unlike type signatures in Haskell, say, there's no syntactical difference that would allow syntax highlighters to help you visually see what's actual code and what's type assertions).

So I wouldn't confuse invariants w/ type assertions. Clojure does have separate type assert/checking in core.typed.

Yes one thing you might use :pre/:post for is type checking, but it can do more than just that.

Right, but unit tests do that too. And that would still be something I would in most cases rather handle in unit tests than in assertions in the code itself, for mostly the same reasons.

How do you write a unit test that tries to assert that no one calls the function sqrt with a negative integer? Because that is precisely the case where pre-conditions can help.

You don't write unit tests to assert that a function never gets called a certain way. You can't, in a dynamic language: a function could conceivably be called with anything, and even in a statically typed language, there are some functions which cannot be determined at compile-time never to terminate without error. (For example, if you're using signed integers, the input could be negative). The goal of unit testing in this case is to ensure assert that if that happens, your code does what you expect it to, whether that's fail with a nice error message, return some default value, or simply bomb out. Depending on what context that function occurs in, it might be guaranteed to never happen (e.g. sqrt is only ever called by function foo, and foo always calls abs on its input before calling sqrt.). In those cases, it's not necessary to write those assertions into sqrt. You should separate what's actually subject to variability at runtime vs what can be ensured by program flow, and write unit tests accordingly. There's nothing wrong with the kind of assertions/preconditions described above, but I remain skeptical that they offer anything fundamentally stronger than a comprehensive suite of unit tests.

I agree that contracts don't offer something stronger than unit test. They offer something different. Of course you can't write the unit test that I asked and that was the point. But it would be kind of pointless too to write the test that checks that an exception is thrown when sqrt is called with a negative argument. It's trivial to see that happens by looking at the code. How sqrt fails is not interesting either, as you are not supposed to recover from that. A pre-condition is a way to specify that. It says: "Don't call me with a negative argument, just don't."

Unit testing cannot prove the absence of bugs and shit happens in spite of their presence. That's why I still value (good) static type systems. But besides a good static type system that can prove certain properties about the code (and I'm not talking about Java's type system here), you can also do design by contract [1], which is precisely what @lkrubner is doing and I find that to be pretty cool.

Well, it would be cooler if the compiler could check those contracts at compile-time and issue some helpful warnings, but at runtime they are still valuable because the code will fail sooner rather than later. Plus you can probably disable them completely, should you experience problems with performance in production.

They also serve as documentation for other developers, documentation that you're forced to keep in sync. This documentation is not about the actual business logic, that ends up being laid out in tests, but rather about interface specifications and invariants.

So there you have it - testing serves a different purpose.

[1] https://en.wikipedia.org/wiki/Design_by_contract

Nothing about what I was saying precludes the use of design-by-contract; in fact, that's what I was advocating. You say that "at runtime [type assertions] are still valuable because the code will fail sooner rather than later," which is what I meant by identifying places where there is a possibility for the wrong type to be used, and putting your assertions there. Sticking type assertions on the inputs and outputs of every function you write would be unnecessary, inefficient and horrible to read:

    def increment(n):
        assert isinstance(n, int) or isinstance(n, float)
        result = n + 1
        assert isinstance(result, type(n))
        return result

So as long as we can agree on that much, then we can agree that there is a value to being judicious about where you should and shouldn't use type assertions.

By the way I'm a fan of static typing as well, although I see value in dynamic languages too. And I definitely acknowledge the limitations of unit testing, although from a practical point of view, a comprehensive set of unit and functional tests is usually robust enough. And, of course, type systems don't make any guarantees against logic errors. :)

Well, I misunderstood your point then.

On your example, you're of course right. I'm also not a fan of checking the actual type in a dynamic language, since it defeats the purpose of it being dynamic. I like assertions that are more useful than that, like:

    def sqrt(x):
        assert x >= 0, "only defined for positive numbers"

        last_guess = x / 2.0
        while True:
            guess = (last_guess + x / last_guess) / 2
            if abs(guess - last_guess) < .000001: 
                return guess
            last_guess = guess

Now clearly this helps, since it aids in readability (this function is defined for positive numbers only) and if you call it with a negative number, it will loop forever.

"you can probably disable them completely, should you experience problems with performance in production."

   Clojure has a variable called "*assert*" just for this
   purpose: set it to "false" for production to 
   turn off all these type checks.

I like Clojure, but this looks rather painful.

If you're dead set on adding type declarations, perhaps you'd be better off with Haskell or something?

I agree, this looks terrible. It seems every discussion about static vs. dynamic typing on HN ends with the following realizations, spread over multiple comments:

- its hard to safely refacture without static types (and the help of the IDE that often comes with it)

- dynamic languages must compensate the missing type checking support from the compiler with additional unit tests, negating the productivity gains

Sometimes it would be nice to have both worlds in the same language, but the way Clojure does it does't appeal to me at all.

I'm actually trying to address this in my current language experiment: https://github.com/mikera/kiss

The idea: add static types to Clojure without compromising on the dynamism / flexibility / convenience

> "Funny that this site provides no tools for posting code snippets"

Indent code by at least 4 spaces. (Just like in Markdown in e.g. Stack Overflow.)

    like
    this
      and indent some more

Frankly, Clojure doesn't need the intense levels of testing that OO languages require, due to immutability. There's a great deal less forking of possible states to examine.

I have a friend who is a strong typing devotee who has worked heavily in Clojure for a couple of years, and he curses the lack of strong typing - but he's comparing to the likes of Haskell and OCaml. As a dynamic typing enthusiast myself, I'm comparing to Ruby, and Clojure is definitely superior in terms of how much it needs to be tested.

What drew me to Clojure over Ruby, though, was actually a matter of data modeling. I'm working with a graph database (Neo4J), and I found that good graph technique is terrible OO technique. I was violating the Law of Demeter constantly, reaching through objects to get to other objects, passing internal state around with the express intent of mutation, etc. Switching to a functional paradigm lined up perfectly with good graph design. No more smells! No more pain! And again, a lot less testing, because I'm no longer fighting against the nature of my data.

I think you mean static typing, not strong typing.

Yep, my bad.

One of the really great things about having rich built-in data structures and a more-or-less purely functional data model is that it's really easy to express transformations and be confident that they work. In Java it's incredibly verbose to express & manipulate something like "a list of pairs of sets of integers" because if that was part of your core data model, you'd want to represent it in a class, and then you have all sorts of problems about what callers expect vs. what they get (e.g., do I get a reference to the original, or a copy? Can I safely mutate this?).

But if you actually have facilities to pass around and manipulate those kinds of data structures in a way with no side effects, it's trivial to verify (first on the REPL, then in your unit tests) that whatever transformation you're writing applies properly down the entire nested chain, and it will always work for anything of that pattern. And it is much quicker to write those kinds of transformations iteratively in a dynamic language than trying to represent it on a type level a la Scala.

That's a good point. A lot of what I built classes and objects for in other languages can really be expressed as simple maps, hashes, lists, and vectors, and more complex types built from them. Clojure's elevation of maps/hashes/vectors to first-class types puts it above Lisp, imho, and its functions to manipulate said data structures are better even than languages like Ruby and Python.

I can't imagine having to go back to Java collection classes again. Ugh.

You are saying hashes, vectors, etc. are not first-class types in Lisp?

Strange...

but just in case someone considering clojure is worried about missing something like haskell's typeclasses (or whatever they're called in scala), it's worth mentioning that clojure does have some notions of "protocols" and "datatypes" that seem pretty similar.

They're not really comparable; Clojure protocols are a more flexible (runtime-extensible, for instance) notion of a (non-hierarchical) interface, and records / deftypes are just straight-up Java classes, with records having in addition nice map-style access to members.

I haven't used Typed Clojure, so I can't speak to that, but if you want to do type-level programming Clojure is probably not your weapon of choice.

sorry, not well-phrased: i meant to say that, taken together, clojure's datatypes and protocols look somewhat similar to haskell's typeclasses, not that datatypes and protocols are similar.

Many times the clojure code is so much simpler, a) even basic testing will catch most of the bugs and b) there are less bugs since the code is so dense. In many ways it's just simpler to understand what 5 lines of Clojure code are doing, than 50 lines of C# code.

On top of all that, sane defaults like immutable data, lack of mutable locals, and sane concurrency primitives means that a whole class of bugs just don't exist in normal Clojure code. All this sums up to a codebase that really doesn't need static typing.

I write Clojure code for a living, and have worked on some really large Clojure codebases, and yet I've never felt the need for a type system. It's a fallacy that large systems have to be so complex that only a type system will save you. Often the better route is to simplify and modularize the system.

Tests, core.typed, prismatic/schema, and/or pre- and post-conditions.

It's still saner than, say, Ruby.

FWIW: "What does this take and what does it return?" is rarely a question that comes up in Ruby for me. In most codebases, thats a pretty easy question.

For others, Clojure wouldn't help at knowing whats going on ;).

That was what drove me away from Clojure in the end. It's fun to write, but a challenge to write in such a way that you can read it later without jumping all over the place.

How big/complex were your functions? Have you seen the style guide? It's easy to write really powerful, nested functions in Clojure, which are damn hard to read the next morning. Taking advantage of thread-first and thread-last macros can really help here. I've finally started to use them heavily. For instance, you could do the following to read a file, parse it from json, and retrieve somekey:

  (:somekey (cheshire/parse-string 
              (slurp "/tmp/somefile.json") true)

  (-> "/tmp/somefile.json" 
      (slurp)
      (cheshire/parse-string true)
      (get :somekey))

In other words, there's much nicer ways to write maintainable, readable code that you can still understand in the morning.

Your "jumping around" comment also makes me wonder how you were laying out your functions? My advice is to go through the libraries of the popular Clojure libraries, you will discover a lot of good practices. It's taken me a few months of Clojure to start writing cleaner, readable code.

I had the exact opposite experience. I found that I was laser focused on a particular chunk of code when working in Clojure. Functions were logically grouped together. I never found myself jumping around.

I use prismatic/schema which can be used to define schemas and either validate on demand, or always-validate function inputs. Very useful and easy to add.

Reply on news.ycombinator.com