Hacker News Reader

> I think Rich is an extremely smart and very accomplished programmer. I think functional programming is really good at some things, I don't feel like many people talk about the things it's not good at.

As someone who used to swear by Ruby but now prefers FP langs, part of learning FP is as much about learning what FP is good at as it is about learning what FP isn't good at. Given FP's proclamation for declarative style and immutability, clearly FP isn't good at domains where imperative styles are important or really, really, really[0] CPU intensive work where mutability is needed.

Although, thanks to compiler research (which is mostly done by FP language users, ahem) languages like OCaml are bridging this gap when you take a look at projects like MirageOS. You get to write your code in a declarative, immutable style that then gets compiled to very fast native code. It's having your cake and eating it, too.

As far as your original complaint of everything in Ruby either being a hash of hashes or a class that just wraps a hash of hashes, I completely agree. That is what drove me away from the language. I'd rather just blow that hash + class relationship up completely. Not to mention, in ML-dialect languages you can use features like Enum/Sum/Product types to define the configuration/syntax of your programs which is a lot better than reading arbitrary keys in a hash, as you have stated.

0 - No, I mean really, really, really, really. "I think I need C to do this fast enough for my use case" is the "I need Cassandra for my 2GB database of 'Big Data'" of the FP world.

>I think clojure helps with the speed of allocating new immutable structures by using copy on write strategies behind the scenes.

No, it's not copy on write in the sense that COW means in other languages. Data in clojure is persistent, so usually even an altered piece of data is not actually copied, only the tiny bit that changed is added, if necessary. This is very different than, for example, Swift that implements copy on write, but the whole data structure is copied even if the change is minor.

Additionally, you say:

>When we're dealing with sequential tasks, mutating data is much faster than allocating new data structures

That would be true in cases where the entire data piece is new; but so much of what you do in any language involves interative changes over existing data, and again because of clojure's persistence, new allocations are often not happening at all in many cases.

...However, I do agree with most of your other points :)

I was introduced to http://hypirion.com/musings/understanding-persistent-vector-... not too long ago on HN. I don't use clojure so I was paraphrasing my somewhat limited understanding. Thanks for the clarification.

That part about speed was more about my general frustration with the "immutable is fast" and "mutable is slow" meme that I hear too frequently. While that can be the case, it isn't always 100% true. Your language and how you use it can play a huge part.

I'm hoping to learn more about clojure data structures in the coming months, so much good stuff.

My guess would be that immutable clojure is still faster than say mutable ruby though.

I don't think anybody is arguing that trie lookup is faster than array lookup. But when it comes to passing data between parts of the system, zero cost copies are much faster than copying buffers or objects.

> Unfortunately this initial simplicity introduces unexpected complexity, things like deep dup and deep merge are now needed.

Not if your data structures are immutable, which is exactly what he is advocating.

Exactly; the whole idea of "deep dup" doesn't even exist in Clojure. It's a non-issue. Some languages go so far as to encourage actually serializing and then reading back in a data structure as a form of doing a deep duplication! (I've done this in Objective-C, based on advice in popular books.) This is an extreme example of the pitfalls of mutable data in many circumstances.

Excellent point, I honestly hadn't considered that. I've spent more time dealing with functional programming than in actual languages with immutable structures. Thanks for pointing that out.

I think my other point still stands, things like optional keys, or missing or required keys still must be shared throughout the app. Everything that touches that hash needs to know its structure.

Yeah, but that structure is also easy to explore and understand.

And in case you want some security you can use libraries like prismatic/schema, that allow you to declaratively describe your expected data-structures akin to a type system.

The web is build on json and not corba because plain maps and vectors are far easier to work with than domain specific objects.

There is a discussion of deep vs shallow hashes. Take Datomic or Datascript for instance. Very shallow datastructures, with a defined schema. Indexes to make this fast.

There’s tradeoffs. Shallow datastructures are under-used.

But OOP objects share fundamental flaws with deep hashes. (And add some more.)

Having to know the structure of a hash/map/dict is strictly superior to having to know the methods and fields of an object. For any given task the two are equivalent, except that one is easily modified/reused/passed to other callers without creating an extremely brittle class hierarchy you're only going to have to rearchitect when your requirements change. As an IBM researcher looking at Watson waaaay before Watson actually existed: "OOP requires you to treat all extrinsic information as if it were intrinsic to the object itself." This is tantamount to saying you must have a complete and consistent programme design before you write a single line. Which is crazy (and probably the reason UML exists.) But don't take my word for it, think it through, it's the only conclusion you can reach, I think.

There is actually a lot of complexity to hashes in Ruby.

The two major sources of complexity are the fact they are mutable and the fact that keys are mutable but hashes are computed when the object is first added.

So if you construct a valid hash, then call three functions passing them the hash, you have no idea if the hash is still valid for the second and third functions without reading the code for all the preceding functions.

What this tends to mean in practice is you are always having to check all over the place that your hash is valid.

They might look the same as clojure hashes but in practice they are used very differently.

> There is actually a lot of complexity to hashes in Ruby.

I agree here

> So if you construct a valid hash, then call three functions passing them the hash, you have no idea if the hash is still valid for the second and third functions without reading the code for all the preceding functions.

I don't find that to be true, even in the Rails codebase. You pass a hash to a function (method) and expect that it won't be mutated. If you're writing a method and you mutate a hash, you're expected to dup the argument so it won't be mutated. This is the convention. There are times when hashes are mutated, but generally that's reserved for methods who's purpose are to mutate its arguments.

Where Rails gets into trouble in its functional passing of hashes isn't in the mutability of the hashes, it's in the composability of the functions. I've never written about this problem before, so i'm not sure I have a great example but it's definitely a problem.

One example in rails is `url_for` it takes a hash argument. The problem is that there's multiple `url_for` methods that all do slightly different things. Some are needed for generating links in email, while some work in your views, and others are designed for you to use programmatically outside of a view context. One of the hardest things about this method is that one `url_for` can call another `url_for`. Since we can never be guaranteed the order of the calls it is really it makes things like having default values, or optional keys. You have to replicate logic in different functions since some may never be called in the order you might expect. This significantly impacts our ability to refactor which in turn impacts our ability to make performance improvements.

I recently did a bunch of perf work in https://engineering.heroku.com/blogs/2015-08-06-patching-rai... and some of my biggest perf improvements were getting rid of duping and merging of hashes. If Ruby had an immutable and performance efficient hash then it would have helped a bunch, however I don't think it would make the general awfulness that is an entirely hash based API to a very complex action (such as url_for) that significantly better to work with.

Take a look at ring for clojure, which is a web server interface very similar to ruby rack.

Because it passes every request as a map you can hook functions (middleware) in between that change the behaviour of the request handling, for example they could add a field for passed params or fields for user authentication.

This is possible because maps are easily extendable and functions down the line don't need to know about additional keys. You can't do that with OO properly.

You are sort of arguing a strawman. First you say that hashes are simple, but then go on to argue how they are not. Maybe they were not simple in the first place? Simplicity does not mean easy. It was "easy" to use a hash. Simplicity can actually be incredibly hard to design properly. I think this is what Rich alludes to in the video.

Edit: It's one of the first thing he says in the video actually - "It's about the interleaving, not the cardinality."

Actually, parent gets it and is neither arguing a strawman nor mistakenly labelling hashes as simple rather than easy.

Rich absolutely argues that basic data structures like hashes are simple -- something that is generally agreed upon. They are simpler than 'objects' because you can use basic comparison operators on them, and you can operate on them with higher-order functions etc.

The question which parent is digging at is whether a larger, complex app, that leans heavily on hashes actually results in an app that is on the whole simpler, i.e. does 'a simple thing plus a simple thing equal a simple thing'. This is something I've heard discussed well on the Ruby Rogues podcast -- I think that either David Brady or Josh Susser may have a good blog post on the subject of 'simple + simple != simple' but I'm struggling to track it down.

Whilst I love Rich Hickey's talks I do find myself coming round to the same conclusion as parent -- if you forgive my possibly incorrect interpretation of their argument -- that the idea that simple data structures are simpler is fairly useless if programmers use that fact naively.

PS. Everyone should watch the linked talk, it's brilliant, and one of my favourite programming talks ever. I recommend it to every programmer I meet.

I think you nailed my original intent quite well, thanks for the post.

Why ignore the message of simplicity in abstractions and design only to nitpick on hashes? I am not a fan of using hashes either. I also disagree with Rich's stance on static typing. But still think the video has a great message that is being lost here.

> Every part of your system must now know the hash structure you're using and often reproduce logic in many places like dealing with optional keys that may be missing.

Why not just write accessors and mutators for your hashes? Also, Python hash a function dict.get(key, default). Does Ruby not have this?

> I think functional programming is really good at some things, I don't feel like many people talk about the things it's not good at.

This is a wonderful tenet in general: let's think about what X is bad at, rather than what it's good for.

While I'm not much of a Rubyist (I've hacked a little in my time, but I fall on the Python side of the divide), working with trees (hashes of hashes are a kind of tree) has never been easier than in Clojure in my experience. I suspect this is true of any lisp.

Could you expand on how Clojure makes it easier to deal with the frustrations your parent comment mentions (deep merges, shared reliance on hash structure, reproduced logic, and handling of optional values)? I've only used Clojure a bit, but I'm not sure how it solves any of these problems (which I also find frustrating in Ruby).

Immutability is a big part of it, as is the wide array of operators for maps and map-like data structures.

Hickey has stated elsewhere that optional values don't really make sense in a dynamically-typed language.

Persistent data structures are part of it, for sure. But on a higher level. On a lower level, if you look at how and assoc-in works in Clojure, the scales will fall from your eyes (they certainly fell from mine). Consider:

(def g {:x "a" :y {:k 71}})

user=> (assoc-in g [:y :k] 72)

{:y {:k 72}, :x "a"}

With this you can walk and mod deeply nested datastructures with ease (and others can reason about it well).

I may be misunderstanding your point, but that's just a well-done function which could be written in virtually any language, the advantage isn't inherent to Clojure.

The difference is that in Clojure, `assoc-in` is effectively a deep copy as well since the data structures are immutable. Most of the complexity involved with nested maps is about the effects of modifying them. In practice, this doesn't exist in Clojure.

> which could be written in virtually any language

What natrius says is completely right on this point (which explains why those functions don't exist commonly outside of lisp) but it should by now make you wonder: if that's true, why haven't they?

OH, I see your point. Thanks for the enlightenment.

It's not true. ImmutableJS is becoming very common, especially for React users: https://github.com/facebook/immutable-js/

People like to talk about immutable.js. It's not common.

8k stars and counting. We used it at the last place I worked (a YC startup) and several of my friends report the same.

Your comment seems more about Clojure's type system and fundamental data structures than about functional programming in general. e.g. Haskell solves some/most of the problems you mention, while introducing different ones and presenting other trade-offs to consider.

Using data structures without a schema is user error, and isn't the fault of any particular language.

The rest of your points have some merit. :)