Rich Hickey: "Simple Made Easy" from Strange Loop 2011 [video]
infoq.com
infoq.com
BTW for those of you who haven't watched it, this talk is not Clojure specific.
I'd be interested in seeing someone present a different and convincing set of concepts. At this point, I think Rich has put together a very good toolset.
Non-Clojure examples would help make the point. He does bring up examples from Haskell (such as type classes) in the talk in places, but doesn't dive deeply into them.
I would love to see the programs that are generated from this philosophy.
I find it a little funny sometimes to hear all these "web scale" folks put down SQL, and then praise something like MongoDB because you can do map/reduce.
I saw a presentation by the author of the Lift framework in Scala and he made a bit of a joke saying "gosh...all this FP stuff...the folks who created SQL heard about that a long time ago".
the only person out there who thinks
SQL has a certain beauty and elegance
On the subject of SQL and Clojure, ClojureQL provides an alternate relational data manipulation language that compiles to SQL. It's not just a different syntax; it allows some composability not found in SQL. The syntax does help though, especially in conjunction with the thrush operator.
If you liked this talk, then you will definitely like "Are We There Yet?" In it, Hickey argues that most popular object-oriented languages make similar mistakes by bundling identity, time and state into objects. He discusses how we might simplify programming by separating these components and rethinking each.
It has a similar theme but focuses on one concrete issue in depth. It has a similar philosophical style while remaining clear-headed and practical. And, in my opinion, it is similarly enlightening. If you couldn't tell by now, I recommend it :)
How many programmers do you know that are learning all kinds of languages and technologies and methodologies and other things to improve the quality of the software they write and yet will probably sit at a desk writing code for the next 30 years? As opposed to starting a business, getting financial free, etc.
Take the guy from Duck Duck Go. He wrote all of that in Perl; talk about easy, but not always so simple (to maintain). What if he spent his time learning Lisp and Monads instead of writing an app that lots of people use?
I'm just thinking my time might better be spent thinking about business ideas, and then trying those ideas out on Heroku. As opposed to learning about some technology that may make me a better programmer but may not really help me get to where I want to be, which is having a successful product that I own equity in.
Is it really worth my time to learn the intricacies of functional programming? Am I still going to be writing code for some corporate overlord 5 or 10 years from now?
Am I still going to be writing code for some
corporate overlord 5 or 10 years from now?
See the slide at 17:20 -- easy stuff (perl,ruby, blurb) gives you 100% speed at the beginning of a project. If you don't expend effort into making things simple, you will "invariably slow down over the long haul."
Then you should think that there may be others who aren't really that concerned about being their own boss and financial freedom(strange as it might sound); others who love learning new perspectives. Other than saying that clojure may not be worth your time, you offer no insight. Thanks for making your preference clear, now can we get back to discussing the talk?
I don't know if it's a binary decision between investing a lot of of time, or investing none. You could skim the reference manual, and if anything jumps out at you, you have a topic for future investigation, or a non-zero quantity of information to base your decision to "is it worth more time?" on.
If you mean on a very abstract level, the balance between learning what others are doing and producing your own stuff, you might enjoy Richard's Hamming essay "You and your Research", pg hosts a copy on his site: http://www.paulgraham.com/hamming.html
I can actually just crank out code that I use. It can be quick and messy. Clojure is not concerned with type theory and provable correctness.
But the end result of a quick hack, in Clojure, is often something (or is made of of parts) that can be applied to other problems by pulling it out into a library, or by abstracting one more argument to a function, etc..
I think someone like Gabriel Weinberg could get along just fine in Clojure, with much the same spirit as hacking Perl, but maybe with better results.
In Clojure, it seems that I can't apply the method of spike solution (http://c2.com/cgi/wiki?SpikeSolution) and then refactor it into working code like I do with languages like Python or C.
What does this imply about the languages you already know?
Don't we all want to "get software just right"?
"Is it worth my time to learn Clojure or Haskell, when I don't even know what I'll use it for?"
Reasonable question. Pick a project, choose a language. I introduced Scala where I work and it solved a problem but wasn't optimal for our team. Then I introduced Clojure and that's working better for us. Real world problem solutions will help you validate your choices (there was a great talk at The Strange Loop on real world Haskell usage at a startup, BTW).
"What if he spent his time learning Lisp and Monads instead of writing an app that lots of people use?"
Like Paul Graham? (Viaweb)
So sticking with this analogy, we should only need to use testing in the more intricate / complex parts of our code. However, current testing best practice seems to be to test everything possible, thus potentially wasting a lot of time and effort into aspects with a low ROI.
There could be some lesson in this...
When I first started testing it seemed like the world suddenly got really scary and now I had to test everything. I ended up testing ridiculous stuff, things that the language would provide by default. (I did this in many languages which is why I don't mention a specific one).
What I've found valuable as I do testing (I do TDD) is that it has made me change how I think about design and composability.
I agree that there should be a greater focus on "what is appropriate to test" but even knowing how to write tests and what to test is a skill in itself.
However, I think regression tests are useful. Once you find a bug and fix it, the things learned from fixing the bug can be expressed in a test, to prevent similar bugs from happening again. In such a case, the test documents an unexpected corner case that the developer was unable to predict.
http://www.eurail.com/planning/trains-and-ferries/high-speed...
These too run on rails.
Rather, tests (and type safety) are "guardrails" that warn when you when you are doing something wrong (they are reactive). As Rich said on Twitter (https://twitter.com/#!/richhickey/status/116490495500357633), "I have nothing against guard rails, but let's recognize their limited utility in trip planning and navigation."
I believe that linking back to the greater context, Rich is saying that simplicity and doing more to think about your problem at the beginning (proactive steps) provide value in building systems that are high quality and easy to maintain. I think he is at least implicitly challenging whether the value you get from simplicity is greater than the value you get from an extensive test suite.
I do not hear his comments as anti-testing, but rather more as pro-thinking and pro-simplicity. Personally, I find tests useful. I write them when writing Clojure code. However, I also get tremendous value from focusing on abstraction and composability and simplicity in design.
The section about testing and guardrails seems to have been blown way out of proportion. I fervently believe in Agile/XP practices, TDD and all such good things. But I'm not naive enough to say that "because I have tests, nothing can go wrong". And that seems to be his main point here.
It makes me think...it seems like all languages and methodologies have a "Way" of the language (call it the Tao of the language). The closer you get to "The Right Way of Doing Things" within a language, the more you reach the same endpoint. And I feel that's what Rich is talking about here.
What I like about this talk is that it could be useful for programmers of any caliber or toolset to hear. If I could have heard some of these principles when I was first learning BASIC, it would have been useful.
I guess it's just that isn't it. There's a lot of talk here about what Rich might have/probably implied. I suppose it would have been infinitely more helpful if he would have just been explicit about it as opposed to projecting a slightly philosophical [sic] point of view.
If you look at around 31:27 in the talk, you will see ten complex things, with proposed simple alternatives. If testing helped people feel the pain of complexity and discover simple things, we would see projects where people had problems writing tests, and switched strategies as a result.
Do you see people moving from the complexity column to the simplicity column in any of these ten areas as a result of writing good tests? I don't. What I see is people cheerfully enhancing testing tools to wrestle with the complexity. Consider the cottage industry around testing tools: fixtures, factories, testing lifecycle methods, mocks, stubs, matchers, code generators, monkey patching, special test environments, natural language DSLs, etc. These testing tools are valuable, but they would be a lot more valuable if they were being applied against the essential complexity of problems, rather than the incidental complexity of familiar tools.
STOP STOP STOP! MAKE IT STOP!
This is the clearest indication of how (dogmatic) testing has become a vehicle that introduces complexity, rather than something that alleviates complexity.
But yeah, on Java and C# land, the complexities of the type system and the class based OO complect the testing, yielding this big complex testing system (which are large enough to be called testing frameworks).
(I don't disagree with your main point, but I don't quite see where those techniques fit in).
* code generators: Visual Studio, Eclipse
* monkey patching: RSpec
* natural language DSL: Cucumber
I can see how monkey patching could be useful in mocking or something similar. I've never really used a language that supports it though.
I'm not entirely sure what Eclipse's code generation has to do with testing, but given the other examples I'll assume I'm being stupid again ;) I'm actually working with a lot of EMF generation stuff at the minute which can be quite painful.
And yes, I have seen developers make large changes in their code towards simplicity because it was hard to test.
If someone is going to write complex code, they are going to do it with or without tests. If someone is going to write simple code, tests are a wonderful tool to have in that endeavour.
Do yourself a favor and take the shortcut of listening to this talk..not to say he may not join a cult religion at some future point in time and come out with crazy crackpot ideas then but everything I've seen and read so far are things that all senior+ quality engineers should find some common agreement with.
http://clojure-libraries.appspot.com/category/137002
my 2 cents
I use lisp -- and half my code is tests.
Stuart Halloway: "Simplicity Ain't Easy"
http://blip.tv/clojure/stuart-halloway-simplicity-ain-t-easy...
Take your language of choice. If you're not in a Lisp already, you can't do much about syntax vs. data, but what about the rest? Is it easy to work generically with data and values instead of making custom classes and mutable objects for every piece of data? Is it possible? Can we make it better? Are there persistent collections or managed refs available? Can we write them as libraries? Within the language's polymorphism construct, can we extend a type we don't control to our abstractions without creating adapters and more complexity around value and identity? What about transactions?
ImageMagick and swftools required.
#!/bin/bash
for s in {1..39}; do wget http://www.infoq.com/resource/presentations/Simple-Made-Easy/en/slides/$s.swf; done
for swf in *.swf; do swfrender $swf -o $swf.png && rm $swf; done
convert `ls *.png -x1 | sort -n | xargs echo` slides.pdf
rm -f *.swf.png
Great talk by Rich Hickey, and thanks for the script.
Simplicity is, of course, key; but a few of his applications of these principles are misguided IMO.
Ex: The "Parens are hard!!" slide. He suggests that parens are "overloaded" in CL/Scheme because they are used to wrap functions, data, and structures all the same. However he completely misses the fact that by representing everything with parens, CL/Scheme remove a lot of complexity in software written in those languages.
AFAIK, the only languages that do macros right are homoiconic. Anything else is too complicated. Just look at Perl 6 and Haskell macros. They require learning special syntax and really crazy semantics. Using them will probably just make your program more difficult to understand.
He also "rails" against testing. He misses the virtues of a proper testing infrastructure in producing reliable software: if you don't test it, how do you know if it works? Because you reasoned about it? How do you know you're reasoning is correct?
True, "guard rail" testing isn't a crutch that will automatically produce reliable software. But I think Rich relies too much on this narrow view of testing to make his point. Testing is good and necessary.
And the jab to the Unix philosophy? ("Let's write parsers!"). Isn't that what we do on the web anyway? AFAIK, HTTP is still a text-based protocol. Any server that speaks the protocol has to parse a string at some point. So what was he getting at there? The Unix philosophy is about simplicity and has a lot to offer in terms of how one can design software to be simple.
Overall though, it's a great talk. I just think that if he wants to get pedantic then he could be a little more thorough and less opinionated. Everything he said about designing simple systems I pretty much agree with, but I think he glosses over some finer points with his own bias of what simplicity means.
As far as his comments on testing - I suggest you read this, http://blog.8thlight.com/uncle-bob/2011/10/20/Simple-Hickey....
The point I was trying to make about parens is that I think Rich is creating a straw man from them. Parens in CL/Scheme don't have any special meaning other than demarcating lists. A Lisp compiler/interpreter just evaluates lists of symbols at the end of the day.
That code and data can both be represented as lists is a major feature of what makes CL so compelling.
And I suppose that is the major contention -- is 'code as data' a complex idea? I say it's simple. There's no difference between the data structure of the code and the data structures the code acts on, so using the same operations on either should be trivial. With a handful of simple evaluation rules and a small number of special forms you can bootstrap an entire language written entirely in itself. Functions, classes, interfaces, namespaces, the whole nine-yards. The list is just an implementation detail and it's a very simple one that enables some very elaborate structures.
length
vector-length
string-length
hash-count
set-count
...
Clojure embodies Alan Perlis' idea that "It is better to have 100 functions operate on one data structure than to have 10 functions operate on 10 data structures.", which is one of the many reasons why I enjoy the language so much.
Oddly enough though, you yourself are pointing out that it is better to have 1 function that operates on 10 data structures. ("length" operates on vectors, strings, hash tables, etc.) On the surface this seems opposed to the quote you chose.
However, it is clearer if you replace "data structure" with "interface". The is classic separation of concerns. When specifying the "what", we can get away with "100 functions operate on one interface", but the efficiency, the "how", can be specified independently based on the choice of data structure implementing that interface.
HTTP is just that, a protocol : a standard way for two entities to communicate. In Unix, there's no protocol, its just text without any standard form, which makes it difficult to write tools.
The "oneness" in other lisps does not make things simpler, nor, in my opinion, easier. The reason why parens (lists) in traditional lisps are not simple is that they complect several different purposes. In contrast Clojure uses list forms (almost) exclusively for "active" expressions such as function and macro calls. For grouping and binding, vectors (and sometimes maps) are used instead.
In this manner Clojure manages to keep the roles of different data structures in code mostly simple.
You've repeated what Hickey says in the talk, but I'm not sure I buy it. [Deleted boring stuff about boring parens here.]
The trouble with the general argument is that you can add constructs that are individually simpler but nevertheless make a system more complex. It's overall complexity that we should seek to minimize.
Of course, sometimes the complex tool might be exactly what you need to solve your problem, making things easier. But in cases where you need only a part of the functionality of this tool, the complexity bites you as all the unneeded functionality (along its cost) is forced on you.
What sort of "overall complexity" does having several data structures in code introduce? As Rich Hickey says in his talk, complexity is not about cardinality.
In Clojure's model, the elements are distinct, and as such there is more simplicity in the system than in lisps where you have to understand the difference ("untangle" the complexity) between several different uses of lists to get anywhere.
I also think having several datastructures makes code easier to read due to the visual differences, but that's a separate discussion. :)
Rich says his definition of simplicity is objective, but it's not. What determines whether a construct has a single purpose? It's whether or not, on inspection, you think it does. S-expressions seem to me to have a single purpose: to group things. How those groups of things are treated by the evaluator is a separate concern. You, following Rich, say no, they have 3 purposes. Ok. Why not 4? A function call and a macro call have different purposes; why conflate those? (I have no trouble reading function calls in s-expressions, but sometimes run into trouble not knowing that they are really macro calls, so this is not a made-up point.)
We have no objective basis for this type of discussion, only emotions and beliefs. Concepts like "readability" are hopelessly relative, but appeals to "ease of future change" are no better; to put it politely, they depend on individual perspective and experience; to put it bluntly, we imagine them.
The definition of a system in this case is anything with one or more components that accomplishes a particular purpose.
It still seems you're using a different definition of complexity than I am. To me, complexity implies unnecessarily intertwined elements.
In your hypothetical library situation, the answer is likely no. Your program still depends on the "library" code in ways that make the two not treatable as standalone entities, so no complexity has been removed.
An inherently complex tool may solve the specific problem it's built for, but it does not combine well with other tools. A simple tool tries to keep itself standalone so that can be freely combined with other simple tools to provide functionality that the original authors of either tool might not have envisioned. Clojure has many examples of this idea in action, but it's not the only language to exhibit simplicity.
I think Rich's definition of simple is straightforward and objective. For example, Clojure protocols fit the definition. They give you freely extensible polymorphism based on object types. Protocols don't even provide a direct way to do inheritance. That can be done by using a map of functions that you modify as needed, but requires no explicit support.
Your defense of s-expressions is rather puzzling. The evaluator defines what s-expressions (lists) mean in CL, and depending on context, there are multiple meanings that are completely separate. Certainly you can argue that Clojure conflates macro and function calls too (it is justified to me though, since they're all operators), but it has at least reduced complexity by not conflating binding and grouping with those elements.
As an added benefit, with few exceptions, whenever you see a list form in Clojure you can assume it's either a function call or a macro of some sort.
What is an example of this?
It confuses the hell out of me :)