SICP in Clojure | Hacker News Reader

SICP in Clojure

afronski.pl

132 pointsMattF42 comments

I've been feeling the past few weeks the repeated impact of hitting a wall. Programming was starting to feel like grinding. I felt wistful for SICP, when programming was full of big ideas and elegant new approaches to problems my unprincipled brain's first instinct was to clobber by brute force -- and learned, quickly, in Scheme, it's hard to be artless. It is, I think, harder code gracelessly in Scheme than to code with grace: it has a way of bending your ideas to a form. I think that's my main complaint with Python. In Python, exactly the opposite is true: it doesn't force you to think overly hard about the way you solve problems, so you can solve more problems! Woohoo! You can Code Like Coltrane, as Rich Hickey says.

For better or for worse. Here's an unfortunate side effect I've noticed: my programs are vastly more complex and efficient than when I graduated from SICP a year ago, which is good, and yet I can't remember the last time I wrote code that was truly beautiful. Code I felt proud to call my code. Whose inner workings I could explain to a child. To me, efficient Python programs conceal so much of how they work, that striving for craftsmanship can feel alienating. It conceals so much. It's like a black box magic trick: in goes the value... (pause) oh, and there's the answer! And the obvious what's happening in between those two points, the less efficient my solution tends to be. The prize winning solutions are often the most opaque, the most dense with imported methods. And my programs look inevitably kind of haphazard and provisional. Like doodles.

My Scheme programs, on the other hand, looked like Swiss clocks. They pleased me. They wouldn't have worked any other way. They molded to the precise, regimented form they had to be. I understood them. They were good.

I like functional programming. I think the wall I'm hitting might be Python. The thought of a little Clojure or Haskell in my life right now sounds like a beach vacation.

> The thought of a little Clojure or Haskell in my life right now sounds like a beach vacation.

This is the reason, why I started to do little side-projects with clojurescript and reagent [1]

I love the way how programming with reagent is visual, with almost no friction. I just start repl, start browser, open the code, and every time I save file, I see how my program changes :-)

Compared to my work, where it can take me the first hour to get into the zone, and additional 20 minutes each time I need to switch frameworks, it really feels like a vacation :)

[1] https://reagent-project.github.io/index.html

I've been writing Python a lot coming from a Scheme/SICP background too, and I've slowly come to a sort of solution to this problem that works for me.

Basically, what it comes down to is realizing that a lot of the libraries and frameworks out there are smoke and mirrors: they provide an initial solution to a common problem, but when you start getting into the details of your problem, the problem you are trying to solve, you end up working around their crappy configuration points. In short, you stop coding and start configuring existing code which is poorly designed to solve your problem.

So what I do is I stop using those libraries. I zealously guard my dependencies, only letting in things that are solid, well-designed, mature, and minimally invasive. I use minimalist frameworks, eschewing ORMs. Most of the time I avoid classes, opting instead to use closures and namedtuples. The result, my code is fast, flexible, easy to understand, easy to test. I migrated around 30,000 lines of code from Python 2.7 to 3.4 in an afternoon--my unit tests caught a few issues and once those were fixed I didn't see any regressions in the following months.

I think the wall you're hitting isn't Python, it's the culture and libraries a lot of people use for Python. But there are a lot of people who don't use those things. Next time you think of reaching for pip or Django, stop yourself: do you need that? Maybe it provides a short-term solution to your problem, but in the long run it's neither a beautiful nor efficient way to write code.

Precisely!

I've been bitten by the data-oriented design bug and it has made my experiences with programming so much better. It turns out Python is really good at this style of programming. There are powerful ideas in co-routines and sub-generators that make stream processing minimal and easier to grasp. Simple ideas like decision tables are so easy to implement that I cringe when I see giant state machines serialized across several classes. Implement backtracking with a stack? Why bother -- functions live on a stack and push/pop: yield-from/yield. We can serialize the state of a co-routine... boom, concurrency. Sometimes it's the little things in Python the surprise me the most and are the reason I keep using it.

Avoid classes where you can! Design your programs around your data. Python has plenty of built-in patterns for manipulating data and aggregates. It's sometimes surprising how simple your programs become when you resist the temptation to start modelling your problem as classes.

I would never discourage someone from pursuing Scheme or Lisps in general (Have you seen Hy? Homoiconic front-end to Python's AST). Scheme does encourage thinking about problems as recursive functions. If iteration is a special case of recursion than Python is breaking ground in the same direction just in different clothes.

would love to see an example of your code style...github account?

A lot of my thoughts on this subject were inspired by this Tweet: https://twitter.com/garybernhardt/status/616327747435036672

Maybe check out Gary Bernhardt's code?

Sorry, there's just no way I'm going to link this account to my real-world identity. I speak far too freely with this handle.

A lot of that feeling, at least for some of us, comes from jobs which require us to implement lots of business logic.

Business logic often requires coding lots of small conditions distributed all over the code base of an application. Especially once you hit the maintenance phase.

Sounds like a Rules Engine to me. Your description implies a brittle solution that may not be representative of the utopic circumstances pursued by our colleague. Look into Drools. ;-)

I dunno. I spent some time learning Clojure but came to the conclusion that it's really only a good tool if you are also heavily committed to Java -- too many Java stack traces, too many functions where the answer is "use Java", too much time looking at Java source code. I don't hate Java, but that wasn't what I was hoping for. (Haskell is a different story, very different pros and cons.)

Interesting you have been spending time looking at Java source code, what problems were you trying to solve?

I have been working professionally with clojure for about a year and haven't had to look at any Java source code.

Mostly trying to better understand clojure constructs and their edge cases. I guess I was expecting a more lisp-y setup where high-level concepts were implemented in terms of a small number of clojure primitives, but it didn't seem to work that way. Probably made it perform better, but it also makes it harder to do anything about the endless Java stack traces.

ClojureScript is a very good tool without any Java commitment.

Interesting. I haven't used ClojureScript.

Check some Python code from Peter Norvig and you will see a beautiful and clever style. It's not the language which stops you.

While I've learned a lot from Norvig's posts, he tends to write Python code that many wouldn't consider "Pythonic" :)

That is just the point, you dont have to limit yourself to "Pythonic". your style can be language agnostic.

I agree with you in spirit, but that's actually easier said then done in a professional environment.

One of the things that dissuaded me from ruby was the idioms that have surfaced recently. I was tired of submitting pull requests, only to have people tell me there was a "more idiomatic way" to do things, then rewriting things in a way I felt was less clear then what I had originally written.

My solution to that problem was to find a community who's idiomatic ideals more coincided with mine (which happens to be clojure), but I understand not everyone has that sort of luxury.

I tried to use Clojure exclusively at work for couple of years. The problem was I couldn't figure out what my code when I try to go back and read it. I was able to write really succinct code by building up abstractions, but it was much harder to figure out what those abstractions meant few months down the line. I realized that inorder to figure out what the code does, I had to re-run the computations in my memory. Went back to python and I am loving it.

Swiss clocks feel impressive, no doubt. Quartz clocks, though, have the advantage of being accurate.

A lot of Swiss watches (even from the most high end brands) use quartz movements.

I felt it was safe to assume, for the analogy, that we were talking mechanical watches.

This has absolutely no relevance to the analogy.

If you are expecting working code to be as elegant as a "Swiss clock," you might want to reconsider what the goal actually is.

Certainly there is something for coding in this way. Just as it can hold a certain delight to have an elegant looking watch. There is a good chance of overbuilding when I see this level of coding from colleagues. Myself included.

This is sort of how I feel after moving from over a year of pure Clojure(script) development, to working with Python again. Its not that you can't write beautiful code in Python, its just that the language encourages a certain style.

You can write in functional style in pretty much any language. It might not look quite so pretty but it can come close. And it can often be usably efficient too.

But you always have to make some serious trade-offs. Sure, maybe you can use map and friends, but you usually don't get the nice things like tail-call recursion semantics (though Clojure doesn't have that either), efficient persistent data structures, proper first-class functions (looking at you, Ruby), lexical scope, etc.

I program in a functional style in all of the languages that I get paid to use, but I'm always sad about the lack of many useful features.

There's also http://ecmendenhall.github.io/sicpclojure/pages/contents.htm... which is an actual translation of the book into Clojure, currently up to chapter 2.1.4.

And the ill-fated sicpinclojure.com site, which looked beautiful and had such great promise, but mysteriously stalled and died.

Looks like vqc below has done quite a few of the exercises. Nice.

Tommy Hall's SICP Distilled seems like the best shot at a complete book now. It's running a bit behind schedule but appears to be in the home stretch. We'll see!

Clojure seems too high level for SICP. Scheme was a perfect fit because of how simple it is and how few language features or data structures it contained. You had to build everything yourself, which was kind of the point.

This is awesome! I would love to see this span the whole SICP book.

There was a kickstarter campaign for something called SICP Distilled, a while back. However it requires your github login which seemed kind of sketchy and also none of the content works anyway - http://www.sicpdistilled.com/. Maybe someone else knows more?

Anyway I look forward to seeing more chapters on your blog.

I've spent a lot of time with SICP and with Clojure. The book changed my life, and I too considered working on "porting" the book, but there would be a considerable amount of friction. Clojure can do everything necesssary, but it's very unidiomatic to build data structures out of pairs in Clojure, and SICP spends a _lot_ of time implementing data structures with pairs. A reconsideration of SICP for the modern age would, I think, take vectors and maps and sets as given, and see what _additional_ magic could be built from there!

Just start with sequences. Lazy sequences, which are so helpful in the expert system database that's built in SICP, are already right there in Clojure without the complications of force and delay. My thought is you could do a lot more with the higher level examples, but a lot of the first half of the book would have to be rethought. (The functional implementation of Conway's Life in O'Reilly's "Clojure Programming" is as good as anything in the first third of SICP, for example; the example of accelerating series convergence from SICP seems less exciting today than it used to, at least to me.)

To the author: on the compound data structures example, it would be much better to return an anonymous function with `fn` than to use `defn`. `defn` is meant to add global definitions to the namespace, not to be used as a value.

This was confusing to me as well since in the Scheme examples they embed functions into each other with define. It may be useful to use letfn to achieve something similar.

Implementing define in a way that allowed it to be used was one of Scheme's innovations relative to some of the other Lisps in the 1970's. It's idiomatic in part because it is easier for new programmers to read and Scheme historically and SICP perpetually are targeted at newer programmers.

Clojure most definitely is not. It's not an accident that Clojure's fn is only two characters. It's design as a language encourages anonymous functions. Along with recur there are many fewer cases where the simpler ways to do something require a name.

The fundamental problem of trying to port SICP to other languages is that SICP is not a book about programming in Scheme. It's a book about software engineering, and the points about software engineering are made in ways that can be readily illustrated in Scheme. Cons is used to illustrate engineering ideas, and while those ideas could be illustrated in Clojure, there's an impedance mismatch between cons and the roughly analogous constructs in Clojure. Part of the baby gets thrown out with the bathwater for little advantage...understanding Scheme probably makes someone a better Clojure programmer, while the force fit of SICP into Clojure probably won't.

“I personally don't think SICP will help you much with Clojure. YMMV.”

—Rich Hickey, author, Clojure programming language

My guess is that this is being taken out of context.

I don't want to put words in Rick's mouth, but I imagine that he means that SICP won't help you learn Clojure the language. Not that SICP won't make you a better programmer in Clojure (and all other languages).

Personally I found it helpful to work through the book in scheme (the language used in the book), porting some of the exercises to Clojure and other languages I'm interested in as a supplement.

For those people in London (England) interested in SICP in Clojure, we are running a monthly study group. Our google groups group is https://groups.google.com/forum/#!forum/sicp-mailonline and the code is at https://github.com/MailOnline/sicp-mailonline (each person has their own branch).

We are just starting Chapter 3 and at the current rate of progress it will be at least a year before we have completed the entire book!

Anyone interested is welcome to join us :-)

There was another person doing SICP in Clojure too, but it's been frozen in chapter 2 for a couple of years now. https://github.com/ecmendenhall/sicpclojure

Hope this one gets to cover the whole book.

Also really interesting: http://xuanji.appspot.com/isicp/

This is exciting; I've been looking forward to doing this exact thing myself.

Some more solutions can be found at https://github.com/gregsexton/SICP-Clojure

The link to previos charpter in the blog post 404's (but I could navigate to it through the sidebar). Anyway, I'm looking forward to reading them when I get home and hope to see the following charpters as well.

I've changed yesterday the structure and I've messed up previous links - it should be fine now. ;)

Reply on news.ycombinator.com