Clojure for the Brave and True – Functional Programming

Are you asking "Is it possible that purely functional programming is yet another paradigm that, like OOP, is fantastically useful but is still destined to be the square peg that is forced into too many round holes?"

If so, then yes, you're making sense. And yes, that's probably what will happen. However, as with OOP, the end result should be a net gain in knowledge and maturity for our very young field and yet another wonderful tool in our tool box.

Having said that, I think that with the "functional wave" comes a bit more understanding that this, too, may not be the perfect fit for every problem. Look at Clojure: they try hard to give you mutability in terms of abstractions that seem to encapsulate what it means for mutability to be a valuable thing.

In Haskell, monads are a great model of "what it means to be imperative". If you think of imperative programming as:

    setup();
    goLeft();
    stop();
    lookAround();
    pickUp(ball);
    turnAround();

In other words: there is a "meta paradigm" going on here and it's just starting to emerge. That meta-paradigm is to provide a language that gives you the abstractions to build the right abstractions to ultimately build the star-shaped peg that fits into the star-shaped hole.

So yes, if you're taking the long view then Clojure might be too opinionated for your tastes (and I have in fact opined on another thread today about what it would take to back slightly away from that lofty perch), then it sometimes can look like yet another oversold silver bullet language. But in the long term, we seem to be converging on something much healthier than the "stop using that paradigm and start using this one because it solves everything!" approach of the past.

I've read your post three times, carefully, and no, I actually can not extract a clear question from it. (I do not mean that in a mean way, just as feedback.) It seems to resolve to an observation that functional programming is not the only paradigm that has produced successfully-running programs, which is certainly true, but not terribly information-rich.

There are actually a lot of interesting possible directions you might be trying to go, but I'm not sure I want to try to guess them. (I will take one stab with: http://en.wikipedia.org/wiki/Effect_system )

I will observe that baking directions aren't a useful comparison; the difference between programming paradigms do not manifest until you have something that is a real size. Imperative is just fine for a single page program. Anything is just fine for a single page program; I have unashamedly written outright spaghetti code for single page programs.

Let's look at your example of baking. Indeed it seems like an imperative (physical) process. When we mix eggs and flour to form a batter we can't get the eggs and flour back! Indeed even cracking the eggs seems like destructive update to me. Think about why that is. We've performed a non-reversible transformation on our ingredients AND we can't go back in time! In this case time is implicit in our understanding.

Now consider trying to model baking mathematically. We want to build an equation that will tell us the state of our cake. This will be function - that is dependent on time - time is now explicit in our model. Now we can say at t0 our eggs are whole. At some time in the future t1 our eggs are cracked. This doesn't negate the fact at t0 the eggs are whole. We can't go back in time and update what the state of the eggs then - that makes no sense! So here we see that when time is an explicit parameter we have time symmetry. The equations don't care if time goes backwards or forwards.

And we can always do this. If you have some process that has a state that changes through time you can model that process with a static model that includes time as an explicit parameter. This means that our model is a higher dimensional look at the real thing. And our actual computation (or experiment or baking etc.) is a degenerate case of the more general model where the time parameter is actual time.

How does this relate to functional programming versus imperative programming. Well it is the difference between theory and experiment. But in this case we also have to deal with the fact that our program is computed (a physical process happening in time) and so we're using a physical computation which follows the rules of mathematics to model mathematical rules using a degenerate case of those rules. So if it seems like there's an abstraction leaking it's because it kinda is.

I like to think about it in this way because it highlights the very tight knit nature of physics and computation. Indeed a general theory of what it means to compute something could lead to a unified theory of physics and vice-versa.

http://repository.readscheme.org/ftp/papers/ai-lab-pubs/AIM-...

After living with a chef for a couple of years, I came to believe that recipes would be much more effective if less imperative and more focussed on exposing the contribution and relative importance of each ingredient/process for the final result. In cooking there is much variation on the inputs: I want to know if the lemon juice is there for taste or acidity so I can tell if I have a bad lemon and for what I can replace it.

It's like GPS navigation: the step by step instructions will often lead you off track. Let me see final destination on the map.

There are lots of posts on blogs, newsgroups, etc. about the aesthetic value of having one's code be math-like, but the working programmer doesn't necessarily care about that. That programmer probably does care about avoiding spooky action at a distance. Conforming to a "traditional math is [beautiful|divine|the true Dao]" ideology does not matter to that programmer as much as knowing what data a function depends on or affects.

Yes, functional programming is analogous to cleaning your kitchen. You heard it here first, folks.

The main is that a destructive functions can call a non-destructive functions freely but a non-destructive operation, say "find best egg" should not have any destructive effects whereas "beat egg" can call "find best egg freely".

I know SQL separates destructive and non-destructive operations very clearly and it also allows the user to back out of some destructive action sequences if all the conditions aren't satisfied. This makes what to do when you don't have butter more easier solved.

I wouldn't use the recipe metaphor except to explain what a computer does in less than 60 seconds to someone afraid of electronics.

One example of a way to structure a functional program is categorising functions according to whether they are a "source" a "filter" or a "sink", (terms stolen from circuit design). The source being your inputs (not pure), filters doing some transformation on the input, over time (pure) and handing off to a sink that causes some side effect (not pure).

In the "real world" You've probably made a lot of simple programs that mostly follow this structure if you've used unix pipes.

If you go along with this, you find that once you start making more sophisticated programs, you need some kind of "memory" so you can progress. Strictly speaking, memory, as we typically use it in imperative languages, is a kind of side effect. In the model above, though, you can make a special kind of "source" which is something like a feedback loop- a source that contains the output of the previous iteration of the program- or some sub portion of the program. Then maintaining a memory becomes a process of continually taking some input, transforming it, creating an output, and then using that output as an input again. (what you might call recursion)

Why would you do this? Because each part of a program like this is self contained, and its ability to do damage to some distant part of a program is extremely limited. It becomes much easier to reason about such a program. When you find a bug (and these are still possible) you can trace the origin from output to source, or source to output. There is no possibility of some phantom distant subroutine screwing up the dataflow.

And this kind of structure is very useful for games, interactive multimedia programs, and other such graphical interactive program.

A commonly cited "real world" example is xmonad. What I find more insightful is this article on replicating pacman in a functional language. http://prog21.dadgum.com/23.html

For a database example, here is the chapter from Real World Haskell about databases (a bit old but still relevant):

http://book.realworldhaskell.org/read/using-databases.html

And here is some example code:

    do
      conn <- connectSqlite3 "test1.db"
      stmt <- prepare conn "INSERT INTO test VALUES (?, ?)"
      execute stmt [toSql 1, toSql "one"]
      execute stmt [toSql 2, toSql "two"]
      execute stmt [toSql 3, toSql "three"]
      execute stmt [toSql 4, SqlNull]
      commit conn
      disconnect conn

As for the question of wanting to mix pure and impure code that much I don't think it actually comes up that much. First of all, I find that I don't often find myself with an impure function that I want to add a side effect in the middle seamlessly - firstly, side-effecting stuff tend to be side-effecting from the beggining and secondly if I do want to turn a pure function into an impure one then forcing me to change the interface helps make sure that I am not breaking any code that used to work because it assumed it was working with pure code. Additionally, Haskell is really good at abstracting code so its not that hard to take a big chunk of logic and split it into a pure and an impure part. Finally, converting code from pure to impure is not that bad - sure, you need to change the syntax a bit but the type checker helps you find all the places that need to change and also helps you check all the code that calles the method you changed so you can be sure that you aren't breaking those.

side effecting operations in the IO monad so you can't run them in the middle of any function and instead must run them inside a special "do-notation block" like I did.

You can have impure code call both other impure code and pure functions and all you can't do is call impure functions from inside pure functions. Sure, its a bit annoying if you ever have to convert a pure function to an impure one (you need to change some of the syntax and all the places that call tha tfunction need to start treating it as impure) but its not that bad: first of all the type checker tells you all the spots where you need to update your code and secondly, Haskell is super good at abstracting code so its not hard to take a big chunk of code with some side effects in the middle and split it into a pure and an impure part.

> to academic program

Its a little pet peeve of mine but it seems that nowadays "academic" doesn't have a real meaning and just tends to refer to whatever people don't like. :)

By itself, this function is useless, but "interact" is a function with signature "(String -> String) -> IO ()" which means that it turns our "unwords . map upcase . words" into an interaction with the environment. There are whole debate here as to how interact is actually still pure, but for the purposes of this argument, interact is an impure effect.

The net program however is very simple: "interact (unwords . map upcase . words)". The pure part is decomposed into many other pure pieces, each with simple, small contracts and independent testability. The impure part also has a simple contract which is described in terms of pure computation.

Another great example is some of the streaming libraries available in Haskell today. Pipes for instance will let you compose a pipeline of pure and impure computations and have them work together nicely.

"fibsPipe >-> Pipes.take 20 >-> Pipes.print"

The first two segments are pure, but have empty "placeholder" slots for impure effects (identity monadic layers). These get merged seamlessly with the "Pipes.print" segment which has impure effects. All together, you get an infinite producer "fibsPipe" producing an impure effect "Pipes.print" while having its resource utilization and termination controlled by another pure computation "Pipes.take 20" which is actually maintaining its own implicit state (the current count of numbers it's let through).

But then you can decompose each piece and understand it on its own in a pure or simple impure fashion.

Now this is not to say that my keyboard is purely functional. The shift key creates state. But that state is isolated and short lived to the point where we can treat it as functional across two key combinations. It is the caps lock key which creates an unpredictable state - I do not know what output I will get without checking its status - the problem with state might be illustrated by the results of hitting capslock when I meant to hit tab while typing a program.

As others have noted, there's really no such thing as a "purely functional program" unless all you're looking for is turning your CPU into a space heater. That said, there's a very real sense in which the parts of your program that ARE pure are much safer and easier to reason about.

The best way to visualize this is that you can theoretically turn an impure function into a pure one, watch:

   void printAGreeting(string greeting)
   {
       IO.printLn(greeting);
   }

So how do you make that function pure? Like this:

    TheEntireUniverse printGreeting(string greeting, TheEntireUniverse u)
    {
        return u.makeSomeAtomsMoveAroundBobsConsole(greeting);
    }

Obviously we can't do this, but (in Haskell, say) the IO monad is a context in which we understand that anything done inside of it has the affect of changing the "real world".

If you pretend that you really had to do the whole "universe dragging" thing around for anything in the universe that WASN'T part of your program (we're assuming it itself is in some black hole somewhere to avoid paradoxes and such), then you would darn well make sure you kept this pain-in-the-ass operation as "thin" as possible. Why? Well, you're lazy!

When you look at things that way, there's an analogy to writing stuff on bare metal hardware. You know you're going to have to write some assembly language (or maybe even direct machine code if you don't have an assembler). What you want is to get to the nice cozy world of C as soon as possible, so what you're going to do is use assembly to code only the smallest surface area you need so that writing in C makes sense, and everything else that doesn't need to be in assembly for performance sake you do in C.

So it is in a purely functional language. The idea is to "get out of IO land" as fast as possible. To use a DB as an example, you'd like to live in a world where your little view of the DB could be expressed in purely functional terms (writing functions that result in a modified version of that view) and only at the VERY LAST MINUTE would you send that DB view to an IO function that actually sent the view down the wire to the DB.

The same goes for files, sockets, gamepad IO, you name it.

And you'd be surprised how little you need to actually interact with the world between getting your data and pushing out state changes. A game, for example, can do a WHOLE LOT OF STUFF between reading the input state and writing graphics commands.

Plenty of that here: http://book.realworldhaskell.org/read/

Other batch processing programs works as well.

I agree with you that it's hard to remember not thinking functionally. I've only been doing it for maybe 3 years and it's hard to remember what it felt like when I was first trying to wrap my head around it.

With macro's, I think you mean to say your own special forms, not functions. They allow you to take in code as arguments, and determine when that code will get evaluated, letting you write things like "or" as macros, when it most languages they have to be implemented as built-in primitives (special forms).