Closures – A Simple Explanation (Using Ruby) | Hacker News Reader

Closures – A Simple Explanation (Using Ruby)

skorks.com

54 pointsbleakgadfly30 comments

"the most popular languages that are in use right now don't support closures (Java, C++)"

That is blatantly wrong. Closures, that is, the implementation structure which stores along with a code pointer the references or values to which local variables referenced by said code are bound, have been in Java since version 1.0. Otherwise you could not write code such as:

class foo { final static int bar = 6; static class baz { int bla() { return bar; } } }

and expect "(new foo.baz()).bla()" to return 6.

What Java is missing (and I suspect what the author is conflating closures with) are anonymous functions, which require closures to function properly for the same reason inner classes require closures.

Closures are not anonymous functions. Closures are an implementation detail which allow anonymous functions (among other structures) to act as expected in statically-scoped languages.

Scheme has anonymous functions and closures.

Java does not have anonymous functions but does have closures.

Emacs Lisp has anonymous functions but does not have closures (and thus does not exhibit static scope).

BASIC has neither anonymous functions nor closures.

(It is worthwhile to note that there are other methods for implementing static scope such as alpha-renaming but they are usually much less efficient than closures.)

Plus, Java does have anonymous inner classes, which can be used in a fashion pretty similar to lambdas although with clumsier syntax than most people would like.

Yours is a crucial point that I think got munged in the Java 7 closures debate, since a lot of Java people didn't know what closures were, and a lot of people on the outside maybe didn't know what Java lacked or didn't lack, so there were definitely a few blog posts that conflated them with lambdas.

> Closures are an implementation detail which allow anonymous functions (among other structures) to act as expected in statically-scoped languages.

I don't think anonymous functions have anything to do with it. Closures come into play when you are creating a function that has access to local variables from an enclosing scope. This function need not be anonymous. For example, in Lua:

    function x()
      local var = 5
      local function y()
        print(var)
      end
      return y
    end

    y = x()
    y() -- Prints 5.

At no time is the inner function anonymous. When it is created it is bound to the local variable "y" and later it is bound to the global variable "y". And yet it is a closure because it has access to the lexical scope in which it was defined.

Also, if closures were an "implementation detail" it would not make sense to say that a language has or does not have closures, only implementations of a language would have or not have them (and the language user would not be aware of whether the language implementation had closures or not).

Good examples of closures / first order functions in java is through a visitor; example below is like the ones in A Little Java, A Few Patterns

new Top(new Anchovy(), new Top(new Tuna(), new Top(new Lettuce(), new Bot()))) .accept( new RemVisitor(new Anhovy() );

This removes those smelly Anchovies from your tuna sandwich -- and can be implemented with a closure or not. If accept() method returns a new Top w/o Anchovies and RemoveVisitor stores what to remove in its field then it is a closure.

This book is cool for lisp programmers to learn programming with recursive objects like composites and decorators.

What happens if bar is not final and/or not static?

If it's not static you wouldn't be able to reference it from the static context (the inner class baz). It could be non-final though, because it's a field not a local.

The second part is the more interesting part, and is part of what people are talking about when they say that Java doesn't have closures: you can't modify closed over local (stack allocated) variables. However you can call methods that modify or otherwise modify heap-allocated objects, for instance if bar was an array you could modify it's contents, or if it was a Vector you could call addElement() or whatever.

In the GP's case, bar is a member field, not a local, so I don't think it needs to be final, and a quick test checks out. But if those were variables local to a method and you referenced them from an anonymous inner class, they would have to be final.

See: http://c2.com/ppr/wiki/JavaIdioms/ClosuresThatWorkAroundFina...

If I recall correctly, from digging into the famous Guy Steele quote "we were after the C++ programmers. We managed to drag a lot of them about halfway to Lisp." I think this decision was made at the time because programmers didn't like the idea of heap-allocated local primitives. I could be totally off base here though. That wiki page mentions something about preserving the threading model, which sounds reasonable, but I don't know enough about Java's history to be totally clear on the motivation.

edited because I'm a pedant.

and the relevant message from Guy Steele: http://people.csail.mit.edu/gregs/ll1-discuss-archive-html/m... (that whole thread is fascinating, massive respect for GLS as if I didn't already, and helped quell some of my hipster Java hatred.)

Glad you know more about Java than I do :) I never knew about the restriction on mutating local variables in the closure. It's strange to think about that coming from the FP world where all mutable values are references to cells that (conceptually) live on the heap.

From the succinct description of closures:

    It remembers the values of all the variables that were in
    scope when the function was created. It is then able to
    access those variables when it is called even though they may
    no longer be in scope.

No! It doesn't remember the values. It binds the variables. This is a major misstep in describing closures. The most important point: when the values of the bound variables change the closure sees the new values.

Please, fix this!

A better simple description of closures:

* You can pass a closure around as a value (to be called later).

* When you create a closure it is bound to the variables in its current scope. When you call a closure, it sees the current value of the bound variables (not their values at the time the closure was created).

Follow this with a simple example, like:

    i := 0
    inc := func() {
        i++
    }
    println(i) // prints "0"
    inc()
    println(i) // prints "1"
    inc()
    println(i) // prints "2"

Depends on the language. "Closure" (at least to me) doesn't imply this bevaior in a language with mutable variables. And then you have pure languages and languages which only support mutation through reference values, in which his explanation would be correct.

> "Closure" (at least to me) doesn't imply this bevaior in a language with mutable variables.

I don't think I can name a single reasonably-known language with mutable variables and closures that doesn't exhibit this behavior.

Indeed. Python is a prime example of a language with closures that doesn't do variable-name binding, due to the way variables and specifically assignments work in general with Python.

I'm sorry? Python is the same way:

    >>> i = 10
    >>> def x():
    ...   print i
    ... 
    >>> x()
    10
    >>> i = 11
    >>> x()
    11

The value is not closed over in any language I can think of off the top of my head. It's a reference to a variable in lexical scope.

What python does slightly differently is writing to variables in outer scopes. But it still closes over references.

The writing to variables in outer scopes is what I meant by a lack of actual name-binding, due to the way Python does assignments; it seems I expressed that poorly. Here's a better attempt. The top poster's example simply doesn't work in Python:

    >>> i = 0
    >>> def inc():
    ...   i += 1
    ...
    >>> print i
    0
    >>> inc()
    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "<stdin>", line 2, in inc
    UnboundLocalError: local variable 'i' referenced before assignment

So variables are name-bound in the sense that you can read from their original name and reference but you can't assign in normal ways. (Python has closures, which I said, just not typical closures that other languages might have.) Leading to the pattern of wrapping an array or dict or class around the thing you're interested in.

    >>> i_wrapped = [0]
    >>> def inc():
    ...   i_wrapped[0] += 1
    ... 
    >>> print i_wrapped[0]
    0
    >>> inc()
    >>> print i_wrapped[0]
    1

One could argue that one shouldn't really have such mutable state inside a closure of the enclosed items, which I do, but a person expecting to (set!) or the like is going to be surprised.

Edit: I guess it should be said that Python 3 has the 'nonlocal' keyword, and Python 2 has the 'global' keyword for global cases, which allows for expected behavior. So the functionality is still there, but still needs an extra statement.

    >>> i = 0
    >>> def out():
    ...   i = 0
    ...   def inc():
    ...     nonlocal i
    ...     i += 1
    ...   print(i)
    ...   inc()
    ...   print(i)
    ... 
    >>> print(i)
    0
    >>> out()
    0
    1
    >>> print(i)
    0

Python is a bad example here, as its scoping rules are inconsistent and frequently surprising.

For me one of the coolest ways of using closures was to implement undo system. It's not very efficient, but easy to write and compose.

Basically every editor operation can return a function that does the opposite. For example (some lua-like language)

function DeleteLine(lines, line_number)

   local deletedLine = lines[line_number]

   -- do the actual deletion - move elements up

   return function() InsertLine(lines, line_number, deletedLine) end

 end



 function InsertLine(lines, line_number, deletedLine)

   -- do the actual insertion

   return function() DeleteLine(lines, line_number) end

 end

Again, not very efficient - but easy to grasp. Without closures would be hard to compose - you would have to take care of a lot of extra parameters where they need to be stored, etc.

That's pretty neat, but returning the undo operation puts the responsibility of actually keeping track of undo operations on the caller side. I think it'd be better having each function push it's corresponding undo operation onto some global undo stack, thereby ensuring every operation gets recorded, without the caller having to do any work (or even know about the undo functionality).

But of course, you don't really need closures for that. For example, Cocoa does it this way:

   - (void)deleteLineAtRow:(NSUInteger)row {
     NSString *deletedLine = [self lineAtRow:row];
     [[self.undoManager prepareWithInvocationTarget:self] insertLine:deletedLine atRow:row];
     // do actual deletion
   }

   - (void)insertLine:(NSString *)line atRow:(NSUInteger)row {
     [[self.undoManager prepareWithInvocationTarget:self] deleteLineAtRow:row];
     // do actual insertion
   }

That -prepareWithInvocation: stuff tells the undoManager to store the next message sent to it, along with it's parameters, on an internal stack. This is of course depends on Objective-C's dynamic nature and there's a whole lot of complexity hidden inside the undo manager object.

But mixing this undo manager concept with closures should be pretty straightforward, resulting in something like this (note that this is a hypothetical example; Cocoa does not currently implement this functionality):

   - (void)deleteLineAtRow:(NSUInteger)row {
     NSString *deletedLine = [self lineAtRow:row];
     [self.undoManager registerUndoWithBlock:^{
       [self insertLine:deletedLine atRow:row];
     }];
     // do actual deletion
   }

   - (void)insertLine:(NSString *)line atRow:(NSUInteger)row {
     [self.undoManager registerUndoWithBlock:^{
       [self deleteLineAtRow:row];
     }];
     // do actual insertion
   }

Thanks for that! I was mainly thinking that by returning functions, one can compose for more complex editor operations, more functions at once, and even simplify them (if they cancel each other). Off course this is not currently possible, unless the returned functions are some kind of AST tree, and there is something that can optimize them at runtime (dynamic programming?)... Probably overkill....

For example deleteBlock might delete 10 lines, and if there is better composability, instead of returning undo to insert 10 individual lines, it can be merged as one operation with 10 lines, basically by having rules, if you have production like "insertLine * insertLine * insertLine * insertLine" - you can make this as "insertLines[lines collected]" - but that's beyond the discussion.

All this just to avoid making insertBlock/deleteBlock special, but reuse insertLine/deleteLine.... lost my thought already - too late in the morning....

To save you time, you can skip the initial rant about how closures are not taught in universities and start reading from the heading "A Concept Explained Simply".

This explanation did not add anything to help me better understand closures and anonymous functions. I already understand that closures allow you to pass an anonymous function around and it maintains the variables that were in its scope when it was created (whether by copy or reference).

The problem, at least for me, is finding a non-trivial example of using an anonymous function. Does anybody have other examples like malkia's[1] that demonstrate why anonymous functions and closures are useful? Or maybe just a good explanation of functional programming?

1. http://news.ycombinator.com/item?id=2919149

I think the classic example of using anonymous functions is probably as arguments to map, filter, reduce, etc:

  > (mapcar #'(lambda (x) (+ x x)) (list 1 2 3 4))
  (2 4 6 8)

In that case, the anonymous function doesn't close over anything. Here's an example where it does (from On Lisp):

  > (defun list+ (lst n)
      (mapcar #'(lambda (x) (+ x n))
              lst))
  > (list+ (list 1 2 3) 10)
  (11 12 13)

In this case `n` is closed over.

Both of these ARE pretty trivial, but the point I'm trying to make is that anonymous functions in many cases just make some code a bit more concise and avoids the need for nested named functions, which I guess is common in Python where `lambda` can only contain an expression.

I'm reading On Lisp now, and there are some examples early on of functions that return functions. In those cases, the functions being returned are almost always anonymous, just because naming them would be kind of verbose and unnecessary.

There are other, more complicated uses, for instance closures can be used as a sort of crude way of making objects, and supporting hacky encapsulation, which is described in a lot of Scheme textbooks, like SICP: http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-21.html...

As for functional programming, it's a lot more than just first-class, anonymous and higher order functions. It also encourages or enforces pure functions: functions in the mathematical sense that do not have side effects, and always return the same value given the same arguments.

SICP is a good start, if you're curious about Lisp. It's a great book. There's other places too though, to get this stuff, if you wanted to dive into the Haskell world via Learn you a Haskell I'm sure you'd see some of what FP is about. I'm less familiar with it though.

edited for clarity and fix the SICP link

Whether a function is anonymous or not is sugar. The important thing is functions aren't special, they can be passed around (i.e. the are 1st class objects).

I actually can't fathom having to code in a language in which you can't pass functions around.

Call backs is my biggest use of function passing. This is really, really huge.

Currying, decorating/wrapping functions

Replacing methods/functions at run time. Great for testing.

Putting them in data structures. Often dictionaries. Often to create more dynamic switch like logic. dict[key] = func_that_handles_key, etc. then later, dict[key](key)

Lots of data is code stuff.

Closures just make the passed functions more powerful / convenient as they capture their scope. Which is often used (at least by me in Python) to create function factories that return custom functions.

A contrived example but a pattern I use often. Logging object with warn, debug, error functions. You could hardcode / boilerplate each of those functions but that is boring, tedious, and nonDRY. So instead you create a function factory and use it to return anonymous functions, like so. Python.

  def log_func_factory(level):
    def closure(msg):
      print level, msg
    return closure
  
  class logging(object):
    debug = log_func_factory('debug')  
    warn = log_func_factory('warn')  
    error = log_func_factory('error')

With real code I'd not repeat those assignments and instead use a loop driven by data.

  for level in ('debug', 'warn', 'error'):
    setattr(logger, level) = log_func_factory(level)

Bam, data driven code. Very DRY, very concise, very dynamic. Maintenance is adding one text string in one place.

For anonymous - GUI event-handlers/callbacks.

Say in MFC, or wxWidgets, or similar you have to explicitly say this even tones to this function, with anonymous functions this is easier, you can just "inline it". If I'm not wrong, in .NET something like this:

With functions (.NET, C# kind of)

button1.Click += new EventHandler(OnButton1Clicked) somewhere you would have to declare OnButton1Clicked

with anonymous function:

button1.Click += delegate(Event e) { Console.WriteLine(e.ToString()); };

or for newer .NET (both of my examples might be wrong, check the docs)

button1.Click += (e) { Console.WriteLine(e.ToString()); };

but what the other folks said in answers to you is how most of the time anonymous were used - map/reduce type of situations, or your "C" sort() if you can plug anonymous there.

In Scheme named functions are simply syntactic sugar for anonymous functions (or Lambda expressions). The same is true in Javascript. http://lpar.ath0.com/2010/10/14/javascript-when-is-a-functio...

Back to the Future II is the best explanation of closures I've ever seen.

I consider myself well-versed on both closures and BttF II but I don't understand this comment. Care to explain?

I'm referring to the scene in which Doc draws a diagram on a blackboard to explain to Marty why the two of them still exist even though they've screwed up history and created an alternate 1985. Essentially, they exist because they're in a time closure and they carry their own pasts with them into 1985-B.

It's not a perfect analogy, but it's close enough that I've been able to use that scene to help make closures click for someone who was having trouble understanding them.

Closures are functions plus their bindings for its free variables. The references to the bindings have lexical scope and indefinite extent.

I dont understand the difference between a closure and an object's state. Is an object one example of a closure? You can pass it around and it contains instance variables.

What are lambdas for in this context? Simply calling the object's attribute would return the value set at creation - even if modified afterwards.

Hope this helps: (Python)

def closure(x): return lambda y:x+y

a = closure(5) #'a' is a function now a(2) #returns 7 a(3) #returns 8

The function 'closure' can be carried around and it remembers the value of x from when it was created.

Got the example from this article: http://linuxgazette.net/109/pramode.html (I believe it was shared a few weeks ago) - great reference for other FP concepts and a good way to get started in FP if you already like working with Python.

Wow, I've seen functions like these in Erlang, but I didn't realize their usefulness until reading this.

Reply on news.ycombinator.com