Hacker News Reader

Re: 8, you don't need decorators to compose functions:

    def comp(f, g):
        def h(*args, **kwargs):
            return g(f(*args, **kwargs))
        # fix up h.__doc__ and friends
        return h

or simply

    (lambda the, args: g(f(the, args))(x, y)

(don't remember comp's semantics, is (comp f g) = f o g or g o f?)

too long; don't read:

Decorators are certainly cool, but semantically they represent something more like a pattern than a FP construct. A decorator represents something you might want to do to lots of functions, a property you want all instances of a function to have without writing it explicitly into each function. Function composition is more along the lines of having two functions which are interesting on their own, but which sometimes you want to compose.

With decorators, it would also be awkward to compose multiple functions. Observe:

    def compose_with(g):
        def decorator(f):
            def decorated_function(*args, **kwargs):
                return g(f(*args, **kwargs))
            return decorated_function
        return decorator

    def h(x): math.sqrt(x)

    @compose_with(h)
    def g(x): 2 * x

    @compose_with(g)
    def f(x): x + 1

versus (for some reasonable definition of apply...)

    def compose(*fns):
        def composition(*args, **kwargs):
            return reduce((lambda computed, next_fn: next_fn.apply(computed)),
                          fns,
                          (args, kwargs))
        return composition

    # define fns as above without decorator
    hogof = compose(f, g, h)

To expand on what I was thinking with #8, here's how I'd implement a decorator in Clojure

(def my-decor (partial comp decor-bevior))

Done.

Well, what python calls a "decorator" is just a function that accepts a function and returns a function with roughly similar functionality. In python, the old way to decorate functions was

    def my_decorator(f): ...
    def my_function(...): ...
    my_function = my_decorator(my_function)

and the new "@my_decorator" syntax is just sugar for this.

Function composition is only one of many things you can do with decorators. You could implement a K-combinator with them if you wanted to:

    def kestrel(x):
        "decorates a function to evaluate that function, but then return x"
        def decorator(f):
            def g(*args, **kwargs):
                f(*args, **kwargs)
                return x
            return g
        return decorator

    @kestrel(4)
    def foo(x):
        y = x + 1
        print y
        return y

    foo(5) # => 4, but prints 6

9. INSERT MACRO RANT HERE

Macros are the main reason I decided to create Adder, a Lisp-on-Python with minimal impedance mismatch. Unfortunately, the first macro-heavy program I wrote turned out to be really slow, because macros engage the compiler, which, of course, is in Python.

When I first tried it, it took something like 50s at 2.4GHz, virtually all of which was the compiler. (The compiler runs at load time; obviously, saving the compiled code for the next run would help.) I got it down to...let me try it now...7s at 3GHz, but that's still too slow for a 200-line program.

If anybody's interested, the code's on Github [1]. To see the macro-heavy example, look at samples/html.+, which is an HTML generator. The framework takes 169 lines; the sample page starts at line 171. To see the output, run:

./adder.py samples/html.+

[1] http://github.com/metageek/adder

(Edit: it requires Python 3.x.)

> Unfortunately, the first macro-heavy program I wrote turned out to be really slow, because macros engage the compiler, which, of course, is in Python.

The Python byte-compiler, specifically, appears to be fairly slow. I hadn't really thought much about this, but while contributing to a benchmark yesterday (http://news.ycombinator.com/item?id=1800396), it wound up staring me in the face. There's actually surprisingly little difference performance-wise in running Lua from source vs. precompiled (both pretty fast), whereas the difference between Python and pyc in my benchmark was wider than every other possible pair in the chart except python interpreted vs. "echo Hello World". (I didn't have any JVM languages, though.)

I don't really do eval-based metaprogramming in Python, but do so on occasion in Lua. I thought I felt better about doing so because Lua is syntactically much simpler (and has scoping rules that make avoiding unexpected variable capture easy), but the Lua compiler itself also appears to be substantially faster than Python's. (It doesn't do much analysis, but still usually runs faster than Python.)

And yes, it's not as good as straight-up Lisp macros, but Lua's reflection also covers a lot of low-hanging fruit that macros would otherwise handle. The biggest thing lacking in Lua compared to Lisp is an explicit compile-time phase for static metaprogramming. (Code generation is an inferior alternative.) Lisp macros win big in part because they can avoid the overhead of parsing, but parsing Lua is fairly cheap thanks to its small, LL(1) grammar (http://www.lua.org/manual/5.1/manual.html#8).

I quite like the approach taken in MetaLua (http://metalua.luaforge.net/) for allowing more advanced cases of meta-level programming. After some exposure, moving up and down levels using +{} and -{} becomes about as readable as meta-programming can IMHO. The explicit modification of the compiler is not so nice, but I'm guessing it is hard to do anything better in a a programming language with some actual syntax.

Of course, what I really would like to use is MetaLuaJIT, to get the best of all worlds :)

Macros were also the reason I wrote Noodle, which was a similar effort (Lisp compiling to Python bytecode) back around 2004. The project died when I couldn't figure out sane, easy-to-learn rules about how to make macros live with modules and imports, and I found I didn't like the syntax concessions I had to allow to make attribute access less of a terrible pain.

I still think something like this would be worthwhile, and I wish you the best of luck! I particularly like your (.bar.baz foo) syntax. And good on you for moving away from bytecode generation; I also found that to be a dead end.

So what are your plans for macros+namespaces? Will macros from imports live in the same namespace?

And what did you mean by "Python supports only two levels of lexical scope" at http://www.thibault.org/adder/ ? I don't recall any constraints regarding lexical scope levels in the VM.

and I wish you the best of luck! I particularly like your (.bar.baz foo) syntax.

Thanks!

And good on you for moving away from bytecode generation; I also found that to be a dead end.

Glad to hear I'm not the only one. :-)

So what are your plans for macros+namespaces? Will macros from imports live in the same namespace?

Actually, I hadn't thought about it--I haven't gotten to the point of being able to write modules in Adder. I think I see what you mean, though: any module has to be a Python module, so macros would have to be expressed as functions.

Strawman: the module could contain a variable listing the macros. When compiling an (import) form, the Adder compiler would check the imported module for that list, and update its internal structures accordingly.

And what did you mean by "Python supports only two levels of lexical scope"

That was a mistake; I've removed it. I think maybe I just didn't figure out how to generate bytecode for a triply nested function.

Of course, there is the remaining problem that Python can't define a scope without defining a function. I didn't want to use the standard tactic for turning (let) into a nested function (I don't trust Python functions to be fast enough), so the current compiler uses name mangling, tagging each variable with the scope depth. (Global variables are untagged, so that they can be accessed cleanly by Python modules.)

It's funny, but the thing I miss most from Common Lisp when I write in other languages is the LOOP macro. It's ugly, and non-lispy, but most loops I have to write can be expressed clearly and concisely using LOOP, and writing the equivalent code in another language is annoying.

I'm tempted to create a LOOP clone for Clojure, then laugh villainously as I unleash it upon the world.

> I'm tempted to create a LOOP clone for Clojure, then laugh villainously as I unleash it upon the world.

DOOOO IT

Loop, like structured editing (Paredit), is one of those Interlisp things that's very controversial and divisive.

There's tons of really wild ideas in Interlisp that seemed to be the half-baked acid trip ideas of West Coast hippies at the time, that are just starting to become rediscovered in the past couple of years (pervasive undo -> reversible debugging, DWIM-like autosuggestions in more places), and even the implementation techniques used are still innovative (for example the error-trapping implementation of Conversational Lisp (http://docs.google.com/viewer?a=v&q=cache:4GnEnGS2XXkJ:c...) is quite similar to how Geoff Wozniak approached auto-defining functions (http://exploring-lisp.blogspot.com/2008/01/auto-defining-fun...)

LOOP is not from Interlisp. It comes straight from Maclisp.

'LOOPS' from Interlisp is something entirely different: an object-oriented extension to Interlisp.

I'm going by the Hyperspec and what I remember from reading Kaisler's Interlisp. From the former:

"One of the Interlisp ideas that influenced Common Lisp was an iteration construct implemented by Warren Teitelman that inspired the loop macro used both on the Lisp Machines and in MacLisp, and now in Common Lisp."

http://www.lispworks.com/documentation/HyperSpec/Body/01_ab....

It influenced it the Maclisp LOOP. The idea. That's all. The CL LOOP macro OTOH is a straight version of the Maclisp version. The MIT version of LOOP came from the same sources, even.

The Interlisp iteration facility looks slightly different. There are Interlisp manuals as PDF at bitsavers...

LOOP is crack. Personally I love it.

Could you port CL's LOOP to Clojure easily? Or would its internal hyper-imperativeness make that tricky?

If you have to create a LOOP clone, why not creating an ITERATE one? ITERATE is cleaner, and it would follow the Clojure trend of bringing out the most modern features of Lisp.

http://common-lisp.net/project/iterate/doc/Don_0027t-Loop-It...

> significant white space

Nit: significant indentation. Mostlanguageshavesignificantwhitespace, somemorethanothers (for instance, at least in 1.8, Ruby seems to have more whitespace issues than Python)

The fun alternative language example is of course Fortran, where at least early versions disregarded whitespace.

  6. I would like to see more FP constructs natively available in Python.

FP constructs such as? Map is there. Reduce is one import away. There is nice syntax for list comprehensions. What do you miss?

<cite>2. iPython was fun to use, helped out a lot, but it's not SLIME.</cite>

what exactly were you thinking about?

Point 6, we are lucky to keep the ones we have, Guido is not keen on map, reduce, etc.

While Guido is not keen on reduce, I don't think he has any desire to eliminate list comprehensions (which are equivalent to map/filter).

In other words, list comprehensions are the Pythonic version of same.

and Generator Expressions[1], a lazy version of list comprehensions [1]: http://www.python.org/dev/peps/pep-0289/

Yeah, but generators have been done much better. Look at Icon, Lua, or Prolog.

I wonder why. I never really use map over list comps, but I've used reduce in several situations where a comprehension wouldn't have been appropriate.