Hacker News Reader

Comment by epscylonb | Hacker News Reader

In my experience overuse of meta programming is usually a sign that the architecture of the system is bad and the software should be split up into separate libraries or web services.

View on HN

That all depends on the language. Lisp (pretty much all of its dialects) heavily emphasizes the use of macros to extend the language and build all kinds of control structures and embedded domain-specific languages (EDSLs).

Absolutely. One could make the argument, though, that many (including me) believe that expressive power and conceptual beauty is Lisp's core strength, not code maintainability or runtime performance.

expressive power and conceptual beauty is Lisp's core strength, not code maintainability or runtime performance.

Well, Arc is actually very maintainable, and quite fast. After all, HN is powered by Arc, and it serves >100k daily uniques on a single core.

Yep! Not saying that it's impossible, just that some of the concepts that Lisp is based on make these things harder to pull off. The reason for this is that Lisp emphasizes flexibility, and the more flexible your runtime environment is, the more ways to do something, which makes it easier to have hard-to-maintain code (and harder to reason about performance, too). But it's not impossible to do with Lisp for sure.

Lisp macros also provide ways to take ugly, hand-optimized code and clean it up to provide a cleaner, friendlier interface. Like all macros, this comes at no runtime cost (since all macros have been fully expanded by this point).

Actually, it's not any harder to write maintainable code in Arc. In many ways, it's much easier, because you write your program as a set of parts which fit together like an arch.

HN is fast because it is feature-barren. That's a design choice pg is free to make (and cutting features is a great way to improve performance), but not a proof of Arc/Lisp's ability to support complex applications with high-performance.

Which features would convince you that Arc is fast?

Just for the record, let's count to 100 million. And add the numbers up while we're at it.

  ~ $ time (echo '(do (= n 0) (for i 1 100000000 (++ n i)) prn.n (quit))' | arc)
  Welcome to Racket v5.3.5.1.
  Use (quit) to quit, (tl) to return here after an interrupt.
  arc> 5000000050000000
  
  real	0m28.561s
  user	0m28.308s
  sys	0m0.249s
  ----
  ~ $ time (echo -e 'n=0 \ni=0 \nwhile (i <= 100000000): \n  n += i \n  i += 1 \n\nprint(n)\n' | python3.3)
  5000000050000000

  real	0m30.244s
  user	0m30.230s
  sys	0m0.013s

It would appear to be competitive with Python on my machine on this particular task. (Also you can make it faster by dropping into Racket.)

A while loop and temporary mutable variables are definitely not the Pythonic way of doing this. More idiomatic:

    $ time python -c 'print sum(xrange(100000000 + 1))'
    5000000050000000
    
    real    0m1.398s
    user    0m1.383s
    sys     0m0.012s

Comparison to baseline:

    $ time (echo -e 'n=0 \ni=0 \nwhile (i <= 100000000): \n  n += i \n  i += 1 \n\nprint(n)\n' | python)
    5000000050000000
    
    real    0m33.140s
    user    0m32.939s
    sys     0m0.023s

I see. That is good to know; I merely chose mutating global variables in a loop because I knew how to do that in both languages. (I am not very familiar with Python.) That is not the idiomatic way to do it in Arc, either. I would normally use a recursive function, like this:

  arc> (time:xloop (i 0 n 0) (if (> i 100000000) n (next (+ i 1) (+ n i))))
  time: 9121 cpu: 9130 gc: 0 mem: 480  ; the times are in msec
  5000000050000000

Or perhaps a "higher-order function":

  arc> (time:sum idfn 1 100000000)
  time: 19889 cpu: 19908 gc: 0 mem: 1224
  5000000050000000

Or use a deforestation macro that I wrote, which is closest to your Python example:

  arc> (time:thunkify:reduce + (range 1 100000000))
  time: 17971 cpu: 17985 gc: 0 mem: 3592
  5000000050000000

Also, here's what you can get by dropping into Racket:

  arc> (time:$:let loop ((i 0) (n 0)) (if (> i 100000000) n (loop (+ i 1) (+ n i))))
  time: 402 cpu: 403 gc: 0 mem: 920
  5000000050000000

I suppose Python has an analogue of that--dropping into C, or at least loading C libraries. Which Racket can do too. Mmm.

Common Lisp was designed for development of large/complex applications ('applications' really means here software that is used in production) with good performance.

For example this CAD system from PTC is based on 6+ million lines of Common Lisp code and under development for two decades:

http://www.ptc.com/product/creo/direct

Lisp macros are a lot simpler and easier to understand than what goes on with the types of metaprogramming done in ruby/rails. Expanding a macro is a lot easier for me to wrap my head around than stuff like dynamically defining methods when something is subclassed or when magic happens in method_missing.

At the same time the Lisp community is one of the fiercest in supporting the idea that nothing that can be a function should be a macro. Metaprogramming just because it is clever is more abhorred in the Lisp community than any other I've seen.

Of course! Macros are distinctly less powerful than functions for one reason: they are not first class values at runtime and thus cannot be passed to higher-order functions.

This is solved by using an operative lisp - fexprs (or 'vau'/'$vau' in kernel and wat) are non-applicative combiners that execute at runtime, so you can have a first class combiner that chooses when/how/how many times to evaluate its arguments.

I've been having a lot of fun recently writing code in wat-pl (my perl port of Manuel Simonyi's wat-js interpreter)

Yep, and it surely depends upon the problem being solved too.

The heavy emphasis of macros in Lisp may also be its greatest weakness. I see all sorts of Clojure code that uses macros where they needn't have, and it tremendously impedes my ability to use the code in ways the library author may not have expected. I love the way petehunt put it - metaprogramming is a last resort, you should feel bad when you use it, and language designers should include it but deemphasize it. (Scala and Haskell nail it I think; Clojure perhaps emphasizes macros a bit too much which harms the output of intermediate programmers. But, Clojure is not designed for intermediate programmers.)

Reply on news.ycombinator.com