Common Lisp as a Scripting Language, 2015 Edition | Hacker News Reader

Common Lisp as a Scripting Language, 2015 Edition

fare.livejournal.com

110 pointswglb40 comments

I have my own scheme-esque language that I use for scripts. 16 of its 48 primitive procedures ~correspond to unix system calls (open, close, read, write, seek, select, stat, link, unlink, mkdir, lsdir, chdir, fork+exec, wait, sleep, exit). It was very instructive to work on, and using something I know inside out is a welcome relief from the uncertainty I feel when intuiting (ba)sh.

Any reason you don't use one of the existing scriptable scheme like guile's?

Wanted to put my article-gleaned understanding of lisp to the test + get at the essence of what shebang-style unix scripting was about.

Here is a reimplementation of "cat" in it:

    #!/usr/bin/env imp

    (def copy-out (proc (x)
      (let f (if (file? x) x (open x))
           bytes (read default-chunk f)
           (if bytes
               (seq (write bytes stdout)
                    (recur f))
               (close f)))))

    (if (nil? script-args)
        (copy-out stdin)
        (each copy-out script-args))

Running external programs is a bit more verbose than normal shell, but there is some optional reader-macro sugar on top of fork+exec, e.g. the effect of the following is that /usr/bin/say gets run with those non-string args converted to strings and a voice comes out of my speaker.

    !(say hello there)
    => ((status : 0) (pid : 2945) (in : [File]) (out : [File]) (err : [File]))

> Wanted to put my article-gleaned understanding of lisp to the test

fair enough!

> get at the essence of what shebang-style unix scripting was about

FWIW, although convoluted, you can do that with guile, with:

    #!/usr/bin/guile \
    -e main -s

Or simpler with chicken:

  #!/usr/bin/csi -s

IMHO the nicest Scheme for UNIX scripting is scsh. Its APIs are really well thought out and represent a lot of design work that hasn't ever been replicated by another Scheme (at least that I've seen).

Some good examples of scsh scripting (some by the author, Olin Shivers) can be found at

http://scsh.net/resources/scripts.html

The manual is also very good:

http://scsh.net/docu/html/man.html

> The manual is also very good

The acknowledgements section made my morning, thanks.

Do you know what is the status of scsh? It's an old project, seemingly unmaintained. Is it considered feature complete? Are there any active users of scsh? What about bugs?

Last time I checked a package for scsh for Fedora was somehow broken and/or conflicting with Chicken and I went with the latter. In general I like the idea and APIs of scsh, I'm just afraid that there will be no one to ask for help if I hit some problems. With Chicken or Racket the communities are active and it's easy to find support. How does this look in case of scsh?

Have you released the source for it?

Nah, because given I've never even tried most of the mature alternatives then it would be a faux pas to sell other people on why they should use it.

Perhaps. But even if it sucks: if it does at least one thing right, the other mature alternatives can learn another trick.

I'd wager scsh would provide a similar experience.

Don't say it's better than the others then, just say "here's my version, what do you think of it ?".

You will always learn something by releasing it.

I'm starting to use SBCL for scripting, and on the system with a Lisp on an SSD drive the startup delay is unnoticeable. My friends tell me that on non-SSD setups, adding --script to startup options makes SBCL load almost instantly. It skips various initializations, disables debugger and REPL, and ignores shebang if it's the first line of a script, allowing you to write:

#!/usr/bin/sbcl --script

in your .lisp file.

I used Common Lisp from the mid 1980s until about 5 years ago when my customers wanted Clojure, if they wanted Lisp development done. I have never seriously used CL as a scripting language but for a while I was using Gambit-C Scheme for small compiled apps and command line tools. Gambit-C is very good for this purpose.

BTW, Clojure is "out" for command line tools because of the JVM startup time but I have thought about Clojurescript + node.

Pixie[1] is also worth a look. It's Clojure-like scripting language written in RPython.

[1] https://github.com/pixie-lang/pixie

Indeed pretty cool, clojure implemented in rpython!

For even more fun, checkout hy (https://github.com/hylang/hy). It's completely compatible with standard python both ways (ie you can import python from hy and import hy from python), so you can import python-sh and do awesome stuff like:

  (import [sh [cat grep wc]])
  (->
   (cat "/usr/share/dict/words")
   (grep "-E" "^hy")
   (wc "-l"))

It isn't clojure compatible, it is like clojure.

And there is http://scsh.net/

There is also scsh (scheme shell - http://scsh.net/) although not being maintained anymore.

There's a plan to do some work on it (including trying to port it to 64 bit) this summer.

An updated version by Roderic Morris can be installed as a library atop the latest version of Scheme 48. It's on github: https://github.com/scheme/scsh

I think it's 64-bit (?). Most of the time it hasn't mattered for me, at least when writing scripts.

You can also just build the "old" version (0.6.7) with the `-m32` GCC flag, and it works fine on 64-bit Linux and OS X. It's very stable in my experience, and works fine.

Considered Racket?

I use Kawa Scheme on the JVM for scripting on Windows. Powershell is forbidden where I work. For some reason, it seems to start up faster than Clojure.

Nitpick, but Clojure's startup time isn't JVM startup time it's clojure.core startup time

Ah, that is why Kawa Scheme starts up faster then.

This appears to be true, eg:

  cat <<eof > Hello.java
  public class Hello {
    public static void main(String[] args) {
      System.out.println("Hello, world!"); }}
  eof

  javac Hello.java
  echo "Main-Class: Hello" > manifest.mf
  jar cvfm hello.jar manifest.mf Hello.class
  time java -jar hello.jar 
  Hello, world!

  real    0m0.153s
  user    0m0.132s
  sys     0m0.020s

vs:

  lein new hello
  cd hello
  # Pesty s-expressions, so hard to work with... ;-)
  cat <<eof|patch -p0
  --- project.clj 2015-04-16 04:13:09.734160136 +0200
  +++ project.clj.patch   2015-04-16 04:09:33.809504079 +0200
  @@ -3,4 +3,5 @@
     :url "http://example.com/FIXME"
     :license {:name "Eclipse Public License"
               :url "http://www.eclipse.org/legal/epl-v10.html"}
  +  :main hello.core
     :dependencies [[org.clojure/clojure "1.6.0"]])
  eof

  cat<<eof > src/hello/core.clj
  (ns hello.core
    (:gen-class))
  (defn -main 
    "I don't do a whole lot."
    []
    (println "Hello, World!"))
  eof

  lein uberjar
  
  time java -jar target/hello-0.1.0-SNAPSHOT-standalone.jar

Hello, World!

  real    0m1.060s
  user    0m1.400s
  sys     0m0.040s

(Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz, SSD drive)

Kawa:

  wget ftp://ftp.gnu.org/pub/gnu/kawa/kawa-2.0.jar
  wget ftp://ftp.gnu.org/pub/gnu/kawa/kawa-2.0.jar.sig
  gpg --recv-key D269E756
  gpg --verify kawa-2.0.jar.sig

  echo '(format #t "Hello, world!\n")' > hello.scm
  # Not sure how to bundle up a scheme script with kawa
  # to a stand-alone jar...
  java -jar kawa-2.0.jar --main -C hello.scm 
  time java -cp kawa-2.0.jar:. hello
  Hello, world!

  real    0m0.284s
  user    0m0.296s
  sys     0m0.020s

And, for the record, with the same hello.scm, and guile (2.0.5 as found in Debian stable):

  # First run:
  time guile hello.scm 
  ;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
  (...)
  Hello, world!

  real    0m0.066s
  user    0m0.044s
  sys     0m0.020s

  #Ed: with precompiled code under $HOME/.cache/guile (?)
  time guile hello.scm 
  Hello, world!

  real    0m0.040s
  user    0m0.036s
  sys     0m0.004s

Lisp on my Raspberry Pi is faster:

    pi@raspberrypi ~/lisp/ccl $ time ./armcl -n -Q -e '(progn (princ "hello") (quit))'
    hello

    real	0m0.112s
    user	0m0.020s
    sys	        0m0.080s

And the expression is first compiled to native code...

How does guile 2.0 do on that hardware?

Don't know. I don't use it.

Is clojure any "better" in real-world use ?

I use the #! trick for scripts. Here's a Common Lisp script that sends me email every morning (run from cron) about AWS resources being used:

https://gist.github.com/franzinc/acde9fd04a6b07745db3

I use SBCL for some scripting, and haven't found the overhead to be that bad just using #!/usr/bin/sbcl. For basic stuff, I just use zsh, so by the time I've switched to Lisp, the task is already at the point where startup time isn't a big factor.

I do have some compiled programs I use, but IMO they're a little too big to be called scripts.

An interesting idea would be the ability to dump a shared library of the core image, and then dump incremental changes as executable images that dynamically link to it.

I wonder if the scales would tilt more in favor of separate images if you added a good tree-shaking step. You can bang out useful scripts pretty fast by pulling in a few helper functions from large libraries, but the more scripts share a library, the less functionality you can discard from the final image.

If you can cut the size of the core language's image in half with just a simple method like this, you could probably get impressive results using more sophisticated methods on images requiring a lot of libraries: https://gist.github.com/burtonsamograd/f08f561264ff94391300

Fellow Lispers may perhaps also find helpful these notes I put together recently on shebang lines for various open-source Common Lisp implementations:

http://ar.to/notes/common-lisp#scripting-with-common-lisp

I have used Chicken Scheme for a lot of script-like tasks.

If one doesn't want to compile first, that's fine. The interpreter's startup time is negligible.

"the EliteBook has a touchpad that is particularly bad, even worse than the Thinkpad's in being triggered all the time by my thumb as I type"

I know what he means. These designers never learn...

Anyone else feel like we could get along without the "'clit' interface" jokes?

Reply on news.ycombinator.com