Lush, an object-oriented lisp for large-scale numerical and graphic applications | Hacker News Reader

Lush, an object-oriented lisp for large-scale numerical and graphic applications

lush.sourceforge.net

50 pointsb-man20 comments

One thing to note is that the authors are foremost very well known machine learning researchers. Reading Bottou's PhD thesis (in French unfortunately for most HN readers) was kind of depressing in a way, when you see the kind of things he did 20 years ago in an environment which seemed more powerful than what we have today. This environment was not lush I think, but it certainly was the inspiration for i: some info in Englisg at http://leon.bottou.org/projects/lush

I've hacked up one or two tiny things in Lush.

To me, the important thing about Lush is not that it's object-oriented, but that it's oriented toward high-performance numerical computation.

It has array types with aggregate operations (like J, K, R, NumPy, PDL, PV-WAVE IDL, etc.) even for interpreted code, plus bindings to all the standard numerical libraries; and it has a decent compiler and supports inline C.

I really like this language. The core libraries have sensibly named functions, getting it to generate or talk to C is trivial, and it's simple to get a GUI or a small script or something going. It has good bindings to SDL, which makes it easy to do some game programming. The REPL has a great online help facility, and interactive reflection is simple, so there's little head-scratching. I used it years back, when my main exposure to Lisp had been CL, and it was refreshingly convenient.

The only problems I have with it are the minor one of the OO syntax (minor as I never made much use of the OO features) and the somewhat more major issue of the dynamic scoping (no closures). It was a deal-breaker for me but was pretty educational, as I had not quite understood dynamic versus lexical scoping at the time.

Except those issues, it's a great Lisp, especially if you need one that plays nicely with the OS and is easy to learn. (And it is pretty speedy to boot, even respectably speedy interpreted.)

(Edited, removed a bit where I repeated myself.)

Has anyone compared this with Incanter, another lisp-based (clojure) stats environment?

http://incanter.org/

Both look like activity has died out recently... This is a shame, I was thinking recently about this issue. It seems that there is a real need for a stats environment based around a straight-forward general purpose programming language that can be easily parallelized and can easily access GPU functions. Although GPU functionality is still pretty narrow, it's growing, and linear algebra seems to have a place in statistical computing.

I have not yet seen anything that fits this bill, but Incanter is the closest. Clojure has the Calx library for OpenCL. Last I asked around, nobody had put the peanut butter in the chocolate just yet.

My Kaggle partner was trying out Incanter and gave up due to performance reasons. Apparently, when Incanter loads a data set, it loads every cell as a hash as opposed to loading rows or columns as vectors. Correct me if I'm wrong - he would love to use it.

I am curious about why you think numpy/scipy does not fit that bill ? Python is a straightforward general purpose programming language, has a lot of stats functionalities, can access GPU functions. While python's story for parallel programming is pretty poor, it is actually quite decent for numerical computation, because parallelizing those is not very difficult in general.

I suppose it's that last part. I tried using some of the ipython parallel stuff, but found it to be pretty clumsy (although this was about 2y ago). Is there a better solution for parallelization in python?

I have never used the parallel stuff in ipython, but my understanding was that it was more for cluster-kind of stuff. This is getting significantly rewritten, also, so things are changing there.

But the way most people use parallel stuff in numpy is: first use BLAS/LAPACK which is multithreaded, then use multiprocessing, etc... It depends on what you are trying to achieve, but my understanding is that as soon as you use arrays and the liked, clojure is not that helpful compared to a much more primitive runtime as python, because all the parallel goodies of clojure are not available anymore. But I have never seriously used clojure, so I may be dead wrong.

Thanks, I'll try that approach.

Don't hesitate to ask on the ML (while I know numpy fairly well inside out, I am not the best person to talk about parallelism). There is also a lot of talks available in the scipy conferences video. See http://conference.scipy.org/scipy2010/schedule.html for 2010.

LUSH is still fairly active. It moves a little slowly, but this is good since it's stable and a lot more mature than Clojure. The authors are still using it for real projects, so it's likely to stay around.

incanter is somewhat slow. I still use it prefferably to R since I already know clojure, but don't use R enough to be comfortable with it. I certainly wouldn't use it to do real-time machine vision stuff, which is one of the use-cases for Lush.

I really wanted to play with Lush, but it seems to have broken with Mac OS 10.6 -- which is why I ended up going with Clojure.

They're not really comparable. From the article:

no automagic memory management, no garbage collection, no functional programming

You're absolutely right -- but it was the decision I had to make.

You left out an important bit of context. Lush is garbage collected.

  Lush brings the best of both worlds by wrapping three
  languages into one: (1) a weakly-typed, garbage-
  collected, dynamically scoped, interpreted language with
  a simple Lisp-like syntax, (2) a strongly-typed,
  lexically-scoped compiled language that uses the same
  Lisp-like syntax, and (3) the C language, which can be
  freely mixed with Lush code within a single program, even
  within a single function.

The thing is that Lush the interpreted lisp like language is quite different from CLush the compiled lisp (un)like language. This is indeed an unexpected quirk and gets people who are starting at it. The two, share the same syntax but are quite different languages. CLush can almost be described as C with lisp syntax. What you are referring to is CLush. Lush on the other hand is fully garbage collected. In fact it used to have a reference counting based garbage collector but the developers were looking at other garbage collection algorithms, mainly Boehms garbage collector. I have not used it in a while so I am not sure if it has been baked in. Something I was looking forward to in the new version was parallel loops. There is a stub for that, but it defaulted to the sequential loops in the version I played with. I think a reference counting garbage collector makes parallelism hard to implement, Cpython is perhaps an example of that, whereas Sisal is a counterexample.

Its a fun language to use, it is so much more easier to interface with C (compared to MATLAB, well even Python/Numpy/Scipy sans weave). But there are a few surprises if you expect Lisp.

EDIT: Replying to gaius here to save on deep nests.

Only if you want to distribute it as a compiled binary, (may be to protect your source). You can distribute the source code as is, and it will run just fine. As far as distributing something self contained, I agree that Lush is not tailor made for that. Because even the successfully compiled parts are typically loaded dynamically.

Right, but in Lisp you can use the bottom-up approach to iteratively and interactively build your program, then save image or whatever to get a binary. If your Lush prototype relies on some feature that can't be compiled, you will need to reimplement that before going into production.

Lush 2.0.1 has specific bug fixes for issues with OS X 10.6.

(no automagic memory management, no garbage collection, no functional programming).

That's a big drawback.

Common Lisp (SBCL) is very fast already, and one can avoid the GC overhead in a number of interesting situations (though not all). Are there benchmarks comparing SBCL and Lush?

Has anyone found a sane way of doing maths (linear algebra, etc.) in Common Lisp? My experience with the GNU scientific library has been disastrous.

Reply on news.ycombinator.com