Comment by dmlorenzetti | Hacker News Reader

Comment by dmlorenzetti | Hacker News Reader

I wonder if Wolfram's 100-times-slower reference point was based on a very particular comparison. Suppose you benchmarked a numerical algorithm that dealt with a lot of matrix-vector operations. And suppose you compared, on the one hand, a C or Fortran implementation that used arrays and native machine finite-precision numbers, against a Lisp implementation that used lists and the sort of "exact" arithmetic described in the usenet post.

This would not, of course, be a good comparison between the languages, but it might well give you a 100x speed difference. Taking Pitman's story at face value, it sounds like Wolfram wasn't very sophisticated about these things, and wasn't open to listening to explications. So he might have extrapolated a single dumb benchmark into a universal truth.

At the same time, it sounds like Pitman tried to convince Wolfram to represent floats as the ratio of two bignums, "with lots of other hidden bits to assure that any decimalization had enough bits to be precise." I can understand Wolfram not particularly wanting to taste any of that pie. I have worked with a simulation platform that uses paired BigNums just for time values (other values in the simulation use regular double-precision variables), and it was a big drain on simulation speed.

I could also see a matrix representation as lists of lists, and when you do anything you're always consing, and replacing elements of lists, and walking lists to get to elements. Compare that to a C-style 2D array of float or double, and Lisp looks horrible - plausibly 100x as bad, and worse as the size of the matrix increases.

Of course, the problem with that is the Lisp data representation used, not Lisp itself.

Except Lisps that are intended for numerical work, like MACLISP on PDP-10s at the time, did have arrays. And as noted, MACLISP was around then faster than DEC's FORTRAN (an issue DEC fixed not too much later).

Lispers aren't stupid, which should be distinguished from how easy it is make a simple Lisp. Making a performant one takes effort on the scale of making any similar language implication good and fast.

I never said that Lisp didn't have arrays. I said that representing a matrix as a list of lists would not perform well.

What I meant to be saying is that Wolfram plausibly may have looked at a bad data representation for matrices on Lisp, and concluded that Lisp was inherently 100x as slow.

Either that, or a rough estimation of the problems he expected to face moving forward -- this is a decades-old company, an amazing outlier in this industry -- made him think this would always be a relevant bottleneck.

We're able to use higher-level languages because they've developed the trick of identifying inner loops and hard coding them in C (this is how we get Torch and train big-ass neural networks in Lua).

I think it's also important to note that Macsyma (or atleast Maxima) has a lot of cruft in the code. I actually like Weyl better, even if it hasn't seen active development in ~2 decades.

P.S: Maxima actually uses a list-of-lists to represent a matrix.

The author mentioned that the Lisp in question and Fortran could operate at the same speed. My impression is that the big use of Fortran has always been numerical algorithms with a bunch of matrix-vector operations — is that not the case? Because if it was like against like it doesn't make any sense, but if Lisp were as fast at matrix operations I figure we still wouldn't be using LAPACK for that stuff.

Matrix-vector operations are part of BLAS, and most performant BLAS packages (e.g. MKL and OpenBLAS) are written in assembly or machine-generated using a special-purpose codegen. You can't write the gemv/gemm kernel in any general-purpose language and expect to beat modern BLAS. But you can call BLAS/LAPACK from whatever language you want. This is how MATLAB succeeds even though it has a dog-slow interpreter.

Reply on news.ycombinator.com