Hacker News Reader

Comment by drej | Hacker News Reader

Julia frustrates me. I was in a mathematical modelling sphere, so I learned the language basics years ago and immediately fell in love (multiple dispatch, optional types, broadcasting, ...). But I couldn't get approvals at work to push it, because it was immature. Then a month later, all my code broke. I rewrote it using new APIs. It broke again. I know they reserved the right to make breaking changes up until 1.0, but they are hindering adoption because of all this. Sorry, guys, but I'll have to stick to Python, Go and (sigh) Fortran.

That being said, I applaud getting good funding for a project that's actually more than beyond MVP, it has happy users/customers and it serves a purpose. That's not all that common these days. Good luck, guys.

View on HN

An alternative would be getting your bosses approve for you to use Common Lisp.

It is mature (30+ years in use, rock solid ANSI standard), many compilers available for many plataforms, highly portable code, multiple dispatch (CLOS is arguably the most powerful object oriented system available), you can also very easily call C libraries with CFFI and there is portable support for threading...

...and well written CL code will approach Fortran and C in speed, at least with SBCL which is a free, popular compiler.

You can also compile to the JVM and to the LLVM, and if you need pro support for it, there is LispWorks and Franz Inc. Ah, and with ABCL you can also easily call java libraries, if you need to.

Yes Common Lisp is freaking amazing, but it isn't as good of an out of the box experience as a lot of these products. How do you do plotting besides pushing to GNUPlot? The REPL is great, but can you specify types? Yes it can reach near C/Fortran speeds, but how much time will I spend optimizing it? Are the built-in linear algebra, optimization...etc libraries good, or must I toil with FFI? How much time is really saved over dealing with all that and learning SBCL, slime, emacs...etc? Data scientists just don't have the time.

I was assuming that the use case cited above by drej was doing a complex, big system requiring numerical computation/analysis. It was mentioned that Julia was rejected on grounds of being immature technology, so I assume we're talking about a big project, perhaps destined to run as a server or as a clustered system.

The use case you imply is different, it is more like a scientist working alone at a workstation for interactive calculations, in which case perhaps Python with Numpy and Scipy is just perfect.

In any case, for the previous use case (complex, big system), i'd use CL for the calculations themselves, for the visualizations I could do interactive web graphics using the extremely nice Bokeh library for Python. I'd use Bokeh under Python (using Flask as a web framework) for the presentation layer, and the Lisp system for the actual calculations.

BTW, CL does have many libraries for plotting: http://cliki.net/plotting

> Are the built-in linear algebra, optimization...etc libraries good, or must I toil with FFI?

You can use the GNU Scientific Library also within Lisp, to do everything you mention and more:

https://www.gnu.org/software/gsl/

https://common-lisp.net/project/gsll/

> Yes it can reach near C/Fortran speeds, but how much time will I spend optimizing it?

I guess you are aware that for maximizing speed in C you need to be very careful on how you write your code. In this sense there is no easy way out in any language. The good part is that Lisp, unlike C, is garbage collected, so working on a piece of code can be easier since there is no need to deal with pointers, allocation, and deallocation. It also supports arbitrary number precision and fractions and complex numbers out of the box, no need to do anything in particular to optimize their performance nor rely on external libs to operate on them. It is particularly good (speedwise) with big integers and arbitrary precision numbers.

As for Common Lisp itself, besides that observation -which applies to all languages-, you need to add type declarations. This improves the performance a lot. You are going to add type declarations on a statically typed language (C, C++, Julia, Java, etc) anyways, so i fail to see how this could take more time than in other languages.

Alright, so slightly different assumptions. You're talking about using CL for just one part of the process which is reasonable if not a little frustrating if one must switch between multiple different technologies.

Binding to GSL was kind of my point. That requires a lot more setup to use the FFI and bind what you need to CL. In Julia I imagine one either does A*b or loads a batteries included library first and then does that. My point being there is a fair amount of overhead to CL to get to that magical development spot :).

I think I've seen that plotting library in the clicki link, but it has a long way to go if you're used to Matplotlib, JS, or what is included in .NET.

The type declarations is another thing that the user would have to research and deal with. In Julia it is just part of what you do, so a bit more natural unless you can declare the types in standard CL without loading a library or something else odd. Can you just (define int(a) 3) and let the compiler guide you? I honestly don't know, but don't recall seeing that in the lisp tutorials and books I've read. My s-exps are probably wrong too lol.

The GSL library is already usable in CL, no need to add bindings, i included the link.

> you can declare the types in standard CL without loading a library or something else odd. Can you just (define int(a) 3) and let the compiler guide you?

Yes, and yes. No need to load anything and the declarations are simple as in "(declare (fixnum a b c))". The compiler does the rest. You can also specify ranges, etc.

"declare" is part of the ANSI standard.

> I think I've seen that plotting library in the clicki link, but it has a long way to go if you're used to Matplotlib, JS, or what is included in .NET.

There are many libs, not just one. But you might be correct, in which case i'd just have the visualization done in JS or .NET (etc), and just load the final for the plot through HTTP from the Lisp server (which is the one which did the heavy lifting.)

It is not difficult.

I didn't know GSL could be used right out of the box. That's good to know. The declarations part is pretty nifty too. I can't believe I didn't notice that before.

Perhaps easy for you and probably not super difficult for most people considering CL, but it's still a bigger pain than having 1 single tool that can build fast code with very little hand optimization, REPL, analysis libraries, built in plotting, and with a familiar syntax. Thanks for all the comments though. You pointed out parts of the CL ecosystem aren't nearly as bad as I thought. You should consider some blog posts if you've done this sort of thing before!

Thanks! I'll consider!!

Biggest changes were from 0.3->0.5. Even from 0.6->1.0 it's not expected to break nearly as much code as before.

It's easier to argue to use Julia now since the New York fed uses it for their full scale model of the US economy

My girlfriend does incredibly complex modeling (she's a computational geneticist). From my limited understanding, it sounds like you may have some of the same challenges that she does.

Do you think Python and Go are more suited to that domain than R or Julia?

R is very, very slow compared to all the other programming platforms (including Ruby, a nice language but with some of the slowest implementations out there).

R syntax is very unlike most programming languages so jumping to other languages can feel strange.

Python with Numpy, SciPy and Pandas, is the current widespread alternative to R and a good starting point. Recommended!!

For all the cases I worked on, R is not slow at all. At home, I wrote a R package to do deep learning (include most common layers, but only conv layer was implemented in C++) from scratch, and I also wrote one in python (using numpy + numba). My R version is same fast as python on the MNIST dataset (without conv layers, hence all code is in plain R or python).

What this means is that for your purposes R is fast enough. But in the absolute sense (or in any case, relative to other programming platforms), R is one of the slowest ever. R is usually slower than Python, and Python in some cases can be 100x slower than C.

Performance, assuming identical hardware, depends on the program itself (that is, on what you are trying to calculate or compute or perform), but if you choose many different programs you can have a comparison between platforms ("platform" in this case means combination of programming language plus compiler or interpreter.)

Here is a graphic comparing Julia to other platforms: https://julialang.org/benchmarks/

You can see that according to that graphic, R can be up to 400x slower than Julia (or C).

There is another independent benchmark here. It does not include R but it includes Python, and it's very interesting.

http://benchmarksgame.alioth.debian.org/

But the best comparison graphs are here, this is worth a look: http://blog.gmarceau.qc.ca/2009/05/speed-size-and-dependabil...

> Python in some cases can be 100x slower than C

Yeah, the Python implemented version is. But people doing serious computing in Python that requires speed are doing it with NumPy or even Cython or just straight up calling C/Fortran libraries in Python.

It's the same in R - you can have good performance of you use vectorized routines, because those routines are written in C/C++/Fortran.

Go isn't meant for computational science, but business code as services. Python is really good in this area when combined with Matplotlib, Numpy, & ScyPy.

Reply on news.ycombinator.com