Hacker News Reader

Comment by qwerty456127 | Hacker News Reader

Is there some lisp that has offers nearly as powerful and fast data processing and tensor algebra facilities like Pandas and Numpy of Python? I would love to use a lisp but am not sure how well are they going to manage with huge 2-3D arrays.

View on HN

Yes there is a really great statistics functional programming language that was founded on being a scheme for data science. Unfortunately the data scientist back then were not training in doing code well so the examples of the language got cluttered up but there is a great functional Scheme right in there.

That language is R. http://adv-r.had.co.nz/Functional-programming.html

I learned Racket (Lisp) and then realized I had so many thing inside of R that I was missing. Hadley Wickham's tidyverse and funtional Advance R book changed everything in my code.

Currently, this comment is heavily downvoted, which I do not understand. If Julia warrants mentioning as having a lispy core, then it’s at least as valid to point out R’s deeply lisp-inspired (and technical implementation based) heritage.

I mean, scratch the surface on any of the FFI docs, and you’ll see that SEXPRs are right there lurking barely beneath the surface: http://dirk.eddelbuettel.com/code/rcpp.html

EDIT: and you can, as the parent comment says, do much worse than Hadley’s functional programming guide as an introduction.

> and you’ll see that SEXPRs are right there lurking barely beneath the surface

Then why the surface? Why can't we just have a lisp with a good data-science library? Scheme seems the easiest of all the programming languages to me. I bet I can teach Scheme (not including advanced FP stuff though) to anybody faster than any other language although I have never used it to code anything serious (because libraries), I actually find it simpler than Python or even VBA. Phenomenally everybody seems so scared by the parentheses but these are not a serious problem given an intelligent IDE with proper outlining and highlighting.

You’re preaching to the choir here.

I’d love to see a data science community around Racket in particular. In fact, you can do quite a lot in Racket today, just with not nearly the libraries and tooling as Python or R.

But as a consolation prize, R is pretty Schemey, and has the best collection of libraries for data around. It’s been a dream for years to figure out a way of making use of more of R from Racket, but time and deadlines keep getting in the way.

I searched and I can't find it but someone was trying to implement R inside of Racket.

Personally I really want a language just for cleaning data in Racket first and then tidyverse.

Have you seen this package yet? http://alex-hhh.github.io/2018/08/racket-data-frame-package....

I think it could be the starting point for some data science work in Racket.

Thank you so for finding that. It is a totally different project but further along then the one I was following a few years ago.

Because they made the decision to be S-compatible, as many statisticians used S+ at the time. At that point in time, the real lispers all used X-Lisp Stat: https://en.wikipedia.org/wiki/XLispStat (in fact, some people objected to the journal of statistical software on the basis that it would only cover XLispStat).

Most programmers, esp from ALGOL-like and C-like languages, hate the syntax. Decades have been wasted trying to convince them to overlook it. Clojure was a surprising exception. Whereas many languages that stayed close to languages people were familiar with got adoption.

Better to hide the LISP'y core underneath a syntax similar to a widely deployed language. Julia was a brilliant example that's getting a lot of uptake vs scientific LISP's as predicted.

The parent asked for a Lisp dialect. R is not a Lisp dialect. It's ridiculous to claim there's a "Scheme" in R. R is not Scheme, it's not even close.

It’s much closer than people think. R originally built on a Scheme runtime, and the DNA of Scheme is still lurking pretty close to the surface.

The meta programming facilities in R still look very lispy, and are used to good effect throughout the Tidyverse family of packages.

If I had one wish, it would be to see something like the Tidyverse exist in something still closer to Scheme... like Racket.

If it's all Scheme under the hood, why can't it be backported easily?

This is valid R

  sum(1:10)

This is also valid R:

  (sum (1:10))

Just wrapping whatever in parentheses does not make a lisp.

There are multiple Clojure options for high-speed matrix processing. E.g.

    https://neanderthal.uncomplicate.org/
    https://kieranbrowne.com/research/clojure-tensorflow-interop/
    https://mxnet.incubator.apache.org/api/clojure/index.html

Its dead now but there was http://lush.sourceforge.net/

Julia can show s-expressions for an expression I think...sorta what you asked, but not really I know.

If you really want something like pandas + lisp then you probably need Common Lisp and have to dig for some libraries.

I'm a great admirer of Julia, but I doubt it would appeal to those seeking a "true" lisp/scheme. Even if it's has a sane macro system, and much of the same power. You can indeed see s-expressions (search for "s-expression") :

https://docs.julialang.org/en/v1/manual/metaprogramming/inde...

This nice talk mentions a bit on how Julia is (not) a lisp (if I recall correctly):

"Julia: to Lisp or not to Lisp?": https://m.youtube.com/watch?v=dK3zRXhrFZY

This might also be of interest: https://tpapp.github.io/post/common-lisp-to-julia/

As well as (maybe) clasp: https://github.com/clasp-developers/clasp

You could always just use Hy, which is a clojuresque language that runs on top of python and can use its libraries.

Hy is awesome, it's a pleasure to use and you can always convert is straight to readable Python to debug. Access to regular Python libraries is seamless.

[0] https://learnxinyminutes.com/docs/hy/

[1] http://docs.hylang.org/en/stable/quickstart.html

Ok. Perhaps that's the language. Which IDE/editor has the most intelligent support for it? There seems to be no Hy plugin for PyCharm (e.g. IntelliJ Idea supports Clojure nicely AFAIK).

Hy has a nice Emacs plug-in (hy-mode) which I use. Other than that I'm aware of Vim and Atom support. I'm not really sure how intelligent they are though.

Reply on news.ycombinator.com