Lua Fun: A high-performance functional programming library designed for LuaJIT | Hacker News Reader

Lua Fun: A high-performance functional programming library designed for LuaJIT

github.com

102 pointssibman31 comments

Even though we occassionally talk about Lua on HN, it frankly still doesn't get enough attention - nor does Mike Pall and his fantastic work on LuaJIT.

Please considering sponsoring [1] LuaJIT so that Mike can continue working on it.

Lua Fun looks like a wonderful addition to the Lua ecosystem.

[1] http://luajit.org/sponsors.html

I can't agree with you enough.

> Even though we occassionally talk about Lua on HN, it frankly still doesn't get enough attention - nor does Mike Pall and his fantastic work on LuaJIT.

Because most HNers are web developers and care only about the browser.

Lua is really strong in the game developer community.

Most? I'd say far from most. I see just as many articles about Go, Scala, Clojure, etc. as I do Javascript.

How much of those are for native applications? The realm where Lua tends to be used most.

Yes, but usually in the context of web development as these are common server side languages for web apps.

pjmpl just meant we don't see a lot of game developer content on here, which is largely a C++ dominated area (with Lua game scripting becoming a lot more common).

> Go, Scala, Clojure

All of them languages which are mostly used for web.

openresty is getting a lot more attention with web developers. and I think its actually strongest in the "other" category but a lot is not very public.

It's quite interesting to see essencially the ideas used in stream fusion [1] to produce practically the same results produced in Haskell, avoiding allocations and often making it much easier for the compiter ro produce vectorised code. At least that's what it looks like to me, correct me if I'm wrong.

Edit: [3] has a good exlanation of how stream fusion works, and it should make it clearer why the linked code appears to be doing the same thing.

see also here[2] for a further paper on stream fusion being used to produce vectorised code. I've linked to the reddit comments because there's some useful discussion there too.

[1] http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.104.7...

[2] http://www.reddit.com/r/haskell/comments/1br0ls/haskell_beat...

[3] http://donsbot.wordpress.com/2008/06/04/haskell-as-fast-as-c...

In case you skipped the article it contains this nugget:

Take this example containing functional composition and higher-order functions:

reduce(operator.add, 0, map(function(x) return x^2 end, range(n)))

LUA converts this into 10 lines of assembler (2 jumps). Also, the LUA compiler is directly generating SSE.

Anyone have any good benchmarks comparing simple functional examples in LUA with Clojure or Common Lisp?

probably not. this is the first time I have seen the outcome of optimising this type of code. I knew it was possible as it optimises my tail recursion.

I know nothing about LUA syntax, so I also have no idea which part of this is new and which part is ordinary LUA.

Is -- calculate sum(x for x^2 in 1..n) a comment or is it the output of a code transform?

"--" are comments in Lua, also, sorry for being pedantic but its Lua not LUA. Its Portuguese for moon.

The code is perfectly standard plain Lua.

The point is not about expressiveness; it's that while this code composes several higher-order functions, it's compiled into the kind of assembly code you'd expect from a fully optimized C routine.

Underscore Lua is a lib I have been using for quite a while.

http://mirven.github.io/underscore.lua/

Looks cool, but hasn't been updated in four years.

mirven does seem to have gone quiet (on Github & Twitter).

Two more up-to-date forks can be found here....

- https://github.com/jtarchie/underscore-lua

- https://github.com/mark-otaris/underscore.lua

NB. The jtarchie fork seems best because its got 59 stars & 5 contributors.

There is a BIG difference between Lua Fun and Underscore. Lua Fun is based on iterators. High-order functions such as map, filter, etc. don't create temporary tables and don't allocate extra memory. Since all operations are performed on the fly during iteration, most expressions use constant amount of memory to evaluate.

I wish that Lua get more attention.

Agreed. Since I started using Lua as the scripting language for my (console) mail client I've had a lot of fun with it.

Apparently, this is part of an in-memory NoSQL database with Lua stored procedures, called Tarantool. Has anyone used that?

http://www.tarantool.org/

Is this similar to the functools package in Python? If so, that would certainly convince me to use Lua for some of what I use Python for.

Yeah, the library has some common functions with "itertools" Python's module [1]. fun.range syntax was even copied from Python's range. What is more, you can also take a look on Haskell's Data.List [2] or StandardML's List [3] packages.

[1] http://docs.python.org/2/library/itertools.html [2] http://hackage.haskell.org/package/base-4.6.0.1/docs/Data-Li... [3] http://www.standardml.org/Basis/list.html

I was just looking for that! Can it be used with the normal lua interpreter too? I want to use it on my löve game, so, no luaJIT.

> Reference Lua may not work and is not supported

Any particular reason for that?

It seems unlikely it doesn't work. But it won't get the performance.

The initial idea was to make a functional library that works best with tracing JIT compiler. Functional paradigm on tracing JIT is the core innovation in the project.

Lua 5.1 can run fun.lua after minor changes (I'll make patch soon), but I am not sure that performance will be good enough.

Do you think the performance is really going to be that bad in the regular Lua interpreter? I wonder if its going to be better or worse than the underscore.lua approach of using arrays instead of iterators.

Also, are you using lots of 5.1 specific stuff (setfenv, ...) or would this also be easily ported to 5.2?

The library code is Lua 5.1 compatible (except bitwise functions in "operator" submodule, but it can be easily replaced with "bit32" or even removed).

Some high-order functions heavily use recursion just to support multireturn iterators (yes, you can use multireturn with map, reduce, filter and so on). I plan to benchmark these parts and add optimized versions for non-JIT Lua if necessary.

I will try to make a patch in the next couple of days. Thanks for your interest!

it would be nice to have this or underscore library inside redis

according to antirez, redis does not (and will not) support LuaJIT. Even though at one point there was someone who had got it working. From experience I know that replacing Lua with LuaJIT, even in a statically compiled project, is quite easy. The reasoning is of course that lua code for redis shouldn't have to do a lot of processing, and thus won't benefit from the extra performance (it also shouldn't run long enough for the JIT to come into action).

Reply on news.ycombinator.com