Looking for new LuaJIT maintainers
freelists.org
freelists.org
On the negative side, Mike Pall is an impressive individual who has contributed a lot to the field of JIT-compiled languages, so losing him is obviously bad for the project. LuaJIT is also a hard codebase to get into, I myself tried to get into it and gave up (I blogged about it last month [1]). There are probably still parts of it that only Mike understands, I guess we will see how much in the next months.
On the positive side, this will forcefully increase the bus factor of the project and take it out of this situation. And maybe we will see it catch up with Lua 5.3 :)
[1] http://blog.separateconcerns.com/2015-06-18-love-hate-luajit...
For those that think that LuaJIT's 5.1 compatibility isn't too far behind 5.3, be aware that Lua changes a lot in its minor versions.
To give you an idea, 5.2 introduced changed how scope works (both some stuff with lexical/global scope, and by redoing environments), introduced the `goto` statement, finalisers, and changed how `break` works.
5.3 introduced integers (Lua was previously like Javascript with only one Number type which was backed by a `double`), and utf8 support.
This means that upgrading a Lua version can be a big deal and break all of the programs written in it. For instance, redis and WoW both still embed 5.1.
Adding support for 5.3's integer subtype is probably going to be a very large project, I would guess that annoyed Mike.
People say that LuaJIT's source is difficult to understand, but I've seen source for several VMs with JIT compilers and this one certainly isn't the gnarliest; it's decently well commented. It only took me 30 min to add a new builtin function, with almost no understanding of the codebase beyond skimming a couple header files. As long as you don't want the JIT compiler to output asm for your language extensions, it's almost friendly.
Currently, LuaJIT relies on NAN tagging which allows it to represent values in 64 bits.
Increasing TValues to 128 bits would be very undesirable, but since luajit already supports 64 bit ints through the ffi library I don't see why they couldn't be stored as boxed udatas/cdatas in the same way.
"The main LuaJIT author (Mike Pall) is working on unrelated projects and cannot accept new sponsorships at this time..." [1]
Since then, Mike has only made bugfix commits to the project.
I really hope that the features mentioned on that Open Sponsorships page start happening after these news, and CloudFlare seems like a good candidate to me: even without knowing how much resources they're ready to offer, I'm willing to bet it will be an improvement over the current situation.
Everyone was surprised when LuaJIT seized the position of "fastest scripting language in the west", and it would be awesome if they could manage to keep that status - or has it been taken over by Javascript already? JS implementations have been improving dramatically thanks to the competition between browsers, and V8/SpiderMonkey are implementing features from LuaJIT's TODO list right now (I'm thinking about vectorized SIMD instructions). A lot of people use benchmarks as a major factor for choosing a scripting language, and I'd rather see more projects using Lua/LuaJIT rather than JS as a scripting language. It will be interesting to see how this turns out. Go LuaJIT!
"fastest scripting language in the west"
"fastest scripting language in the east"
But then again - Lua originated from Brazil and heavily used in game companies in the U.S. Not sure how much they use LuaJIT though.
I remember reading what he had researched so far a couple years back and being really excited, but it seems like the work was never really continued. I'd be interested to know if there was some fatal flaw that would've kept the collector from living up to its potential? or was it just a matter of the sizable effort required to write a new collector, more-or-less from scratch?
BTW, previous HN discussion of the new GC: https://news.ycombinator.com/item?id=4053969
[1] http://www.freelists.org/post/luajit/how-to-read-the-source-... [2] http://www.freelists.org/post/luajit/how-to-read-the-source-... [3] http://www.freelists.org/post/luajit/how-to-read-the-source-...
http://hhvm.com/blog/431/on-garbage-collection
and some other stuff from page 6 forward: http://hhvm.com/blog/page/6
1. Boot, setting up system wide resources like DB/memcached connections
2. Start doing work
Then any process forked after step #1 is sharing those sockets and can accidentally corrupt the work that the parent process is doing. With threads, you can share the connections through a pool that can be simply mediated within the process.
Yeah, you can design around it. But with threads, you don't have to.
Here's another: serialisation. If you want to parallelise any processing that builds up any complex data structures, you have to serialise them between your parent/child processes which can make it hard to keep the "richness" of the objects like interdependencies between processing runs. In addition, now your processing bottleneck becomes how fast your single-threaded parent process can deserialise the responses (which in Python in particular can really be quite slow).
And not everything can be serialised. It's not trivial to share sockets with other processes that have already been started. You can't easily accept a connection in one process and pass it to another to do the work and send the response (it's possible, just not as easy as passing an integer around like you can do between threads).
Yes, but:
> for 1) if you can put your sockets 'close on exec' no?
Yes, if you're the one that opened the socket. If you're using a library that opens that socket for you (e.g. a memcached or postgres library in my example), you don't necessarily have access to the socket itself. And now instead of children corrupting the parent's socket, you just have children with closed sockets. So you still have to solve the inherited-resource-sharing problem, FD_CLOEXEC isn't magic fairy dust.
> 2) how about using shared memory?
Possible but has its own downsides (you can't lock with regular posix mutexes, you have to make sure that the memory you're sharing is all in the shared block so you can't just use malloc, which means that you can't share objects that you don't control the allocation of, like, say, libraries' objects).
Yes, these are solveable problems. There are lots of other ones, these are just easy examples. Like I said:
> Yeah, you can design around it. But with threads, you don't have to.
For example, if you've figured out a small increase in reading a value from a table, you have sped up all other data structures, OO code, etc., because the table is used for everything.