Hacker News Reader

Hacker News

Comment by sanxiyn | Hacker News Reader

Parent Full thread

sanxiyn

Those who do not learn history are doomed to repeat it.

http://qinsb.blogspot.kr/2011/03/unladen-swallow-retrospecti...

View on HN

Those who do not learn history are doomed to repeat it.

I'd bet the Dropbox team is aware of unladen swallow and the issues it faced.

Applied too broadly, that quote would mean "never try anything where others have failed."

I can't speak to the non-technical reasons mentioned in the post, but there's good reason to believe that LLVM is in a very different state than the time that Reid wrote this, particularly wrt JIT support: the JIT engine has been completely replaced. Both of the things that he mentions (lack of back-patching, lack of gdb support) have been added to LLVM mainline.

LLVM's JIT support has changed a lot, but one thing didn't change: "LLVM code generation and optimization is good but expensive."

Even LLVM's "fast" code generator is slower than most JITs' code generators.

Definitely true; there are some overheads that could be low-hanging fruit to optimize, but the real solution is most likely to use a tiered-compilation system. ie only invoke the full LLVM code generator once a function has been called 10,000 times. It's definitely an open question as to how to get faster compilation times out of it; we have a simple LLVM interpreter but since LLVM isn't designed for interpretation it's pretty slow.

I think you can look at the people who are interested in adding LLVM tiers to their JITs (ex Apple, Facebook) to see that we're not the only ones that think there can be a place for "good but expensive" code generation in a JIT.

Like this?

https://github.com/dropbox/pyston#compilation-tiers

From Dropbox to the python-dev mailing list:

"(...). As for Unladen Swallow, there are some reasons to think that LLVM has matured greatly in the past few years, particularly in the JIT engine which has been completely replaced. I'm not sure if that's the only part to the story; I'd be interested in talking with any of the people who were involved or knowledgeable about the project."

It's my impression that Unladen Swallow failed because of insufficient manpower, not because of the general approach. It's true that LLVM is missing some features for a JIT like this (mainly OSR and back-patching), but those can be added. For example, there was a discussion a few months ago on llvm-dev about adding patching support [1] (it was originally meant for garbage collectors, but would also work for JITs).

1 - http://lists.cs.uiuc.edu/pipermail/llvmdev/2013-October/0665...

It is my impression that LLVM has matured quit a bit since unladen swallow. Not least in the JIT domain. A lot of unladen swallows development was spent on improving LLVM and not on implementing Python as fast a possible. I seem to recall that the LLVM developers have had discussions on how to better support JIT compilers in LLVM, since unladen swallow, some of it precisely because unladen swallow failed.

Guido works at Dropbox. I am sure he is advising - and I bet, he has seen pretty much all the previous attempts closely to let them make any obvious mistakes.

> Guido works at Dropbox. I am sure he is advising

Guido worked at Google during Unladen Swallow, he was not involved. From the comments of Pyston's announcements, here's the extent of GvR's involvement:

> Guido's advice has been extremely helpful, but so far we haven't been able to get any code from him :/

Reply on news.ycombinator.com