LuaJIT 2.1 Profiler released | Hacker News Reader

LuaJIT 2.1 Profiler released

permalink.gmane.org

72 pointsqwertzlcoatl20 comments

LuaJIT is a work of great beauty and efficiency. I've been working on a commercial product for small ARM boards where rapid prototyping has been essential because of ever changing requirements but the speed of the finished code is also paramount. LuaJIT has made this project an absolute pleasure. It wraps C libraries cleanly, it runs within 5-10% of native code speed on the ARM (I checked because I didn't believe it at first) and it's 100% compatible with all the Lua 5.1 supporting libraries I need. This new profiler will be a great addition to the toolbox along with ZeroBrane Studio's debugger which I just discovered this week and which also rocks.

Mike Pall needs several Jolt Awards.

That is no lie. I like lua, but it never seemed to be the best fit for the problems that I need to solve. Nevertheless, if I could pick any one person to work on my language/runtimes of choice, it would be Mike Pall.

Now does Mike have any kids? Start pointing them in the direction of Rust, por favor :)

Now I'm only waiting for ZeroBrane to integrate the profiler into the IDE and LUA beats the rest by far. I'll try to integrate the profiler and debugging API into the lua derived potion and p2 as well.

The most interesting conclusions of Mike Pall reflect my experiences: sampling profilers are often much more usable in practice than instrumentation:

As you might have noticed, I had to change my plans compared to the original approach presented in June. The main problem with the instrumenting profiler was finding high-precision and high-speed timing sources for all platforms. (...) The necessary pipeline flushes shadowed the actual timings up to the point where the measurements were less accurate than with a sampling profiler! Other platforms offered only inaccurate timing sources or none that are accessible from user mode. And to top it off, the instrumentation added considerable overhead. (...) I had to scrap that work and decided to go with a sampling profiler.

And I don't know any other scripting language with a built-in sampling profiler. Does anybody?

python

Python doesn't have a built-in profiler – it has various add-on profilers [1].

[1] http://docs.python.org/3/library/profile.html

V8?

Smalltalk? Lisp?

Sampling for CPU performance, instrumentation for control flow analysis and memory allocation.

The distortion effect of measurement is higher with instrumentation, and is the chief reason I prefer sampling when trying to find performance problems.

But finding CPU performance issues is far from the only reason for using a profiler.

There are built-in sampling profilers in Chrome and Firefox (though in the latter case the UI for the profiler is an addon), if that counts...

I have to whole-heartedly agree with Mike Pall and acqq's assessment that sampling profilers are vastly superior to instrumenting profilers when it comes to measuring true performance. Instrumenting is only reliable in languages that are so slow that instrumentation has no substantial effect on performance – instrumenting is fine, e.g. for Ruby or Matlab, but not fine for C or Fortran (or LuaJIT).

Also, Julia has a built-in sampling profiler (built by Tim Holy):

http://docs.julialang.org/en/latest/stdlib/profile/

Hard to say whether Julia qualifies as a "scripting language" though – that's a pretty meaningless term these days.

Great news, up until now there was no good Lua profiler.

You can thank GIANTS Software for this one: http://luajit.org/sponsors.html

GIANTS Software GmbH is sponsoring the development of a low-overhead profiling functionality for LuaJIT 2.1, starting in June 2013. GIANTS Software develops a variety of simulation games for desktop, mobile and consoles. These games make extensive use of Lua for scripting and modding. Switching to LuaJIT was instrumental in reducing the CPU load and sustaining the required frame rates on all platforms.

Existing profilers for Lua and LuaJIT are based on Lua hooks and debug queries. The use of these generic mechanisms incurs a high overhead. Execution of a program under control of such a profiler causes substantial slow-downs. Actual use of the program (gameplay) may be impossible in some cases.

The goal is to design and implement a new profiling functionality that has a much lower overhead, better control of detail and high flexibility.

It kind of astounds me how many previous sponsors are anonymous. (Well, sponsored features, I should say. I guess it could be the same one anonymous sponsor.) I'd be interested to know, just in general terms, what kind of real-world situation(s) there are where the downside to being public about this is greater than the goodwill/respect they would get for doing it. Maybe they do not want to be known as a company willing to sponsor development that they don't "own"? Could there be legal ramifications?

Most of the sponsors who wished to remain anonymous didn't want their competition to know they use LuaJIT. :-)

Aha, that makes sense. Well, good luck to them on keeping their "secret" weapon under wraps :-)

In this day of multiple megabyte applications, it's amazing to me how much Mike Pall is able to fit into ~300 kB.

Mike Pall is without doubt one of the most talented developers of our generation, alongside people like Fabrice Bellard and Jeff Dean (to name a few).

Slightly off-topic, but does anyone know how I can download the complete mailing list archive data (of luajit)? Many of Mike's posts are very informative and useful, and I'd like to be able to search/read them easily, offline.

Either use NNTP, or http://gmane.org/export.php

When I grow up, I want to be Mike Pall.

This. Is. Awesome.

Reply on news.ycombinator.com