JIT Compilation for Emacs Lisp | Hacker News Reader

JIT Compilation for Emacs Lisp

tromey.com

180 pointsdeng50 comments

It's fantastic that people want to tackle the complexity that is Emacs Lisp and work on a JIT for it. It might not be the most fashionable thing these days, but a faster Emacs would make life significantly better for countless developers.

I am having high hopes for this project.

Completely agree

Also add to this the trend or rise language servers .. emacs can continue to be used as a solid ide alternatives

And .. guile emacs hopefully one day

Any work to improve and re-use emacs is good for everyone

> And .. guile emacs hopefully one day

I'm hoping for a Common Lisp emacs, myself. Scheme as specified just isn't a big enough language, and if you pick one of the better implementations (e.g. Racket, which is awesome), then it's not really Scheme anymore (e.g. Racket, which is really its own language by now).

I would still call it Scheme, just not R*RS Scheme.

I hate to admit it, but the slowness of Emacs is why I avoid it. A good JIT which hopefully speed it up would be amazing.

ELISP is a pretty fast language, faster than VimScript or JS for sure.

EDIT: Also, Emacs is bloated is bloated because it's not just a editor its's an Web Browser, Email Client, News Reader, Music Player and File Manager. If people want a light emacs, they can go use uEmacs or become the last JEDi.

It's a good glue language, like Python. Better than Python in the sense that its internal representation is based on tagged values instead of ubiquitous pointers: for example, 43243 represents itself and isn't a pointer to a some reference-counted PyObject representing an int.

And because Emacs Lisp a lisp, you get extensive syntactic malleability that you can use to build DSLs and convenience facilities.

Elisp is actually a very nice little language.

Your absence from emacs-devel is felt, there was so much excitement when you were around.

“Fast” is really a meaningless and vague word. What specific part of emacs lisp is faster than javascript in what regard?

For what it’s worth, V8 among with some other JITs are some of the fastest engines in the world: https://arewefastyet.com/#machine=29&view=breakdown&suite=as...

> V8 among with some other JITs are some of the fastest engines in the world

I find that doubtful, when the jvm is so close to native code in terms of performance, and mozilla said that wasm (or was it asm.js? Forget which) was still 50% slower theoretically than native code.

> still 50% slower theoretically than native code

How can something be theoretically slower? What's the difference between theoretically slower and practically slower?

Only practical implementations have a speed that you can talk about.

I don't know what you mean by 'theoretically' as an adverb there.

I suspect it means something like "according to calculations using a noticeably approximate model of reality". That's what people tend to mean, anyway. "Theoretically" is often code for "My model does not correspond to reality".

It annoys me, actually, that "theory" has this negative connotation of noncorrespondance.

Hotspot among with other JVMs are no doubt fast -- but partially that came from Java being a static and compiled language.

Not saying that JavaScript is faster, but JavaScript is fast. If you look at the benchmarks I posted, you'll notice that 50% slower is hardly the norm -- many benchmarks have the VMs performing at a near-native performance, and the VMs are even faster in some cases.

In fact, number crunching in JavaScript has been fast for a long time, and even faster than Java in some cases: http://www.stefankrause.net/wp/?p=144

I'm not trying to advocate JavaScript's performance in anyway however. I'm merely pointing out the fact that performance cannot be single-handedly described as "fast" or "slow", especially "for sure". Context matters. A lot.

Not being an emacs user, can you compile all that out to slim emacs down?

You can, but it's probably not worth the effort. Most of that's in the form of lazy-loaded byte code. Other than some lightweight stubs for the entry points, Emacs doesn't load those pieces until you use them.

Which use cases are slow for you ? you used it on large code base or parsed cpp source ?

A quicker emacs would be neat, but most of the time emacs gets speed from flexibility (it might be slow but I can automate something) and design decision (magit for instance blows 99% of the fastest git interfaces)

ps: one part that realize is that I don't like emacs daemon, and even curated my setup on windows (probably the culprit) takes a few seconds. It's not slow but often I want a buffer right away so I use notepad u_u; stupid I know.

Also worth mentioning that whatever is slow, profile it to find out why. Sometimes only the smallest tweak is necessary to get it up to speed again. (Different tweaks for different configurations, so can't be a default.)

For interactive use, profiling is as simple as M-x profiler-start, doing what causes the slowdown, and then M-x profiler-report.

Good point, emacs has a lot of tooling (edebug and tracing too) but it's rarely demonstrated, and since emacs is so large..

In my case there's nothing slow but IO, I should try on another windows laptop with a faster SSD (mine is low grade) to see what is what.

With an Emacs server/daemon, it loads faster than Vim for me.

I would happily agree but I lost the one emacs and 119422 buffers mindset. I get lost in my own shit. I should spend time improving my ibuffer grouping but even then.. I like the idea of pop, edit, kill (which causes my harm).

If you bind crux-kill-other-buffers[1] to a key, you might end up with a workflow you're happy with.

[1] https://github.com/bbatsov/crux/blob/master/crux.el#L496

My solution: Accept that there will be dozens or even hundreds of buffers and use helm-for-files (which lists recent buffers first) to switch between the buffers I care about at any given time. Works really well.

I think I need some fancier categorization like day columns, and content tags. But I'll keep your suggestion in mind

I've noticed that pasting large amounts of text (> 1000 lines) can make it grind to a halt for several minutes. I don't know if it's due to the mode I'm in.

More than the number of lines, it's the number of characters on a single line that causes the slowdown. Thankfully I don't have a real usage scenario where I handle thousand character line files.

I can edit multi-thousand line (5k+) Org documents just fine. So it's probably the major mode or some minor mode(s), or a combination.

In my experience that's often due to major modes that try to format and syntax-highlight on the fly. Keep in mind that some forms of pasting are really equivalent (from Emacs' PoV) to a crapton of keystrokes being entered.

Anyway, the workaround I have for the occasional time I have huge things to paste is to switch the buffer to fundamental-mode before pasting, and then switching back.

As a curiosity, what kind of things are you pasting that are that big?

Few editable systems are happy with 1K lines. I remember a discussion where only one editor was happy dealing with that, all other became sluggish.

1k lines does not stress many editors. It only stresses emacs when the major mode is doing lots of stuff to highlight and format.

I regularly used other editors on C files with 30k+ lines, 15 years ago when machines were slower.

Emacs' real weakness is an over-reliance on regexes for highlighting and poor integration with sources of semantic and symbolic information for richer highlighting, editing and browsing. That, and synchronous operations for the same.

> 1k lines does not stress many editors.

but folks here are talking about _each_ line >1k characters, not total number of lines in a file...

it's true that the syntactic system is too ad-hoc for heavy duty

If you are dealing with large text files of JSON or XML of 1MB or larger you can turn the buffer into Fundamental mode and turn off line numbers.

I’ve found that the reason why large files are slow is due to formatting and linting and not the size of bytes on disk.

With fundamental mode and no line numbers it feels as fast as Vim or Sublime to me.

Long lines are slow, but it handles well otherwise. For long lines, I disabled bidi-display-reordering and switched to fundamental-mode and it really helps.

Honestly, I don't much notice emacs being slow, esp. given how much it's doing for me under the hood.

One key thing is to always always always use emacs in dæmon mode, rather than starting up a new emacs for each file (as would be done with vi). With an always-running emacs instance, start-up time is no longer a factor, and the editor itself is plenty fast enough once it's running.

I use a nox emacs with minimum .emacs for quick editing in terminal. Its startup time is comparable to vi.

I don't find emacs slow, and I spend most of my day in it. What is annoying, is that when it blocks, all frames and windows block.

Maybe this will improve in Emacs 26: https://www.reddit.com/r/emacs/comments/7ta67n/is_emacs_mult...

Cool! Speeding up Emacs is always welcome.

I will bring the obligatory guile Emacs rant:

I am very much biased,but guile Emacs is very much a viable option IMO. Guile3.0 will be natively compiled (or at least jitted), and the work Andy Wingo is putting into guile is just amazing. The guile codebase is nice, and most of the hard work integrating it into Emacs is already done.

The elisp runtime has some optimization work left, but it seems to be quite a lot of low hanging fruit.

Guile is a very nice scheme implementation for those who are brave enough to leave the warmth of elisp.

I'm torn. On the one hand, I definitely want emacs to be faster as it does more and more for me. However, I also want more and more of the things that folks are doing in emacs to be driven by other processes. Preferrably ones I can use outside of emacs to get answers.

As an example, I get that emacs would have to parse the entire buffer I'm in in order to syntax highlight as I'm typing. For that, I think this is going to be required. However, emacs should be able to offload to another process to look up external symbols. GLOBAL is the tool I use nowdays, and updating the tags database or looking up symbols is something that should be getting heavy lifting in an external process.

Or am I looking at things the wrong way?

IIRC tags databases are big files that need to be parsed and operated on via Elisp, so a faster interpreter would be beneficial there. I don't think a faster interpreter would hinder integration of external tools, given often they do not only provide speed, but also quality software with proven longevity.

Traditional ones, yes. However, modern tag databases are just that, databases. Sending a query to a database to find all callers and such can be offloaded to another process easily enough.

Note that this isn't really that new, for the modern tag. Cscope was already an external command. And you already offloaded the tag creation to an external process. I see very little need for the tag navigation to all be in elisp.

The final selection, of course, can still be in elisp. Since that will be user input driven.

Emacs now has C extension support, which might be helpful in fast interactions with external tools too.

> GLOBAL is the tool I use nowdays, and updating the tags database or looking up symbols is something that should be getting heavy lifting in an external process.

What do you use for GLOBAL? ggtags.el works like you describe (calls start-file-process to start a GLOBAL process).

Apologies, I meant that to be an example of how I think it should be. Not a hypothetical, either. Like you say, you can already write things this way.

My only complaint is it still blocks sometimes. Overall, works like a champ.

This is the area that lsp in general is trying to address. Most of the problems with elisp isn't its speed, imo.

> However, I also want more and more of the things that folks are doing in emacs to be driven by other processes. Preferrably ones I can use outside of emacs to get answers.

… there's life outside of emacs? grin

I'd rather do stuff in-process, but with good support for asynchronicity. It's nice that I can M-. to the definition of anything I'm using, nice that I can patch it in-memory and nice that it's all written in a decent language (elisp isn't perfect, but it's pretty good).

I agree. I think language servers are the way to go, ideally leveraging the same lexer and parser as a production compiler / interpreter to avoid mismatches.

I'm also torn on that idea. I think multiple parsers and such are healthy for the ecosystem. Mainly because I am heavily swayed by the dangers of monocultures.

So, to that end, I think it is healthy that there are at least 3 powerful javascript implementations. I just wish there was more equalization in the marketplace for them.

And, I cannot argue against the merits of a language server. Having a single one would be more efficient in many ways. I just find our fixation on "not reinventing the wheel" to be detrimental in many ways. I'd rather we were "don't get stuck with a monoculture."

And I say this as someone that highly respects the LISP community. It is impressive how much you can do with really old libraries there.

> Even better than this would be to improve the calling convention so that all calls are less expensive. However, because a function can be redefined with different arguments, it is tricky to see how to do this efficiently.

Clojure has a compiler option toggle for this called direct linking. It just says that functions can't be redefined (unless they are marked redef using a metadata key). You can also mix and match libraries compiled with or without direct linking, for example the core library uses this compile switch.

What happened to Guile? It seemed like a promising way forward that didn't stray too far from elisp.

I am also hoping for guile Emacs, but I am pretty sure I am hoping in vain. The amount of bikeshedding going on in emacs-devel even for small non-controversial changes can sometimes take humorous proportions.

I have used guile-emacs and back then even org-mode ran fine. Elisp-compatibility is very close to 100%, and the runtime is getting better by the week (guile3 will do native compilation!), but nothing will ever be enough.

So instead of having a runtime not bound to Emacs, with all the cons that would have with libraries and such (Imagine a guile-orgmode that contained the vital parts of org scirptable in guile without Emacs), we are stuck with a language tethered to an editor bound to be just an extension language.

Reply on news.ycombinator.com