An experimental Vim fork with multi-threading capabilities | Hacker News Reader

An experimental Vim fork with multi-threading capabilities

github.com

75 pointstarruda33 comments

  - syntax highlight
  - decent automatic indenting
  - semantic auto completion popup
  - on the fly compile
  - on the fly syntastic check(lint tool)

In vim, those settings are very sluggish in my old AMD 2 core 1Ghz cpu laptop. Sublime Text and Emacs both are quite reasonable.

I think vim could be more responsive.

This slow response problem is only reason for trying other editor. Please make this real.

Wanting the "engine" of emacs but the interface conventions of vim, I've been using emacs with Evil (https://gitorious.org/evil/pages/Home) a bit lately, and quite liking the combination. It implements a fairly good portion of vim, enough to cover just about everything I regularly use, which was a pleasant surprise (I'm used to "vim compatibility" layers usually being pretty shallow).

I don't use the last four of the features you're mentioning, but I can confirm indeed that syntax highlighting can make things terribly slow with large files and/or complex languages (I'm a Perl 6 guy), so yeah if multi-threading can help, I hope it gets implemented soon.

The patch for this on vim-dev has been completely ignored for other a month by all vim developers, so don't get any hope this will be accepted.

1 month is not a long of time. Specially when you are waiting for core developers of a busy project. Besides that this month had several holidays. Maybe you just need some faith.

Depending on faith will probably lead to bad decisions, a rational appraisal of available evidence and some critical thinking will probably be more useful. And let me present some evidence that may destroy any remnant of that faith you have:

https://groups.google.com/forum/#!topic/vim_dev/-4pqDJfHCsM%...

That's our async/timers patch, we pretty much gave up for now because debugging vim was a nightmare. Right now something is stomping over regexp objects in memory. We have no idea where. We've found unsafe pointer use, etc. Vim code is pretty much all globals, 25k lines and 400+ ifdefs for at least one file, some crazy file encoding, pre C89 C that still compiles on an amiga, mixed tabs and spaces. The vim code base is pretty much the worst code I've ever seen and Bram doesn't seem to really grok our patch. He's also been demanding with his requests that we feel are over the top. That's fine, that's his choice. We're focusing on other plugins right now, we put in many months of work into vim. It kinda works as long as you don't use leader keys.

Thanks for the link to an interesting discussion. While I can't speak to the quality (or otherwise) of the vim code, I wouldn't characterize that conversation as "demanding" or "over the top". Even the most recent patch is reported to cause hangs when doing something as simple as pressing the leader key. You can't rationally expect a maintainer of mature software to sign up for that. I doubt Debian or other downstreams would be happy to hear that the vim maintainer had signed up for that.

If the fork is maintained for a year or so, fixing problems with the timer code and feeding patches for actual vim bugs back upstream, and adding tests for all of that, at the end of that time this patch will be quite different. It might even be in a form the vim maintainer can merge in good conscience. As an added benefit, by that time some compelling plugins that take advantage of these timers might be written. As it stands now, the use-case for this patch is rather hypothetical. Indeed, if after a year there aren't a few plugins around that make good use of this functionality, it calls this entire effort into question.

> As it stands now, the use-case for this patch is rather hypothetical.

I agreed with you up to this point. As it stands, many utility plugins (like CtrlP and Syntastic) and bundles/ftplugins (vim-python, my own vim-pantondoc) would be improved by this functionality. It is obviously wanted/needed by the Floobits plugin, which motivated the creation of the patches proposed last year. So, I don't think the effort can be reasonably put into question, as you say, even today. People do need async in vim, and also people would want upstream to handle this ASAP, by deciding on some strategy to provide it.

This said, this new message queue API seems cleaner than Floobits', but it seems to cater to python interop only, which is a mayor downside, since many vim plugin users opt out on python or simply lack python support.

Actually, python interop was only a for quickly testing multi-threaded plugins.

See the issue tracker for examples of vimscript functions that can be added on top of the message queue: https://github.com/tarruda/vim/issues/2

Thanks a lot, didn't see those. A bit more documentation would be really helpful ;) Otherwise, great work!

...many utility plugins (like CtrlP and Syntastic) and bundles/ftplugins (vim-python, my own vim-pantondoc) would be improved by this functionality.

Thanks for the correction; clearly I mischaracterized the use-case.

> If the fork is maintained for a year or so, fixing problems with the timer code and feeding patches for actual vim bugs back upstream, and adding tests for all of that, at the end of that time this patch will be quite different.

There are few problems with the timer code. Vim's code is a monster. As you yourself admit, you don't know the vim code, so how could you possibly judge how demanding the requests are or make any of these other claims.

> The use case is hypothetical

We're building a business on real time editor plugins that isn't hypothetical at all.

I have to agree with the peer poster. That thread only shows a level of attention to detail that is required when producing software that's intended to behave reliably.

Bear in mind that drive-by patches have to be maintained forever by the person who is reviewing them. It's often easier to not respond at all, or to implement the work yourself, than to explain all the nuances around correctness and maintainability that may not be obvious to a new contributor.

Some projects are more accepting of patches, but they also then have to accept the of patches to fix those patches, and patches to fix the patches that fixed the patches.

Other projects try to get things right the first time so that there isn't a second time.

I wish more people understood this.

Perhaps with enough pressure from the community there's a chance this will happen?

Meanwhile I will keep maintaining the fork up-to-date with upstream, so plugins can make use of the patch through a simple 'if has("messagequeue")' conditional

Hmmm. So the benefit of this is offloading plugin processing onto a separate thread(s)? I guess I see the theoretical benefit of that, but I'm unsure of how useful this actually is...

Every time you launch a process from inside vim (like grepping multiple files, generating tags for a large project, etc) it totally freezes the user interface. You can not execute any long task in the background, which is really annoying. So having a multi threaded Vim is very much overdue.

There is an ongoing discussion on the vim-dev mailing list regarding a proposed patch by Thiago Arruda [1]. I have no opinion on the technical merit of the patch, but as a user, I would warmly welcome a move in that direction.

[1] https://groups.google.com/forum/#!topic/vim_dev/65jjGqS1_VQ

GUI apps should be multi-threaded. But, process handling is OS task. Unix already provides facility for backgrounding of processes.

:!long-process&

I agree, but how do you get notified that 'long-process' finished running?

Even if you poll using CursorHold-like hacks the user still needs to type something.

:!{ long-process ; notify-send "done" ; } &

All that does is display a popup on the desktop, it does nothing to notify/callback to the vimscript code that started the process

Yes. For many things that's what you want. I wasn't trying to say "this will meet all needs and make this entire discussion irrelevant"; I was posting lest anyone find it interesting/useful.

I think there are some use cases, like getting the syntax highlighting being processed by vim in another thread, that would help the responsiveness (or feeling of) if this was supported natively.

Wouldn't a simple fix be writing a little stub that acts like the Tee command line utility? It would immediately hand back to Vim and pipe onwards the command you sent it.

Could be a temporary kludge at least until OP gets going.

I am actually using vim-dispatch[1] that works fairly well.

[1] https://github.com/tpope/vim-dispatch

Ever had to sit and wait for vim to parse a gigantic file to get syntax highlighting correct? Just waiting for it to know the line numbers can take a while.

One thing I run into a lot is that sometimes tab completion scanning ctags locks up the whole process for several seconds.

You haven't tried some of the heavier auto-complete plugins. As it stands right now, they have an annoying tendency to block the UI while you're typing! It's bothered me so much that I've even considered writing my own vi clone based entirely on an asynchronous message queue such as this!

ever tried youcompleteme? the redraw bothers me, but that's a different issue

http://valloric.github.io/YouCompleteMe/

The redraw issue is also related to vim not being thread-safe. With this fork youcompleteme can probably do something like 'vim.defer("YcmTriggerSyntastic")' from a background thread to invoke syntastic and redraw the screen.

An indirect attempt to troll for side-effects and hilarious race conditions perhaps?

Is this a bi like vimproc[1]?

1: https://github.com/Shougo/vimproc.vim

My understanding is that Vimproc can only kick off async processes, it can't actually call back into the main Vim thread once a process has completed.

Reply on news.ycombinator.com