Bram Moolenaar responds to Neovim
groups.google.com
groups.google.com
To Diego: Prove him wrong. I suspect that Bram has been maintaining the code base for so long, that he might need a bit more than a mere message to warm him up to the idea of a refactoring to the scale that you're undertaking. Also, you might not be the first person to contact him with such a proposal. I wonder how long it took for others before they gave up. Build something tangible and contact him again later with proof of concept, something that would make this more than just a fleeting dream.
To people questioning Bram's position: You need to look a bit further than your nose. The fact that you want to create plugins or extensions does not put you in the mainstream as a Vim user. It's an extremely popular editor of almost religious proportion, with lots of existing plugins and extensions. So we know that people use it and making plugins is possible. With all of its annoyances it gets the majority of the job done for most users. One can't just wake up one day and say that they'll change things, not even Bram at this point.
The most desirable outcome in my opinion would be to have Neovim be a continuation of Vim (Vim 10.0?) rather than just another fork, and for that, having Bram on board would be a boon. His longevity as a maintainer is impressive.
Is that reason enough to exist? Maybe. But what's the long-term aim here? Without a strong point of differentiation to attract users and thus developers NeoVim will be left forever playing catchup with Bram's original creation -- constantly rewriting new features to remain relevant and forever fixing both old vim bugs and new NeoVim-specific bugs.
This is shallow and short-sided opinion. If neovim achieves goal of "easier to maintain codebase" then it will have much faster feature development, bug fixing, and extension development. All extremely user facing and exciting.
I think this is a bit naive. There's no plan of any kind laid out for NeoVim, not even a goal against which progress can be concretely measured. New and compelling features must be well thought-out, well designed and well executed; they don't just fall out of the sky as a consequence of "easier to maintain code".
People who basically want him to never change (and that is fine) can be happy with mainline. There are others who would like to see some significant updates, particularly allowing better threading support.
https://groups.google.com/forum/#!topic/vim_dev/-4pqDJfHCsM%...
Your response is to tell the Internet that Vim is wrong for not accepting broken patches?
It came down to a fundamental design decision—and after months of hard work, they got a non-answer.
"There are enough vim users out there (myself among them) to warrant supporting a fork for as long as we're around. We'll try to keep it up to date with the latest changes."
I don't think anyone should expect Bram to chase them down for updates to the patch. Reading through the discussion, it seems clear to me that while good-natured, the feature was not ready to be merged into VIM.
I think the bigger problem was just that nobody could agree on the cancel/freezing timer issue.
To me that's a design decision a maintainer has to make, not someone submitting a patch.
Each iteration of the patch was provided with a whine about whether it was ready for inclusion yet ... when the reality is, that as the patch author, it's your job to make sure there aren't more bugs and the patch is ready for inclusion.
Every time someone found another issue was a red flag to any sane maintainer that your patch wasn't ready for inclusion. Every time you whined about having to iterate, you made it clear that you couldn't be trusted to have written something stable and maintainable.
If you can't deal with working on mature software, stick to startup code.
Sounds to me like you need to go farther. "We should improve what we have" totally fails as a criticism when written by the same guy who won't let the product in question be improved.
If there's good reasons not to admit patches, fine, let's hear them, but don't give me this glib "we should improve what we have" crap. I can hardly believe that's basically his only answer.
http://www.joelonsoftware.com/articles/fog0000000069.html
I realize Neovim is being characterized as an "aggressive refactor," but I'm sure that's the way Netscape thought of it as well.
tl;dr Things you should never do: rewrite the code from scratch.
And who knows--37Signals might well have reused some of the innards.
Fantastic game though, thanks :)
I can clearly see the benefit of this old wisdom under context of limited resource and limited time, esp. in a commercial company.
However, as an experimental, personal(?), non-profit, free/open-source project, I do think neovim has value in its own right:
1. It won't affect any existing plan of vim, and won't change any existing code in vim. We can still happily use vim as usually. 2. If it succeeded, then we would have another powerful free/open source editor in our community. 3. If it failed, then we as a community have nothing to lose. The author might be unhappy then, but he would at least gain lots of experience.
There are so many rewritings happen in this community, and they are definitely good things:
- svn is a rewriting of cvs, and git is a totally new one (with more features?) written from scratch. - nginx was written even if there's apache httpd - There's zookeeper, but people wrote etcd - etc.
In an OSS context, money is not always the biggest mover, and doing this might actually succeed, or at least not fail because the company fails. It certainly may be that the people lose interest before it gets anywhere, but who cares! The only reason that I could see is that someone wants to fold all their code changes back into vim's original codebase, and they both seem to have different end goals, so eventually they will drift away from each other.
Code evolution.
Neovim is being sold as a VIM refactor, but apparently with no new functionality
Very different from Linux (had an aim of being a playground for new features), tmux is different from screen as Clang is different from GCC, Subversion is a totally different beast from CVS
Where did you get that idea? When I read the project page (https://github.com/neovim/neovim), two of the explicit goals are:
"Enable the implementation of new/modern user interfaces without any modifications to the core source.
"Improve the extensibility power with a new plugin architecture based on coprocesses. Plugins will be written in any programming language without any explicit support from the editor. "
It is exactly designed to be a playground for new features, but with the goal of them not having to be in the core.
"There's a new project that is called Neovim that seeks to refactor and modernize the codebase."
"There's a new project that is called Neovim that seeks to refactor and modernize the codebase"
Someone should try to make some money replacing tabs with spaces and snake_case with camelCase.
It's basically trying to improve over vim, still using command/insertion modes, inverting the commands (by using movement/action instead of action/movement)… It's pretty configurable and the codebase is new.
definitely worth a try!
I believe tmux / screen is a beautiful example, given Screen has a lot of black magic and has a very challenging codebase to work with. A lot of the philosophy built into tmux are cumulative learnings deduced from what was done correctly or incorrectly within screen. This materializes as features missing (or implemented with different intent) from tmux that exist in screen -- the philosophy behind sessions is one such big underlying principle change.
I like to think of this as similar to using a genetic algorithm to evolve an overly complex/convoluted solution over a period of time (based on some assumptions which may no longer even be applicable), and then explicitly defining/reconstructing a comparable solution using more deterministic and intentional approaches. The focus of these two development cycles are fundamentally different and perhaps neither wrong (the former attempts to discover novel ways of addressing a problem whose solution is unknown, the latter prunes this known search space).
If it gets subsumed into Vim I'll use it without having to choose.
If it gets traction as a second thing, I'll try it.
If it's better than Vim I'll keep it.
Let them write, good luck, have fun, I'll try it if I hear from HN that it's good enough to use.
I don't think it violates "don't rewrite" because these aren't the Vim people. Vim itself won't "suffer" from the ill effects of a rewrite.
Considering vim itself is a from-scratch 'vi' clone, that's interesting advice. :)
But in any case, I see this as more of a fork than a rewrite. It's more akin to the XEmacs/GNU Emacs situation, where someone submits significant patches upstream which aren't accepted, and then forks their own version to continue that line of development.
The main difference is that a clone is often written by developers who had no involvement with the development of the original system. But that's not always true, especially for any old or sizable software system, where the original developers may no longer be involved.
A rewrite may start with a clone, but the code is no longer a clone when you start the rewrite.
A refactor is a subset of a rewrite.
Now we can have meaningful discussions about them.
Rewriting is a significant risk, and rewrites often fail because people don't appreciate that fact. But there are ways to rewrite successfully, and often doing so is the correct course of action.
A big problem with rewrites is that it puts a strain on development resources, and it's not possible to maintain both old and new code at the same time. Another big problem is prematurely committing to the rewrite, before it has proven itself. When the inevitable happens and schedules slip, defects arise, and features need to be cut this can prove disastrous for a rewrite that you've foolishly committed to in advance. Also, people often ignore the value of years of productive use which provides testing and validation of immense value.
If you have the resources often the best choice is to spin off a new project for a rewrite and have them compete in the open. With luck the rewrite could become mature enough to overtake the original (as has happened with many projects). Alternately, work in phases doing refactoring in steps, perhaps with new features behind feature flags.
Many people think that rewriting code is easier than green field development, in reality it's often more difficult. It takes a great deal of skill, and often luck, to manage a good rewrite.
This trend is particularly bad with those who've gotten involved with software development since 2005 or so.
I chalk it up to a lack of experience with massive, multi-decade software projects. If all a developer has worked with are small, recently-written Ruby on Rails apps, for example, then I don't think they can appreciate situations where rewrites of some degree are the only viable option.
Hopefully this attitude regarding Spolsky's writing will change with time, as these people encounter the real-world situations where significant or total rewrites are basically the only possible option.
For me there's a useful sort of arrogance, the "I could code that in two weeks even if I don't know the language" kind of arrogance. I say useful, because that had gotten me to jump into projects that I never would have dared touch.
But the dangerous side of that arrogance is, "I know enough to say what what these other people are doing is stupid". Every time I learn a new sort of software or a new domain, I am reminded that it's much easier to make bold pronouncements from ignorance than from knowledge.
So it's more like taking the old netscape code and making gecko.
That first commit also shows it is not a rewrite, though, since it has a comment starting with "Import vim from changeset v5628:c9cad40b4181"
These guys don't have these pressures, nor did (as per haberman's comment) Linux, tmux, Clang, WebKit, etc as far as I'm aware.
Sentiment like this is not valuable. The experience that it comes from is often informative, but as a conclusion, this explicitly ignores circumstances that sometimes do make rewriting a good, compelling course of action.
Truly bad, unmaintainable code exists. It exists on projects that have long intended futures. It exists in all sorts of environments on all sorts of projects for all sorts of reasons. To discount the idea of a rewrite out of hand is a failure to objectively review the specific circumstances around whatever it is you're working on.
Often there are better ways, and yes, often a rewrite is a bad idea. But absolutes are so rarely valid, and this one is most certainly not universally true.
Don't really think so. At least as far Netscape 6 went, they came up with an entirely different framework and moved over to Java (as far as I remember). I have rarely seen that sort of thing characterized as a refactor (even on their mailing lists).
Some of us have been using Gecko as their renderer for almost sixteen years now. Seems KHTML(w), the base of both Webkit and Blink[5], is the same age. Seems we've gotten a good run out of the existing codebases. In fact, this means that Gecko (and khtml) have been in use twice as long as NCSA Mosaic, which the first Netscape was based on[6]. Imagine how Spolsky must feel about the Servo rewrite.
1. http://www.seamonkey-project.org/
2. https://en.wikipedia.orgwiki/Mozilla_Application_Suite#Marke...
3. https://en.wikipedia.org/wiki/Netscape_6
4. https://en.wikipedia.org/wiki/Netscape_7
Of course you can rewrite from scratch. You just should not expect to reach the same feature coverage instantly, and should approach the work methodically. Most of all, you should have a good reason. "This code is spaghetti" is usually not one. "I want to implement feature X and the current codebase does not really support is" could be a good reason.
Also, a rewrite is pointless from the point of view of code quality unless you are familiar with similar codebases operating in a similar domain.
Main advantages or rewrite are, that when you are a domain expert and implementing and designing core routines, you get constant flashes of intuition "Aha, I should add this interface here because that will enable me to implement Foo and Bar using this instead of writing a FooSubsystem and BarSubsystem" and so on.
If you have no experience from the domain, you get those insights only after you've written the code and are actually new proud owner of new heap of semi-maintainable code.
Rewrites fall into two categories: essential to business and doomed to fail. If it's essential that the business execute on a rewrite, it will happen or the business will fail. If it's not essential, it very well might be the reason the business fails (siphoning away talent and squandering time when a refactor or benign maintenance would suffice). Precisely because massive rewrites are so risky, they are undertaken by the foolish (myself firmly in this camp). The heuristic under which they might succeed is bare necessity.
So, the engine that Spolsky was saying was a mistake has now been in use for twice as long as the engine it was replacing.
http://www.theregister.co.uk/2003/09/11/bill_joys_greatest_g...
And there is no reason we could ever improve on that 35 years later. we need those optimizations...
"It took a long time. It was really hard to do because you've got to remember that I was trying to make it usable over a 300 baud modem. That's also the reason you have all these funny commands. It just barely worked to use a screen editor over a modem. It was just barely fast enough. A 1200 baud modem was an upgrade. 1200 baud now is pretty slow.
9600 baud is faster than you can read. 1200 baud is way slower. So the editor was optimized so that you could edit and feel productive when it was painting slower than you could think. Now that computers are so much faster than you can think, nobody understands this anymore. "
I think neovim could be a great project and a major change for vim, but it could just as easily never reach a viable state. I won't suggest they give up, but I'm also not putting a lot of hope into it. Bram's vim is amazing, and even vi is stellar. I don't really see a downside whether they succeed or fail.
I use it on a daily basis and have found that it copes with conference wireless far better than vim does.
Bram's conservatism makes perfect sense for someone who has to maintain this huge, consequential, intimidating codebase. Why hasn't he gotten some help?
Nevertheless, people still try. A lot of them fail, a few succeed.
The vim code base is around 300K lines of C code. I'm sure its nothing like Java where 90% of the code is get/set methods, high level delegating classes and exception handling code. 300K lines of code which does the actual work is simply too much code. And one should exercise great caution while dealing with it. Regression is just one thing, I'm sure there are a lot of places in the code which can silently break things you can't detect only to pop up in some strange case in the wild.
I found this hard to believe, so I ran it through cloc:
----------------------------------------------------
Language files blank comment code
----------------------------------------------------
C 113 33366 61819 285817
vim script 1186 22871 32599 145700
C/C++ Header 63 3175 4626 19735
300K lines of code which does the actual work is simply too much code. And one should exercise great caution while dealing with it. Regression is just one thing, I'm sure there are a lot of places in the code which can silently break things you can't detect only to pop up in some strange case in the wild.
In a way, it's like a direct application of the style in /The C Programming Language/.
Some people forked Sodipodi, and created Inkscape. I can remember that one of their first tasks was to make the code compatible with a C++ compiler.
Nowadays, Inkscape is a very good tool, and I hope that Neovim has a great success too.
Main flaws: Apart from implementation issues, non-authenticated encryption susceptible to bitflipping.
I don't think there's a way to learn that except by making the same mistake yourself, possibly more than once.
This is not a total rewrite, it is a massive overdue refactoring.
Sounds like the code of pretty much every non-trivial C program that is meant to run of many platforms, not just the five or ten most popular ones.
I'm not saying that this is a good thing, just that I don't know any good alternatives that don't sacrifice compatibility with lots of platforms.
The very first commit in neovim includes this log:
- Process files through unifdef to remove tons of FEAT_* macrosIt also makes merging back to vim difficult to impossible.
Expect to see 'neovim' development dissipate rapidly.
There's nothing wrong with that as long as those typedefs are properly isolated in some platform-specific header files.
It's natural to have an initial feeling of being overwhelmed by that kind of stuff. However, it often turns out that there's nothing to gain from removing that stuff.
The only real cost is when people expect a platform to work, but nobody from that platform is willing to put in the testing effort.
That function is over 400 lines and has over 50 ifdefs. What does it do? It waits for any key to be pressed. Unfortunately, Vim is single-threaded and synchronously character-driven, so it has to try and process some autocmds and other events in the same loop.
Oh and the wait loop uses gettimeofday(), which is not monotonic.
To get another taste of Vim's source, take a look at https://github.com/b4winckler/macvim/blob/master/src/eval.c (Warning, may crash your browser.) That file is 25,000 lines and checks hundreds of different ifdefs.
I've never read the vim code before and I was surprised how cleanly written and well-commented it is, given my expectations set by this thread.
It will drop support for lots of legacy systems. That will allow simplification of the code - at the cost of those on amigas.
I love vim. I use it for all my development. I've only been using it for a couple of years though and I can see the warts that are being addressed. They were fairly evident upon joining the ecosystem and I don't think "it's too hard to change" is a viable long term strategy.
I would absolutely love neovim to succeed - and I think that something like it is essential if vim is to have any place in the future.
If it's often, then it's not always, which means that in some cases it could be done better. Is Vim one of those cases? I don't know, but you don't give any reason to suggest that it isn't.
As for Bram, he can, and most likely will continue his good work on vim. I am kind of disappointed that he is not more supported of this effort though.
- Excerpt from http://en.wikipedia.org/wiki/Bram_Moolenaar
Or, at least, I think that's the best way to summarize the essential miscommunication between these two camps. Vim is, in the spirit of Unix, a single purpose tool: it edits text. Why would it need threading? async? background jobs? You can't type in two places at once, right?
NeoVim wants to be something like an IDE-lite. Text editing, yes, but with the ability to do background compilation/syntax checking, a bit of debugging here, some REPL-ish things there. For that you need a better async story, better process control, etc.
So, in the end, I don't actually see a problem here. Vim will cater to one audience, NeoVim to another. For my money (or donation to Uganda), I'm happy using tmux splits, sending text between vim windows and REPL windows, and keeping Vim primarily as a text editor with syntax highlighting. I would appreciate if some events could be backgrounded (e.g. syntax check on file save), but I don't think this needs a complete refactor.
You might want to. See https://floobits.com/
If you've got a clean separation between the editing engine and the rest of the IDE suite of processes, that is the Unix way in my view.
Of course, Bram knows the codebase and I don't, and I may well have misunderstood the neovim plan, so I'm not sure which of us understands.
"I’ll take this opportunity to coin Carmack’s Law, as follows: Fight code entropy. If you have a new fundamental assumption, throw away your old code and rewrite it from scratch. Incremental patching and modifying seems easier at first, and is the normal course of things in software development, but ends up being much harder and producing bulkier, markedly inferior code in the long run, as we’ll see when we discuss the net code for QuakeWorld. It may seem safer to modify working code, but the nastiest bugs arise from unexpected side effects and incorrect assumptions, which almost always arise in patched-over code, not in code designed from the ground up. Do the hard work up front to make your code simple, elegant, great—and just plain right—and it’ll pay off many times over in the long run."
That may be a problem for vim but not for neovim. Neovim doesn't have a legacy user base, they can focus on the currently used systems.
I really love this business model, the developer works on something he is passionate about, we can support the project and influence its future, all while strengthening the community around it.
As a Go programmer, I feel a bit sad/ashamed to be saying this, but look at Plan 9 - it failed to usurp Unix (and its descendants) as the dominant OS, because the latter was "good enough", and was already more widely supported and used.
Plan 9 vs. Unix is a very different comparison than Vim vs. Neovim, and there are a lot of other factors at play, but the same principle actually holds. Improving or extending an existing system in a compatible way is a lot easier than trying to establish mindshare with a completely new tool.
The tradeoff is that extending existing systems leads to cruft and bloat. So if you can establish mindshare with a new tool, you have an opportunity to make a much more elegant one. But that's a big "if".
EDIT: I think a better analogy may be an attempt to refactor the Linux kernel into a microkernel. I'd certainly love it if this magically happened - microkernels are much easier to work with, and much more elegant. But it'd be hard to make the case that that'd be a worthwhile endeavor at this point, given the costs of doing so.
If you view Go as a "rewrite" of C, then perhaps, but I wouldn't. It's a new language altogether. C can be embedded in Go easily, and even if it couldn't, there's no reason that one has to "win out" over the other. There's less direct competition between two languages than there is with a family of OSes. And Go was never intended to replace C completely - it just provides a better alternative for some subset of what C/C++/Java are used for. On the other hand, any given system is going to run only one OS[0].
In any case, as I mentioned, it's a tradeoff. There's no absolute answer, though I think he's right here in that the benefits don't outweigh the costs for this example.
Put another way, much as I might like to run a microkernel, I'd have a hard time concluding that it'd be the right move for the Linux project to spend time refactoring the entire codebase into a microkernel!
On the other hand, to see an example of a rewrite that was successful, look at Reddit, which rewrote the entire codebase in its early days. GCC could also be considered an example, depending on how you look at it.
[0] You can run more than one via virtualization, sure, but a number the benefits of Plan 9 come from having an entire set of computers running the same OS.
Although incorrect and annoying, they are at least becoming a way to identify people who likely don't know what they're talking about.
Plan9 was a pretty significant departure from Research Unix V7, but not as significant a change compared to V10... which didn't matter because nobody outside Bell Labs was using V8/9/10 anyway, since everyone was working on various forks of V7 (BSD, SunOS/Solaris, Xenix, etc.).
> Research Unix V8, V9, and V10 got virtually no uptake, despite being incremental evolutions of the original upstream, due to a mixture of inertia and licensing restrictions from AT&T.
I'm not sure what you mean by inertia. v8-v10 were not widely released to avoid another BSD incident.
v8-v10 were 4.3BSD derivatives, so I'm not sure I'd call them incremental evolutions on the original upstream. They removed and/or reworked all the new BSD stuff though, and the basic userland was very custom. Some of it survives (in some form) in Plan 9.
However, it's sure easier to refactor than start from scratch! At least LO has a working product. Best of luck to the Neovim guys :-)
All Thiago got was: being ignored, one of the worst ways of disdain. Not just even a 'thanks but not interested'.
Other two guys who tried to provide patches to be merged on the same area also suffered from being mostly ignored (he replied to them but just once I think).
Trying to rewrite vim is the wrong approach. Trying to replace it in a backward-compatible way, the way vim did to vi might received a lot better.
2. :%s/vim/neovim/g
3. profit!?!?!?!?!?
Seriously, screw that guy.
edit: Yes I know who 'that guy' is and it's great he's been working on Vim for all these years, but he certainly has his head up his ass if he thinks Vim users would not benefit from some big refactorings, a more accessible code base and some dropped platforms. If all those Amiga programmers love Vim so much, why would they be disappointed with an eternal Vim7.4?
It's not too late for Vim to lose the editor wars.
For python at least, having 'preview' as part of your completeopt has it display the docstring of what you're completing. And :h completeopt suggests that's intended behaviour for all types of completion (where it makes sense), too.
I would say that for the majority of people Vim is just working fine.
That 'guy' has been maintaining Vim for years and all he asks for is to donate to his charity.
Also, did you read his response? He doesn't say don't do it. Just wouldn't be easy and some current functionality may be lost. In his mind, it would be better to work on the current code base and do small changes. I would say that's an entirely reasonable response.
It's comments like this that turn people off from working on open source.