Making bracket pair colorization faster
code.visualstudio.com
code.visualstudio.com
Moreover, he has stated that he was tired of maintaining the extension and seems to be fully in favor of this move:
> Author of Bracket Pair Colorizer here.
> I follow this thread with interest, my extension is something that grew a bit out of control, and I grew tired of maintaining it.
The comments here suggesting that the Visual Studio Code team was somehow wrong for making this improvement or that they otherwise wronged the extension author are not reflective of the actual process nor even the extension author’s own feelings.
The blog post does a great job of describing why bracket colorization isn’t appropriate for the plug-in architecture anyway. Making it fast can only be accomplished within the core of the editor code which has more direct access to syntax parsing.
This is a win for everyone all around. The Code team did a great job with this feature and the associated blog post.
https://devblogs.microsoft.com/python/don-jayamanne-joins-mi...
This can still be seen on GitHub - if you look closely, the official repo at https://github.com/microsoft/vscode-python is a fork!
Curiously, some bits of code went kinda full circle in the process - vscode-python reused the (open source) Python debugger written at Microsoft for Python Tools for Visual Studio.
"Oh I see why. .. yeah that's a better idea."
Happens to me all the time ;)
Ironically, when we rewrote the Python debugger later, we did the same exact thing with pydevd (https://github.com/fabioz/PyDev.Debugger), and for the same reasons - why reinvent the wheel when you can take an existing one that's already better than anything you have? It's also better for the ecosystem, since improvements all flow upstream.
> Without being limited by public API design, we could use (2,3)-trees, recursion-free tree-traversal, bit-arithmetic, incremental parsing, and other techniques to reduce the extension's worst-case update time-complexity (that is the time required to process user-input when a document already has been opened) from \mathcal{O}(N + E)O(N+E) to \mathcal{O}(\mathrm{log}^3 N + E)O(log 3 N+E) with NN being the document size and EE the edit size, assuming the nesting level of bracket pairs is bounded by \mathcal{O}(\mathrm{log} N)O(logN).
just hurts to see after having been burned by Apple's private APIs blocking legitimate app developers from doing something Apple will release next cycle themselves, even if it was to everyone's benefit and a net positive
That way lies Emacs ;)
That's the key difference between Emacs and most other editors: there's no limited API for extensions. There's a base C runtime, and all the lisp code on top is at the same level. There is no difference in access rights between a core Emacs package (coming with Emacs) and an additional, user installed package.
Of course, there are a lot of documentation, conventions and best practices to support this. And all the code is accessible.
In total there are ten ways to combine new advice with the existing set of methods, but the most commonly used are :before, :after, and :around. All the :before functions are called first, then the outermost :around function (which may or may not call the next :around function), then finally the :after functions.
Also it is common to define explicit hooks, which are just lists of functions that you will call at documented points. This is functionally identical to advice, except that it is also a good signal that the author intended you to do so.
The terminology is interesting too. Advice is deliberately modeled after the generic method combinators in Common Lisp. One of the authors of the Common Lisp spec, Gregor Kiczales, went on to define the term “Aspect–Oriented programming” and later developed AspectJ. Although the phrase appears nowhere in the documentation for Emacs Lisp, it is definitely appropriate!
Or maybe we could both be more concise by just telling people to go read The Art of the Metaobject Protocol.
(Ug, the way autocorrect works sometimes..)
Partly this is because the nature of the Lisp language makes these changes easier, but it is also the case that many “extensions” are actually included with Emacs. There are over a 1.5 million lines of lisp code that are included in the Emacs repository, though most of them are not enabled by most users.
Other extensions come from a wide variety of sources (and of course many users write their own code), but over the last 10 years most of them have been moved into installable packages hosted on elpa.gnu.org. There is a little over a million lines of code there.
It takes just a couple of minutes to check out both git repositories (for Emacs and ELPA), so any time you want to change something in Emacs it is quite easy to search all of that and find out exactly what, if anything, you might break.
VSCode vs Atom
WebExtensions vs XUL
Mod API vs patching game files
Vim vs emacs
In my opinion, the ONE big thing that killed Atom was LSP[2]. VSCode was slightly better in everything, but good performance and LSP destroyed Atom and damaged all other editors (like Sublime).
But MS plays dirty here too. In VSCode, LSP works with many inner hacks. You can't get the same experience with LSP in other editors because some features are part of the editor, not just LSP. But I think LSP is excellent and use it in Emacs and Vim too. For Rust, it's maintained by the Rust team and the default "engine" for editors.
Anyway, I'm still using Atom today with a minimalistic theme and setup to edit markdown files with code and as a text editor with a friendly GUI overall.
What animosity? The animosity other people have on behalf of the author who has clearly said he has no beef with this series of events? If the author wanted to make algorithmic changes like the VS code team have done, I'm sure they would have been more than happy to discuss this on the open source issue tracker (which is very active).
> just hurts to see after having been burned by Apple's private APIs blocking legitimate app developers from doing something Apple will release next cycle themselves,
The big difference here being that the app developers are happy that this has happened, and they could have attempted to do this work because even though the APIs are private, the source is available and they readily take pull requests.
I don't see why this would make a difference. Say I'm working on a project. There's a hole in my project. And you offer something to fill the hole.
But your action can't then bind me to refrain from fixing my project. Once my project no longer has a hole in it, you've lost a line of business. But you've lost that line of business because it solved a problem that was correctly fixed. The situation now is obviously better than the situation before. If I needed permission from you to fix my own thing, how could that conceivably improve anything?
I recently commented on it here:
https://news.ycombinator.com/item?id=28556588
Microsoft engineers are top-notch and this is another feature I'll be using daily. Good stuff!
The hate for Electron comes from how the average Electron application works. Not only almost nobody cares as much as VS Code team does, the very choice of using Electron itself is usually an act of not caring.
That's a little misleading. If you write the same program in idiomatic C++ and Python then it's almost guaranteed that the C++ version will be much much faster even before you have done any profiling or performance optimisation. So there is some magic pixie dust.
Block the event loop in other languages and you'll also see shitty results..
For 10 years an ArrayBuffer has been able to be sent by reference but good effort on disinformation.
transfer Optional
An optional array of Transferable objects to transfer ownership of. If the ownership of an object is transferred, it becomes unusable in the context it was sent from and becomes available only to the worker it was sent to.
JS is very performant if you know how to use this hybrid architecture. The C++ guys above are shitting all over JS when an Electron app can simply use C++ transpiled to WASM if they really wanted to. Electron isn't really for JS as a coding language. It is much more for the awesome cross-platform UI you get with HTML, CSS, and JS. A lot has been done to optimize rendering engines. The same techniques that render snappy web pages can be used within Electron. All the griping above is really just ignorance.
https://github.com/microsoft/vscode/search?q=wasm
https://github.com/microsoft/vscode/search?q=arraybuffer
https://developer.mozilla.org/en-US/docs/WebAssembly/Using_t...
https://developer.mozilla.org/en-US/docs/Web/API/Worker/post...
If that is really how they got their performance then no wonder that few others actually managed to do it, because most teams wouldn't write their Electron app in C++ and then compile it to WASM. The difference is that Microsoft has a ton of C++ engineers so they could do it easily, but I doubt many Electron teams put out job postings to hire C++ people.
Funny that the C++ guy is talking about abstractions, where them V-Tables at?
In these situations it is common to have a problem that is really easy to solve but the API abstractions you have to work with doesn't support the operations at the speeds you need to solve it. If you have never experienced that then you aren't working on performance intensive projects where every bit counts and your input on this topic doesn't matter. I have worked on performance intensive libraries crossing programming language boundaries and the abstraction boundaries absolutely puts a limit on the amount of performance you can get. Well designed abstractions are less obstructive but plenty of them aren't well designed and even the well designed ones have overhead.
This is why it’s very difficult to have conversations. Most programmers really don’t even know how computers work.
Here is a real kernel: https://news.ycombinator.com/item?id=7958194
It's so difficult to have conversations with engineers that don't know x86 assembly. They never stop complaining. If only I could NOP them! ;-)
To get rid of abstractions you almost should be writing more directly in some sort of COM+ language.
(To explain the punchline for those that don't catch the joke: COM+ was the codename/early name of what became .NET.)
The funny part is OP is talking all about uOPS when if he was really hardcore (like I am) he'd know that these dlls in turn call ntdll, many calls in ntdll are undocumented but much faster than their wrappers in the other dlls. But no sane person is going to strictly do ntdll calls except for the most performance critical code.
OP just doesn't know enough about C and C++, he probably grew up on C++ and forgot about the old C apis. I used to reverse engineer and delve deep into the windows API. I know a little bit more about performance than the average high level programmer.
And ultimately, .NET does a fine job with performance. C++ coders crapping all over .NET should take a look at the Objective C API of Apple. It's the default and every Objective C call incurs overhead and is basically a wrapper around the undocumented C API. But I don't think anyone ever complained about this abstraction, because it's such a stupid and small amount of performance to harp about. The convenience outweighs the tiny little uOPS loss.
Your specific example - inlining functions - is not particularly illustrative, since C++ can inline just fine across translation unit (and thus also static library) boundaries with link-time optimization. What it can't do is inline across shared library boundaries, but large C++ apps are usually mostly statically linked for redist anyway.
https://news.ycombinator.com/item?id=17648139
> What it can't do is inline across shared library boundaries, but large C++ apps are usually mostly statically linked for redist anyway.
I'm guessing you've never looked inside System32 on Windows or /usr/*/lib on Unix. I don't see shared libraries as a bad thing, they are great for reducing disk and memory usage. Let's not make bullshit up about how most applications are statically compiled with all their dependencies ;-)
Java is also not JS. Semantics of Java are much better suited to effective compilation than that of JS, due to static typing and (in many cases) early binding. Consequently, modern Java VMs have the best JITs in the industry - faster than V8 - and they're still not on par with C++.
Most applications dynamically link to system libraries. On Linux, this is kinda fuzzy because package managers handle everything; and yes, I agree, on Linux the norm for distro-packaged software is dynamic linking. But stuff packaged as Flatpak etc is much more likely to be statically linked to anything other than libc. And idiomatic C++ tends to involve lots of templates, which are inherently "statically linked".
(Also, native code is broader than C/C++ - it includes e.g. Go, which drops even the libc dependency, and Rust.)
On Windows and macOS, though, where "app is a self-contained folder" has been the rule rather than the exception for a long time now, dependencies that don't come from the OS are often statically linked.
You keep saving face. So you admit Linux is largely dynamically linked. Well Windows is too, even to C stdlib, look at how many versions of MSVC runtimes are in your system32 dir after just a few installs. Templates are preprocessor so obviously they are "statically linked."
Microsoft has a COM API (which a lot of C++ uses) but when I used to develop, I'd just call kernel32, user32, advapi32, and other system C APIs directly. COM is a POS imo. DirectX is a decently engineered class-based API. But the rest of them have a lot of flaws.
If you were really hardcore you'd know that these dlls in turn call ntdll, many calls in ntdll are undocumented but much faster than their wrappers in the other dlls. But no sane person is going to strictly do ntdll calls except for the most performance critical code.
I used to reverse engineer and delve deep into the windows API. I know a little bit more about performance than the average high level programmer.
And ultimately, .NET does a fine job with performance. C++ coders crapping all over .NET should take a look at the Objective C API of Apple. It's the default and every Objective C call incurs overhead and is basically a wrapper around the undocumented C API. But I don't think anyone ever complained about this abstraction, because it's such a stupid and small amount of performance to harp about. The convenience outweighs the tiny little uOPS loss.
Yes, I'm well aware that many system DLLs on Windows in turn call into NTDLL, where the actual syscalls are. And yes, I agree that it hardly matters in practice - but it was your premise that inlining across shared object / DLL boundaries is crucial! In practice, yes, it almost never is. And yes, .NET is perfectly fine perf-wise, and even JS is fast enough for most cases. I've actually spent most of my career writing C# and Python, after writing a bunch of C++, and I very much appreciate the productivity gains those abstractions offer.
But this is a very different point. Native code is still measurably faster where it matters, and JS/V8 can't keep up for very good reasons.
Those gains have materialized in distributed computing, where the 1% cases where C++ wins in micro-benchmarks don't really matter, when network latency, databases, load balancers and the whole lot come into play.
However, I notice a significant difference in responsiveness between VS Code and Sublime Text -- enough that I changed my workflow back to Sublime Text because the very slight latency difference annoyed me. So I do think there is a baseline difference between the frameworks used by those two apps that no amount of optimization can overcome. It's sort of like the acceleration difference between a truck and a sedan: sure, powerful trucks can sometimes out-accelerate anemic sedans. But if you put a similar drivetrain in both vehicles, the vehicle with less mass (or memory footprint, in the framework analogy) is going to win.
The slowdown is architectural or design based. Sending one HTTP request and not updating your UI until the request has completed fully is going to have _way_ more of an impact on the perceived performance and responsiveness of a native app.
Even then, it's a pretty poor exception. The "good" performance of VSCode doesn't scale. It's a pretty barebones editor by itself, and once you load it down with extensions, it slows down quite significantly. The python extension, also written by Microsoft, is one that was bad enough to make me leave VSCode for good.
Correction: built on top of a bytecode-compiled language that, few years ago, became a native-compiled language (both AOT and JIT) - initially as an experiment/optional feature, and as of ~5 months ago, this has been merged into the main repo branch.
Hmm, not sure I buy this, sure it's not as fast as going native on every platform. But let's be honest the alternative isn't 3-4 native applications built with care for each special little subgroup. It's a MacOS only app in the US market or a Windows only app everywhere else.
Businesses don't have an unlimited amount of money to spend addressing every tiny market so I'd say the complainer's focus should be on building a better X-plat story if they really care rather than whinging that some application isn't built to make the most of the 0.1% of the addressable market they find themselves in.
Electron comes with an enormous, deep, absolutely stupid big pool of web developers. I genuinely can’t emphasize enough how big it is, and how easy it is to hire from.
Montevideo, Montenegro, Monterrey, doesn’t matter. Hot, not particularly expensive developers are ready to churn out React UIs in your area!
Looks at the web.
As Linus Torvalds put it, it's worth writing the whole thing in C for no other reason than to prevent those 'developers' from 'contributing' to it.
Or better yet, make a web engine from scratch, that is, write the rendering engine, compositing engine and so on using Electron primitives.
Because both of these work very well when using Qt as a base, see LXQt and KHTML.
JS can get close to C++ in speed but the same program in JS will always need more memory.
And what's unacceptable about wxWidgets performance?
And here is a must read book to do great UIs in Swing,
Here are examples of Java applications based on Microsoft's Fluent UI design,
A lot of it, given the comments I see in relevant threads, comes from the inefficiency of every electron app having its own copies of node and chromium in memory (and on disk, and being transferred over the network when installed/updated, though those are smaller issues than run-time performance). Though that is unavoidable given how it works unless particular LTS versions can be enforced so everything is tested against those making sharing easier/possible. I'm not sure how much of an issue it really is anyway: how many people are running several instances of Electron applications at once?
Also I get the impression that a fair amount of it comes from people who are parroting what appears to be a popular sentiment, without actually understanding or having skin in the game! This says something disappointing about technical communities.
They put a lot of thought into it like you should when designing any application for any platform. They also have a team of very competent senior developers who have previous experience with developing IDEs like Eclipse.
A lot of people complaining that Electron makes people choose an “act of not caring” but nobody actually does anything about it. Last I checked Electron didn’t make other UI kits go away.
I’m not without criticism for Electron. But I begin from a position of good faith on the subject, and appreciate that a lot of these apps simply wouldn’t have existed otherwise.
I'm not aware of any public plans to move VS Code to WebView2, but I would be surprised if it doesn't happen eventually.
Ideally, unlike in TeX, additions to nesting levels wouldn’t require a re-flow (i.e. typing a bracket wouldn’t make the code constantly shimmy around on the screen), but rather these would just be changes in the perceived size of these characters, while keeping them where they are on a monospace grid (since a lot of code assumes a pure monospace layout for indentation et al.)
In my imagination, this would work by just having a set of bracket/paren/brace/etc. graphemes in each font, that have increasingly-long ascenders/descenders, while the base size of of the “functional” part of the grapheme remains constant. As if there were five versions of the letter g, with an increasing length of the stem on which the lower hook rests.
As such, these magnified brackets would be rendered similarly to the way emoji are rendered in terminals: the grapheme would eventually “leak out” of its monospace box (in the emoji case to the sides; in this case above and below), to the point of eventually overlapping surrounding graphemes, rather than re-spacing the text to give it room.
If someone else remembers this and the name I'd appreciate it. I could be remembering wrong too about how far it went with such decoration.
bracket-highlighting-1 ... bracket-highlighting-6
unexpected-closing-bracket
The issue isn't about the speed of the language, but the code not having access to information it needs from the extension API so having to go long ways around to work it out. The new version may be in a faster base language, but the majority of its speed boost comes from having access to information that the editor already knows without having to mess around re-deriving it.
First world problems: Bracket colors are different now and especially XML looks off... seems I need to fine tune that some day (but not today).
It shows enabled, but immediately under has an unchecked checkbox.
editor.bracketPairColorization.enabled
For everybody else, no.
Before that, I got some basic knowledge about CS theory, master theorem, asymptotic, all these things, and was a typical skeptical developer who was thinking like, "yeah, that's cool, but you don't actually need it. At all, basic knowledge is enough."
Today, I will recommend that anyone go through basic algo and ds courses (Sedgwick book or course, Skiena book too) and try to practice them.
Now I'm even trying to participate in leetcode and codeforces contests. In the end, after the first months of frustration about how stupid I was and how I couldn't find a solution to easy problems, it's started to be fun, and I loved it.
At least that's my understanding, I'm a long time emacs user, but I don't have a very deep knowledge of the internals.
Tree-sitter is much more general (and I guess way more complex) and most likely cannot use some tricks we use for "simple" bracket pair parsing. For example, we can almost always re-use nested bracket pairs when characters are inserted/deleted, because the bracket pair language is so simple and it does not matter where a bracket pair is in the AST. But when parsing C# and adding a single opening bracket at the beginning of the file, I doubt namespace/class declarations stay namespace/class declarations.
Also, long lists in Tree-sitter seem to cause linearly growing incremental parsing time - that's why we use balanced (2,3) trees. You can try it out on their playground [1] by selecting JavaScript and adding some 100k `{}`s. On my machine, adding a single character takes 50ms. When there are 200k bracket pairs, it takes 100ms. When all these brackets are contained in a single bracket pair however, adding characters after this single root pair is fast again (<1ms). But to be honest, Tree-sitter is still mind boggling fast.
Will have to look more into the linear growth you mention.
The linear scanning behaviour you describe is due to the change in the left and right parse context for all of those pairs. Yes, it’s linear when you invalidate a subtree, but in the case of a simple bracket pairs language, the damage is limited to scanning through the children of top level brace pairs, and each child subtree is trivial to check.
From the IGLR paper:
> In a state-matching implementation, each node representing a nonterminal symbol contains a record of the configuration of the pushdown automaton (the ‘parse state’) when the node was shifted onto the stack. A subtree can be reused when both its left and right context are unchanged …
Imagine { is prepended to abc(def, {ghi}). I believe the “abc” needs re-parsing, and so does the () subtree as their left parse state is now the “looking for }” state instead of “looking for ({[“ and “looking for )” respectively. But it’s limited to one level deeper — after you split the () subtree, every subtree inside it is still in the “looking for )” configuration on both sides. Specifically, the “def”, “,” and “{ghi}” are fully reused. There are only three possible parse states, so generally you get a lot of subtree reuse. You get even better subtree skipping performance by making long sequences without an brace into a single node, instead of eg tokenising by word and not grouping them. (In this case, “def, “ instead of splitting that.)
So realistically for your C# example, only the namespace node is split, and the linear scan is through all the top level brace pairs within the namespace but no deeper. You’ve described IGLR’s best case scenario for incremental parsing an initial brace insertion. Languages that don’t have namespaces would be worse off (but still not too bad). For more realistic languages than {}{}{}{}{}{}{}{}.js I think this approach would work very well.
If I liked brace pairs (I don’t) then I would implement this dead simple new grammar and turn it into a Neovim plugin. The existing nvim-ts-rainbow plugin uses queries on existing languages, so is very good at giving perfect/correct pairs and customising per-language, but exhibits the invalid syntax problem and also performance issues which appear to be resolved for most people, but may be back with bigger documents. (Edit — you would need a few variants to account for comments and strings. That makes it a bit more annoying.)
It really depends on how you parse. In VS, at least, this works, in a sense that, while the resulting code is invalid, the editor can still correctly semantically highlight it, and provide code completions etc.
https://code.visualstudio.com/api/language-extensions/semant...
per variable basis, so apparently different than the vs code thingy with the same name.
Also lots of extensions for similar stuff, often named "rainbow" something.
Then by querying them only when the document changes could make it another 10,000x faster ;)
Get off my lawn (while I water it with my tears).
This is a really nice and clear blog post on an algorithmic challenge wonderfully addressed to a satisfactory conclusion (the rebalancing and node reuse etc work out so perfectly); must have been fun, thanks for sharing! I don't know / haven't thought much about text editors so a question for the authors or anyone who understands: from the mention of "the bracket pair AST", it seems that a separate AST is created just for this bracket-pair problem, is that right? I imagine there was/is already (at least one) another AST already being computed in the editor, for other reasons? If so, how did you decide between trying to do this with the same AST (making that one more complex), versus computing an additional AST just for this problem?
That is right.
> I imagine there was/is already (at least one) another AST already being computed in the editor, for other reasons
Not in the renderer process.
The point is that more concrete ASTs that model more features of the language are less likely to have reusable nodes. When you have just bracket pairs, you can reuse any bracket pair that was not modified! However, when prepending a JS file with `[`, a class declaration probably does not parse as class declaration anymore, so you cannot just reuse it and make it child of the array expression (what you could do if it was just another bracket pair).
This is cool although I feel like the caveat of the ambiguity of < and > could isn't the best experience, especially for TypeScript and C++ I imagine.
The parser is language-independent, but the tokenizer used by the parser looks up the tokens of syntax highlighting to decide if a bracket character is an opening/closing bracket or just text.
Where do you see this? The article talks about nesting levels being bounded by O(log N) levels where N is the length of the document, and from what I can tell this is just for analysis and it supports arbitrary nesting levels… Oh, is your question about the actual colours used for the brackets, i.e. arbitrary levels of matching are supported, but the colours repeat every 6 levels (e.g. brackets at levels i and i+6 have the same colour)? If so this is a UX choice (I can't see any technical reason) and I imagine the considerations might be something like:
- There are only finitely many colours that can be usefully distinguished visually by the average person, so we need to pick some limit L,
- The limit L needs to be small enough that all L colours are easily distinguishable and memorable, but large enough that it would never be confusing whether a certain coloured bracket is at nesting level i or i ± L.
I guess the thinking may have been that 6 is large enough typically, e.g. it's likely to be clear from context whether a certain bracket (say blue) is at (say) level 2 or level 8.
There is no technical reason for this limit of 6.
> All colors are themeable and up to six colors can be configured.
[0] https://code.visualstudio.com/updates/v1_60#_high-performanc...
Interesting this NEEDS to be enabled. ...or will it be set to `true` on new installs?
Seems like a super useful feature that should be on by default to me..
People who are resistant to change (and likely to complain) will just not be affected.
I wouldn't turn it on for existing users, but as you say, having it on for new installs does make sense though!
Constructing or incrementally updating an AST is not cheap. In particular, most ASTs are very fragile and there are characters when inserted at the wrong position invalidate the entire AST (e.g. prepending a C-like document with `/*`). Even prepending documents with `{` could render all nodes of the AST invalid, as they might get a different meaning (a class declaration might now be parsed as block statement).
The AST for the language of well-formed bracket pairs is very robust. There is no character that invalidates the entire AST. There are characters though that invalidate all tokens, but there is a separate asynchronous (slow) method that updates them in the background and only then incrementally the bracket pair AST.
I think you have a worldview where text is being edited and ASTs are constantly being replaced via parsing.
In an AST-based language, it doesn't necessarily have meaning to add some random character at some random spot, in the way that it does with text. With text, you can only recreate the entire AST by parsing, and discover that the parse is broken.
With an AST-based language (such as https://darklang.com), the editor could inform you that that character cannot be added in that place. Or it could allow you to put it there, informing you that that particular piece of text has no valid meaning, while the rest of the AST is still valid. (Darklang uses both approaches).
If you want something along these lines, but a bit closer to mainstream, with easy access to popular libraries etc, that would probably be F#.
Also, "Feel free to skip the sections on algorithmic complexities." repeated so many times sounds rather patronizing.
Also, I think the old bracket colorimg extension has an option to underline the entire section of code between the brackets pair you're currently in.
> Even though JavaScript might not be the best language to write high performance code
Why not?
I love JS but this statement is absolutely true.
What the actual f** ?
42k lines of code and they mention it casually without even thinking about explaining the reason behind this monstrosity?
And yes, I fully expect the comments below to contain stuff like "This is nothing, back in the day we had a 1M loc Java class."
https://raw.githubusercontent.com/microsoft/TypeScript/8362a...
In it, CoenraadS (the extension developer) writes
"I follow this thread with interest, my extension is something that grew a bit out of control, and I grew tired of maintaining it."
and
"If it could that would be great, and my extension could be deprecated completely."
There was also discussion with the author about updating the extension to prompt users to switch to the native functionality, and it appears CoenraadS was also involved in the process/design of implementing the feature in VS code.
This appears to be a shining example of doing it right, and giving the original extension author credit.
You must have missed the part where it is now 10,000x faster. Given your complaint about performance this should excite you!
Is there no background code analysis in VS code? That would be a reason to switch.
> the asynchronous communication between the renderer and the extension-host severely limits how fast bracket pair colorization can be when implemented as an extension. This limit cannot be overcome
Nope, I'm not convinced. If it can be done internally, then you should expose more internals until it's possible to do it with the public API.
I bet this could be made really performant if vscode had incremental parsing, like tree-sitter+Atom.
> This is another challenge of the Bracket Pair Colorization extension that affects performance negatively: it does not have access to these tokens and has to recompute them on its own. We thought long about how we could efficiently and reliably expose token information to extensions, but came to the conclusion that we cannot do this without a lot of implementation details leaking into the extension API. Because the extension still has to send over a list of color decorations for each bracket in the document, such an API alone would not even solve the performance problem.
This is perhaps the core difference between VS Code and Emacs (which a lot of people believe is being superseded by VS Code as the "extensible editor") - in Emacs, there is no such thing as limiting access to information. Outside of things that get hidden accidentally[0], any bit of elisp code can access everything.
It's not just a practical difference, but also a philosophical one: Emacs plugins are not designed to expose a narrow API, because it's impossible to enforce anyway. There's a structure to it, so nothing prevents one from creating good abstractions - those abstractions just have to be designed with extensibility and interoperability in mind[1], because the users (including other package developers) always have an option to just hook into, advise, override or replace any piece of code in your package.
Performance-wise, the impact of it varies. On the one hand, this level of flexibility prevents Emacs from making important breaking changes[2]. On the other hand, nothing ever has to wait for a better API design - if there's a way to make a feature faster by hooking to a dependency's internal, people will do just that, and keep doing that until the API blesses the use case.
--
[0] - Like state the C core doesn't expose, or some implicit state shared by a bunch of closures - though you can get at the latter if you override whatever is hiding the state.
[1] - E.g. by offering hooks as a blessed, well-defined and stable way to interact with internals to cover 90% of interoperability needs.
[2] - Like doing proper multithreading, or replacing Emacs Lisp with a more polished Lisp - though myself I feel ELisp is good enough as a language.
This means that plugins which work great in Emacs, where there's no IPC overhead, might be pitifully slow in VS Code. Alternatively, plugins that work great in VS Code might bog down Emacs (yes, the Emacs plugin can spawn a new thread and work there, but I don't think that's the typical approach of plugin authors)
Note that Extensions are able to directly manipulate the main application bundle a la Emacs (this is what https://marketplace.visualstudio.com/items?itemName=be5invis... does, for instance), but that is discouraged by way of a checksum failure putting "Unsupported" in the title bar. Of course, that checksum code could be removed by the extension too, but doing so without informing the user would be considered in bad taste.
That said, I do get the point on wanting things to be IPC based, but that feels like a large jump in complexity for most items. I'm very grateful for the model of extension in emacs, where you do have to learn complexities if you are building a complex plugin, but you can go very far before you get into the realm of complex plugin.
The extension model is the same in VS Code, all the author needs to do is write basic JS (or TS). VS Code core does the heavy lifting of creating the extension host process to run that code and exposing the `vscode` proxy object to the extension's code, which enables communication between the extension and the renderer in a manner that appears identical to as if the `vscode` were a simple object. See the minimal hello world sample [0] for the basic case.
End of the day it's a tradeoff between latency and throughput, emacs chooses latency, vs code chooses throughput. Both have their ups and downs, largely dependent on the size and frequency of the task at hand.
[0] https://github.com/microsoft/vscode-extension-samples/blob/m...
I agree the difference is emacs is done such that it is all exposed to all developers. There are no special parts, as it were. Such that a hello world looks like (defun my-ext () (message "hello, world")). If binding this to a key, you simply need to add (interactive) after the argument list.
Obviously, things ramp up quickly, but the point is it isn't some special framework to make extensions. It is just a function.
As you say, Emacs has no special framework for extensions - it's all just functions and variables, plus a bunch of higher-level concepts (e.g. minor and major modes, customize, autoloads) to pick and choose from.
Tradeoffs all the way down :) If we didn't have them we wouldn't be engineers...
This is an odd shift of the word "stable" in software. For a long time, folks took stable to mean simply "doesn't crash." But, it also needs to mean "remains unchanged" if you want to use it as a foundation of other work.
Of course, I say that, and have to ack that the web itself has proven a major exception to this supposed rule. Nothing has been stable in that entire landscape, but it has continued to grow at an astonishing pace.
Why not? If the extension is popular that suggests the functionality is something people want, and surely if you're working on the next version of something you'd be interested in things people want?
One hopes that they wouldn't unilaterally do this to things that are monetized, but otherwise it's hard not to see how it would be a win for everybody.
This has nothing to do with, and isn't even exclusive to Linux. If you have an axe to grind, at least grind it well.
Also, to my knowledge, tree sitter cannot move nodes around (which is also very hard for languages that are not as simple as the Dyck language [1]).
For bracket pairs, you can easily reuse all (...)-pairs in the following example, even though their height in the AST changes significantly: old text: (...)[(...)] new text: {{(...)}(...)}
With the editor's native performance 10,000 times better, I suepect the number of installs of their extension will plummet.
It's hard to compete with 1st party powers, as all the hoopla around app stores (and before that, the famous "Embrace, extend & extinguish" [1] strategy) show all the time.
That said, I'm not saying it's a hostile move, it's just ... interesting.
[1] https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguis...
We openly discussed various approaches with CoenraadS and other extension authors here: https://github.com/microsoft/vscode/issues/128465#issuecomme...
> Author of Bracket Pair Colorizer here.
> I follow this thread with interest, my extension is something that grew a bit out of control, and I grew tired of maintaining it.
> If there are some quick wins, I can still apply them, but I think my extension is so hacky it is easier to do 1.b or 1.c
Seems CoenraadS is completely in favor of the native implementation (and probably could have been called out in the blog post as few people will find this context).
Plus it helps him getting rid of 399 open issues.
In retrospect I guess I'm a bit disturbed that I even think about these things in terms of fame/popularity, when it should be solely about tool capabilities and power for its users. :) Shame on me, back to reading FSF doctrine.
That being said, my first impression from the article was quite the opposite of yours. I thought it was great news for CoenraadS. He developed something so useful, that it was integrated into the core. That's impressive! The plugin is obsolete now, but in this case it looks like "mission accomplished".
Of course this move would look different in other app/extension ecosystems where extensions are monetized and/or not open source.
- Make an extension to solve a problem that probably should have been in the core
- Product owners notice and realise it should be in core
- Work with extension developers to get it in to core
- No longer need to work on extension
That's just silly.
> I follow this thread with interest, my extension is something that grew a bit out of control, and I grew tired of maintaining it.
The quote above is from the maintainer. Looks like he is happy to let it go!
Quote source: https://github.com/microsoft/vscode/issues/128465#issuecomme...