Hacker News Reader

Comment by duttish | Hacker News Reader

I've never looked into LSP, how would you design it?

Well for one I wouldn't design it so both the LSP and the editor need a synchronized view of the underlying file.

One of the largest issues of LSP is that each language implementation needs to do everything separately, and that means each language will work differently.

Imagine if all the editing tools in microsoft word were specific to the language you used, and if you mixed German and English, each had different tooling.

Now if you mix Rust, HTML, JS, and CSS, they'll all have separate tooling, seperate "go to definition", and separate refactoring. Worst of all, none of them can see the definitions of the other ones.

So what you'd actually want to do is parse each language into your AST, with proper annotations as to what is what, and have all the "go to definition", the UI rendering, highlighting, refactoring, etc all done generically by the IDE ontop.

Which is much much closer to how Jetbrains IntelliJ does it, and why their tooling can handle "find usages" on an HTML element to find matching querySelector in .js files and matching selectors in .css

And if you subscribe to the AI stuff, you'd also want your AI to operate on this AST so it can learn skills that generalize across all languages.

Wouldn't a normalized protocol based on AST vocabulary and tree operations make more sense ?

That would require the editor to be able to build an AST - defeating the point of LSP, because now the editor needs to know every possible language again

I meant the language server builds the normalized, lossless AST and gives it to the editor. Changes in the AST can be provided as events/diffs to the editor. The normalized, source-mapped AST is effectively the stable language protocol. (Possibly given in a columnar, database form via Apache Arrow, but thats impl specific). Was discussing alternative architectures with a collague since even the best language servers really suck on large projects.

Start with synchronous function calls instead of JSON. Microsoft knows how to do that - they invented COM and OLE. Function calls enable whatever data sharing is necessary to maintain a coherent view. Imagine trying to do OLE with JSON - just wouldn't work. (Does OLE still exist?)

Stupid idea.

So the LSP crashes and/or goes into a runaway memory consumption loop. And your main application dies with it.

Or what if you want, you know, to be able to use the same LSP from TWO different applications at the same time?

Never mind issues with other managed runtimes not expecting to deal with something else in their address space.

Moreover, it's not just stupid, it also does not actually solve _anything_.

A synchronous function can also have obsolete indexing information if it races with the code updates.

But it doesn't have to, because you call AboutToUpdateCode and the LSP doesn't return until it's safe, and then you update code and call DoneUpdatingCode.

Yes, it does because the physical code update is asynchronous. Or you'd have to do a roundtrip for every little edit, resulting in unusably slow experience.

I don't know what you're talking about. What is "physical code"? A function call roundtrip costs nothing, that's the reason we are saying it should be function calls.

TCP roundtrip overhead over localhost costs approximately nothing.

But running the LSP itself is slow. It's not instant, and you can't run it in a blocking way for each keypress.

A function call costs nanoseconds. You can sprinkle them literally everywhere. A JSON-formatted TCP round trip to localhost costs, I'd guess conservatively, at least ten microseconds, 1000x slower.

OK. Here's my simple implementation of an LSP:

> func doLspStuff() { sleep(60000); }

Now sprinkle them after each keypress. Do you see the issue now?

I feel like MS actually learned their lesson with synchronously integrating the language intelligence into the IDE. Old versions of VS would hang or crash based on bugs in the language tooling trying to provide intellisense. You'd restart and it'd work fine till you hit some other weird edge case. Generally this settled to a level of rare-bugginess where you were happy enough with the advantages not to go back to Emacs/VIM, but still annoyed at the occasional restart needed.

In no way does this mean LSP is a perfect solution, but anything synchronous would be a step backwards.

That doesn't really fix anything. The filesystem is fundamentally a racy shared data structure. If you made the entrypoint API synchronous, any half-way decent editor would shove the LSP queries to the synchronous API into a different thread, because "Do Not Block the UI Thread" is a fundamental principle of good UI programming.

Once you get past that, JSON-over-TCP is just another kind of asynchronous RPC mechanism, one that has the advantage that you can build it in just about any language with out-of-the-box tools. Trying to make a plugin system or a COM or CORBA or OLE based system really cuts out the ability to build language servers in most languages, because you have to be able to build the code in just the right way.

Big assumption that the editor, which is required to make calls synchronously, would make them on a different thread. Isn't correctness more important than avoiding plugin calls in the UI thread? The good thing about having an actual function interface is that you can tailor each one to its actual needs, instead of insisting high-overhead async is okay for everything. It's like GDI vs X11.

I don't know about you, but I do not want to block on every keystroke. Program analysis can take time, asynchronous feedback is a natural fit. None of the features I use from LSP should be synchronous.

I can almost guarantee you that a synchronous API would yield a much more complicated design just because many operations can not be expected to reliably work within a few milliseconds.

You want to rename something across multiple files? Well now you add file system overhead and blow straight past reliable frame timings. Good luck waiting for that operation to finish. Of course you could make the API beginRename, and queryRename, or whatever you fancy to see if the operation was successful, but now you're back to what you wanted to avoid: an asynchronous API. Do mind that the example is actually one of the better cases, as many things you might want to do with a codebase are actually more expensive. You will feel the hiccups from waiting in the UI thread and you will loathe the program for it.

Why do you assume renaming must be completed when Rename returns? Rather, the LSP must have a consisten snapshot when AboutToSaveFile returns.

I assume that the result of the rename operation must be known. I.e. if it was successful or not, or which files were affected. That implies 2 function calls. One to start the operation, and one to query its result. That is, by definition, already an asynchronous interface. The only other (i.e. synchonous) way to achieve remote procedure calls is by blocking until it's finished.

If you insist on showing the result of a rename on the next frame, then yes, it must block the next frame until it's done.

Would you write your editor that way if it didn't use LSP but was specific to one language? If not, but you would write it that way with a language plugin, why? You still have to write a good product. You can't blame the plugin API for everything you get wrong.

Actually, COM is inspired by DCE/RPC and the initial versions had some similarities.

Parallel to that, IBM had SOM on OS/2, which was even better allowing for metaclasses and proper class inheritance, it was the key mechanism between Smalltalk and C++ on OS/2, where Smalltalk enjoyed a role similar to .NET on Windows nowadays.

OLE naturally still exists when using Office natively on Windows, other vendors seem to have forgotten about it.

COM's role on Windows has grown since Vista, and the Windows team redid many of the Longhorn ideas originally implemented in .NET into COM/C++, with WinRT being an evolution of COM.

I've never heard of anyone calling SOM 'better' at anything this century. I came across it when it was the foundation for OpenDoc at Apple, your comment brought back many bad memories

Compared with COM's design, it was much better.

Classes and metaclasses sound like architecture astronomy here. COM is fundamentally just a standard ABI and a way to look up DLL paths in the registry. There are more layers on top of that, but that's the basics and all you need for a plugin architecture.

That applies to SOM just as it does to COM.

Interesting to complain about one and then ignore exactly the same boilerplate for the other.

Leaving aside the fact that Microsoft's tooling for COM has already had multiple reboots between VB OCX, MFC, .NET Framework RCW/CCW, .NET ComWrappers, ATL, WRL, WIL, WinRT, each one with its share of astronomy.

No one uses the bare bones vtbl and nothing else.

You assume I know anything about SOM.

Everything uses the bare bones vtbl. That's how there can be so many things, they're all using the bare bones vtbl. Obviously there's application specific stuff as well because a bare bones vtbl is not useful by itself but that's like saying if TCP is so good why is there HTTP?

Because I thought this was a further discussion and did not notice another username, sorry about that.

The point being that beyond the plain vtbl, SOM was much better designed than COM, in the sense of overall architecture for application development.

> Start with synchronous function calls instead of JSON.

LSP is a function call protocol. So that is already done?

Reply on news.ycombinator.com