Hacker News Reader

Comment by williamcotton | Hacker News Reader

I looked at the dependencies and noticed that it wasn’t using an existing Protobuf parser which means they reimplemented the parser from scratch. Perhaps due to a lack of error recovery in the existing implementations? I don’t have the energy right now for further inspection.

It is definitely best to reuse the parser for the runtime when implementing an LSP but to do so properly means implementing the parser itself as a standalone library. Even better is shipping the semantic analysis as well!

Implementation drift is definitely an issue.

But great project anyways, just wanted to put my thoughts on the matter into the conversation!

View on HN

I think its worth considering but I don’t agree it’s always best to use the language parser.

A language parser should be correct. A LSP parser should be fault tolerant. My understanding is you cant have both.

It absolutely can be done [1] but it is a lot of effort to do high quality error recovery. It's easier in languages with natural "synchronization points" [2], harder in languages which don't have them.

Having built a lot of protobuf tooling, I'd estimate that protobuf largely falls into the former camp; most of your time working with .proto files is operating on fields which terminate using `;` at the end of the line (though multi-line is also possible).

[1] source: I've done it for SQLite SQL at https://github.com/LalitMaganti/syntaqlite/

[2] e.g. SQL naturally has this at statement and expression boundaries which covers almost all of the cases people care about.

Adding to this, rustc's parser is both correct and fault-tolerant, yet rust-analyzer ended up building its own for different reasons (needing a CST and not an AST).

I settled on starting with a CST using rowan, doing semantic analysis, having an agnostic editor services, exposing both an LSP interface with tower, and a WASM interface (for Monaco in the browser [non-LSP!!]) and having the runtime binary doing the interpretation/compilation with the same pathway, with a slight jog, as the editor services. Phew!

It is my understanding that basically all LSP implementations use tree-sitter because of its incremental rebuild capability, and this requires re-implementing the parser.

That's just not true. rust-analyzer does not use tree-sitter. Neither does gopls. Nor Pylance. Or tsc. In fact I don't know if there is a single popular LSP that uses it (but there probably is).

Many editors use tree-sitter, but that is separate from the LSP.

buf is a parser / compiler as an alternative to protoc https://buf.build/docs/migration-guides/migrate-from-protoc/

Ah, so does the LSP just call out to the Buf binary?

The lsp is a subcommand of buf, from the page: `buf lsp serve`

The LSP is one of the subcommands provided by the buf binary. My LSP configuration for protobuf is `buf lsp serve`.

Makes sense, thanks for the clarification.

Reply on news.ycombinator.com