Hacker News Reader

Comment by vyaa | Hacker News Reader

I think its worth considering but I don’t agree it’s always best to use the language parser.

A language parser should be correct. A LSP parser should be fault tolerant. My understanding is you cant have both.

It absolutely can be done [1] but it is a lot of effort to do high quality error recovery. It's easier in languages with natural "synchronization points" [2], harder in languages which don't have them.

Having built a lot of protobuf tooling, I'd estimate that protobuf largely falls into the former camp; most of your time working with .proto files is operating on fields which terminate using `;` at the end of the line (though multi-line is also possible).

[1] source: I've done it for SQLite SQL at https://github.com/LalitMaganti/syntaqlite/

[2] e.g. SQL naturally has this at statement and expression boundaries which covers almost all of the cases people care about.

Adding to this, rustc's parser is both correct and fault-tolerant, yet rust-analyzer ended up building its own for different reasons (needing a CST and not an AST).

I settled on starting with a CST using rowan, doing semantic analysis, having an agnostic editor services, exposing both an LSP interface with tower, and a WASM interface (for Monaco in the browser [non-LSP!!]) and having the runtime binary doing the interpretation/compilation with the same pathway, with a slight jog, as the editor services. Phew!

Reply on news.ycombinator.com