Hacker News Reader

Comment by craftkiller | Hacker News Reader

I'm not the GP, but personally I'd consider syntax highlighting perfectly solved if it was handled in the language server. That way we could have highlighting performed by 100% accurate parsers instead of approximations. It wouldn't be as fast as in-editor simple tree-sitter highlighting, but I don't see any issue with my text appearing uncolored at first and gaining color as my editor polls the language server as I type.

Currently language servers support semantic highlighting which is close, but as far as I can tell, it seems meant to be supplemental to in-editor local highlighting rather than a full replacement.

View on HN

You would only get the top half of your code highlighted because a compiler doesnt usually continue when encountering syntax errors due to partial code

That's sort-of already the case whenever we start a string in any language that supports multi-line strings. You type the first " and then the rest of your file gets re-interpreted as a string. It is annoying but I still think its a worthy trade-off, because I spend a lot more time reading code than being mid-edit with invalid syntax.

Syntax errors are not limited to just typing a quote

I never said they were. I'm just pointing out that we already experience the same problem in the case of quotes without the world ending, and I don't think expanding the problem to all syntax errors will make it much worse.

It would make it significantly much worse. Because every incomplete statement that you're just writing would break all syntax highlighting right after your cursor. And if you use any modern editor or IDE this doesn't happen.

If the code was valid at some point (e.g. when opening it) it could remember the old colorings of the rest of the code.

Though handling opening and closing a string and still ending up in an illegal state would need some more logic.. Maybe not worth it.

> If the code was valid at some point (e.g. when opening it) it could remember the old colorings of the rest of the code.

If you continue down this line of thinking a few more steps, eventually you produce tree-sitter.

> You would only get the top half of your code highlighted because a compiler doesnt usually continue when encountering syntax errors due to partial code

Sounds like a feature - don't need to scan for tiny squiggly red lines, or examine some tiny font output in a status bar somewhere.

I'd love it.

> You would only get the top half of your code highlighted because a compiler

Your comment implies a language server taps into the language's compiler/interpreter. That is a popular misconception. Almost all LSP severs don't actually use the compiler backend of the language they are servicing. All the LSP server implementations I've seen at least just use a static parsing approach (or even worse i.e. just a tokenizer) similar to what Treesitter does. It is just not limited to a single file.

That's why I still think Treesitter has the potential to not only improve syntax highlighting but also to simplify language servers or even -- in the long run with some extensions of the the language modes -- replace it altogether.

But some do. Most notably clangd. Still clangd tries parse broken code using recovery heuristics (which can still be useful in a compiler to produce decent error messages).

> Your comment implies a language server taps into the language's compiler/interpreter. That is a popular misconception.

But it is a misconception which framed the discussion, not a misconception of the answerer. The claim was that LSP was

> performed by 100% accurate parsers instead of approximations

But you want syntax highlighting performed by approximations, because when editing your code is only approximately valid.

It would be even better if the code was always valid because we only edit the AST itself, like what paredit does for Lisp modes.

No.

(The code being valid s-sexps doesn’t even guarantee it’s syntactically valid after macro expansion, but even if it does this is a bad idea.)

One interesting instance of semantic highlighting occurs in the Lean 4 language. It has extensible syntax whose parsers can be expressed in the language itself. Since parsing and evaluation are intertwined, highlighting can only really be effectively achieved in an LSP lest you re-implement the entirety of Lean.

Reply on news.ycombinator.com