Text -> parser -> AST -> job done. If it's any different in an IDE vs anything else I'd like to know how.
It needs to be able to handle:
printf("hello");
prin
printf("He
The solution is to pick synchronisation points to start parsing again, i.e. ; at the end of a statement or } at the end of a block.
> How can you be sure that that } is the end of a certain defined block
If it's not in a string, what else is it but a typo? If a typo, it fails to parse but so long as it doesn't crash, fine.
int main() {
int x;
{
int x[];
// <-- caret here
x += 42;
}
Yes, of course parsers can account for the indentation in this case. But more generally this kind of parsing is sensitive to a series of edit sequences, not just the current code. This makes incremental parsing a much different problem from ordinary parsing, and also is likely why ibains and folks use packrat parsing (which can be easily made incremental).
Make still does this. (That's the "Stop." in the famous "* missing separator. Stop.") Many errors in Python still do this.
As late as 2010 I still saw some major C compilers do this.
99% of the toy compilers written for DSLs do this, or worse.
Good error recovery / line blaming is still an active field of development.
True. But let's get terminology straight: that's not a compiler science, that's parsing science. And it's no more compiler science than parsing a natural language is.
Parsing - formal theory like taxonomies of grammars, and practical concerns like speed and error recovery - remain a core part of compiler design both inside and outside of academia.
There are varying techniques to improve error recovery for all forms of parsing but hacked-up recursive descent (certainly the most common kind of parser I still write for my hacked-up DSLs!) have poor error recovery unless you put in the work. Most LR parsers are also awful by default.
When I was in university most focus was on LL and LR parsers with no discussion of error recovery and more focus on memory usage/bounds than speed. I also have no idea how common it is to teach combinator-based parser grammers these days; stuff like ANTLR and yacc dominated during my studies. This would add another level of unfamiliarity for students going to work on a "real compiler".
I think this needs to be qualified. I don't think a packrat parser is going to beat a deterministic top-down parser. Maybe the packrat parser will win if the recursive descent parser is backtracking.
Packrat parsers are equivalent to arbitrary lookahead in either LL or LR grammars.
See https://youtu.be/Jes3bD6P0To
Tree sitter is based on LR parsing (see 23:30 in above video) extended to GLR parsing (see 38:30).
I've had enough of fools on HN posting unverified crap to make themselves feel cool and knowledgeable (and don't kid yourself that you helped me find the right answer by posting the wrong one). Time I withdrew. Goodbye HN.