Not a compiler expert, but I got confused when reading this, when is it available?
Not a compiler expert, but I got confused when reading this, when is it available?
There is a longer and more precise description of the problem as part of this FAQ of the generics proposal:
https://go.googlesource.com/proposal/+/refs/heads/master/des...
…which includes an example and the statement “It is a key design decision of Go that parsing be possible without type information”.
In Go, you can do this fairly easily with the go/ast package.
Then you take the AST and do something with it. That something can be compiling the code, but also writing some sort of tooling like a linter, or identifier rename tool, or generate documentation from it, or whatnot. When compiling the code you need the type information, for a lot of other purposes you don't really care.
It's pretty valuable to keep the parsing as simple as possible; it makes it easier to detect errors, improves the quality of the error messages, and makes it easier to write tooling. It also keeps the code a lot simpler, easier to understand and modify, etc.
Lexing/tokenization doesn't produce an AST; it produces a token stream. Parsing (which may or may not be preceded by lexing) produces an AST.