A syntax error might be exactly what they’re looking for e.g. they’re feature testing a new bit of syntax or a compiler extension.
> so perhaps filtering based on the type of error would be best done as part of the build system functionality for doing the feature checking.
Which would require every compiler to have detailed, consistent, and machine-readable failure reporting.
Handling the output from several of the most recent GCC and Clang versions would probably cover the majority of cases, and add in MSVC for Windows. If the output isn’t from a recognized compiler or doesn’t match expectations, the only option is falling back to current behavior. Not ideal, but better than the current status quo…
A mitigation here would be to make result of autoconf only provide instructions for humans to change their build config, instead of doing it for them silently. The latter is an anti-pattern.
FWIW, the approach you propose is how the UI tests for rustc work, with checks for specific annotations on specific lines, but those have the benefit of being tied to a single implementation/version and modified in tandem with the app. Unless all compilers could be made to provide reasonable machine readable output for errors, doing that for this use case isn't workable.