Instead, do any of the following:
* Compile normally without -Werror, but have a test suite which compiles all your tests with -Werror. Make sure to only accept patches which don't break the test suite (probably using some CI system). If you don't want actual tests, a "test suite" which just makes sure `make clean && make CFLAGS=-Werror` doesn't fail would suffice.
* Use -Werror for regular builds, but enable only a fixed set of warnings, not -Wall or -Werror.
* Make your build system record warnings, and complain about them for every single build, not just the builds which happen to recompile the files which trigger warnings. This doesn't _guarantee_ that your project is warning-free like the other two do, but I imagine a lot of the reason projects end up with warnings is that people just see the warning once, and then never again until the file has to be recompiled.
I can't count the number of times I have wanted to use some project, but then had to dig through the project's Makefile or autotools m4/shell soup or gn files or meson files or cmake files or ad-hoc compile script or some other build configuration, just because my compiler is newer than the author's.