Because the language works by textual inclusion and so a "translation unit" isn't really just your code this is much more likely to result in nasty surprises, up to and including ODR violations.

That's not really true. If you link a shared library that was compiled with -ffast-math, that will affect the entire program. https://moyix.blogspot.com/2022/09/someones-been-messing-wit...

Or even on a per function basis, at least with gcc (no clue about clang...)