Not disagreeing with your point, but it couldn't be a compiler optimization, could it? The compiler isn't able to infer that the class will not be inherited anywhere else, since another compilation unit unknown to the class could inherit.
Possibly not in the default c++ language mode, but check out -fwhole-program-vtables. It can be a useful option in cases where all relevant inheritance relationships are known at compile time.
Which is good, but may not apply. I have an application where I can't do that because we support plugins and so a couple classes will get overridden outside of the compilation (this was in hindsight a bad decision, but too late to change now). Meanwhile most classes will never be overriden and so I use final to saw that. We are also a multi-repo project (which despite the hype I think is better for us than mono-repo), another reason why -f-whole-program-vtables would be difficult to use - but we could make it work with effort if it wasn't for the plugins.
I assume it could be or is part of link time optimization when compiling an application rather than a library?