In my tests with STB-style headers written in C between a few thousand to a few tens of thousands of line of code, the compilation speed difference between including the header with disabled implementation, versus including with the implementation section removed from the file is absolutely negligible. This is because the implementation block is already skipped in the preprocessor, and never parsed by the compiler frontend.

Again, this is different from typical C++ single-header-libraries (e.g. pretty much all C++ stdlib headers) where the implementation code is inlined or in template code.