All code in the header leads to a lot of duplicated work for the compiler, which means slow build times. You should avoid this unless it is absolutely necessary for performance to inline methods.
All code that would otherwise live in .c files is between an #ifdef/#endif block which is only activated in a single compilation unit in the whole project.
Not sure how this approach would lead to redundant "function definitions" which would need to be deduped by the linker. The only overhead is in the preprocessor for skipping the implementation block, but that happens pretty much at IO speed - it's not comparable with the parsing overhead in typical C++ headers with template and inline code.
Still, it seems like an ugly kludge to cope with a breathtakingly antiquated way of doing things.
What ever happened to keeping it simple?
I think it's mainly a fix/workaround for the breathtakingly antiquated build systems in the C/C++ world ;)
In the end, STB-style single-header vs. a single .h/.c pair is not all that different, both are equally straightforward to integrate into a project.
The actual problem are libraries made of dozens/hundreds/thousands of header and source files and coming with their own complex build system setup.