For example, in Firefox, I use the dark theme for the UI, but that’s completely unrelated to the content, and prefers-color-scheme matches the content, not the chrome. Content I keep normal, light.
It’s quite possible (not uncommon, even) for the active tab to have a light background and inactive tabs to have a darker background. (A decade ago, this is what would happen by default on the default Windows XP theme, though it was the XP blue rather than anything dark dark.)
Or Chromium: like Firefox, it has themes; but beyond that, incognito window chrome gets a dark background.
Admittedly, (prefers-color-scheme: dark) matching makes it almost certain that you’re rendering against a dark background of some form, but it not matching tells you absolutely nothing about the background.
The fact of the matter is that your favicon could be rendered against absolutely any colour, and you need to make sure your icon will work on very light colours and very dark colours, and not be too bad on anything in between.