Even then fingerprinting would still present an issue, even without explicitly logging in, with most browsers.
For example: https://coveryourtracks.eff.org/
Also have a look at this: https://www.amiunique.org/
So you might need to have a browser that lies and presents configuration information that is common enough not to be unique, probably an OS inside of a VM might be one of the possible starting points. Outright denying access to some of that might actually help identify you, but pretending to be a common setup might not even work that well.
I'm frankly not sure whether privacy on the web is even truly possible nowadays, at least without a lot of effort. Even with a VPN, I treat the web as something that is more or less "spying" on me regardless, in the metadata collection and storage sense.