This! Forget about average user. As a technical user too I don't know how I would compare fingerprints every single time without making a mistake. I could install software or write my own to do this on desktop but what would I do on cell phones?
And TOFU requires "trust" on first use. How do I make sure that if I should be trusting the website public key on first use? It doesn't seem like any easier to solve than PKI.