If you just throw the attestation away (or do something morally equivalent, like log it and then only read the logs once in the first week when you're interested) then you annoyed the user (this prompts me to permit attestation, and explicitly refuse) for no purpose. Hopefully in authentication it's not necessary to explain why higher friction is not good.
If you use the attestation to make decisions (e.g. allow Yubico products, refuse everything else) now you've got the burden of those decisions. You need to stay on top of new products, evaluate them and decide what's acceptable. For a private site you can control the burden by e.g. requiring employees to use the company branded Yubico authenticator, and just making sure you allow any new batches Yubico ships when you buy more, but for a public site this is definitely a real piece of work for a small team.
If you're doing that work - to what end? Do you have the information to make wise choices? I think the user is far more likely to know whether the authenticator suits their purpose than you. They may know that - because they're constantly working with abrasive chemicals using their bare hands - their fingerprints are trash and so the fingerprint reader you recommend doesn't work, they go with the PIN entry device. Or despite your preference for USB-C they work all day with servers that only have USB-A ports so they want a USB-A authenticator.
For a private site you actually might have policy coming from above that's most easily implemented using attestation. Like stupid password policies, at some point you've told the Powers That Be what you recommend, they've ignored it, you just do what they said and roll your eyes. On a public site it's just crazy.
Attestation compromises FIDO's "Relying parties shouldn't care what the authenticator is" design because now you do care what the authenticator is. So as a purist that's enough reason to rule it out.