There are levels of security, or more like levels of risk one can take on. For the home environment, I'd agree that for most wildcards aren't much of a risk. For an SMB, LE with individual certs may be ok. For a multinational bank or similar, yes absolutely HSM on an airgapped CA seems like a better approach.
Don't forget to update that hardware then. And to inform whoever inherits the infra to update te ca after 10 years.