There can be plenty of friction depending on your use case.
https://developer.apple.com/documentation/virtualization/run...
I shy away from x86 images too, even thought I use docker.
Some challenges included Docker images being arch specific, old Terraform modules lacking ARM support (forcing an upgrade we'd otherwise defer), and reduction in tooling we'd consider for production. We generally worry about arch specific bugs, although I don't believe we've seen any (other than complete failure to run certain tools).