The heat/consumptions and single-core speed aspects are not negligible.
In addition (although this is definitely something for a minority of users), I suffered blocking VFIO scheduling issues with AMD (A10) while I didn't with Intel. Even Ryzen seems to be immature for VFIO (but I didn't personally test this).