I believe the main issue is file syncing between macOS and the container host OS.
I don’t have a M1 mac, but on my Mac a standard Linux VM has no impact on performance or battery life. The minute I use Docker the machine become sluggish and halves (literally) the batter life, I now avoids is as much as possible.