Drift, as the other child comment mentions, but also loss of version control. I do not want to have to trawl through someone's shell history to figure out what they changed, nor do I want to have to redirect `kubectl get foo -o yaml` output into diff.

If everything is in code, and you have a reasonable branching strategy, it's much easier to control change, to rollback bad merges, to run pre-hooks like security checks and configuration validation tools, etc.