Typing a letter and deleting it still counts as a modified file for me with VSCode, unless there's some platform-specific behavior or setting I'm missing. (I tested on Linux.)
I just tested (VS Code version 1.55.0 on Debian Bullseye, Linux 5.10.0-5-amd64) and it DOES show as modified to add then delete a character. I wonder if there's a setting somewhere or such.