That was initially confusing to me as well, but it isn't that bad. If you open a .png (or similar) and edit it, there's a button in the File menu (I believe shortcut ctrl-shift-e) that exports directly to the original filename.
After ctrl-shift-e I have to hit enter in response to different pop-up dialogs three times, and then still answer a fourth confirmation when I quit gimp. If I'm editing a bunch of files that's not fun. With the Saver plugin I have it set up so I just hit ctrl-s like I used to, and can exit without being bothered.
There isn't one. Ctrl-shift-e is "Export As" which opens a save dialog. Ctrl-e doesn't show up until you save the file manually at least once.