You can "Export" on any format :P
Now it's 1. Load PNG 2. Edit PNG 3. Export as PNG (retyping the extension) 4. Quit 5. Say that yes, you do want to quit even though there are unsaved changes (because there are not). 6. Worry about whether there really WERE unsaved changes after all.
I fully understand why graphics professionals prefer the XCF-only approach; what I can't understand is why the GIMP people flatly refused to let the rest of us use the software the way that suits us.
(or File menu + "Overwrite {filename}". I complain a lot about GIMP's UI but I don't really see a problem here.