"does no harm" might be an exaggeration here. It's doubtful you'd have a functional chip. But the fine structure remains, clearly.
Toss some nitric acid in there, forming aqua regia, and you'll dissolve the whole chip including any gold.
I have a couple of fully functional examples on my desk that say otherwise. If you actually want to keep the device usable to the point that you can still solder it to a board, then you normally preserve most of the package, bond wires, and leadframe which requires more care during decap. For this particular specimen we didn't bother because we just wanted the ROM.
Here's an example of a fully functional decapped device soldered back to a board: http://i.imgur.com/UebB3FO.jpg
My lecture notes at http://security.cs.rpi.edu/courses/hwre-spring2014/Lecture3_... go into more detail on various methods, chemical and otherwise, for decapping with and without preserving the leadframe.