I don't think that's true. The underscore is valid in RFC4648, but the exclamation point is not.

And the RFC explicitly says:

    Implementations MUST reject the encoded data if it contains
   characters outside the base alphabet

So, unless I've missed something in the RFC (and I admit I only skimmed it), change_me! isn't valid in either traditional or URL safe base64.