Testing both your code and the message system is exactly what you want, since if the message system is broken in a way that upstream didn't catch, you want to learn about it during testing and not production, if possible.
You can get the best of both.
Also, when your integration suite (or customer reports) discovers that the payment gateway fails on punctuation, another unit test should be added with a mock that responds the same way, and an E2E test added with punctuation in the input data and a failure expectation.
What makes you so certain you would have included punctuation in the input data if the test had not mocked the gateway?
Unit tests are really for purposes of documentation. They show future programmers the intent and usage of a function/interface so that others can figure out what you were trying to do. Mocking is fine here as future programmers are not looking to learn about the message system here. They will refer to the message system's own documentation when they need to know something about the message system.
Integration tests are for the more classical view on testing. Beck suggested this is done by another team using different tools (e.g. UI control software), but regardless of specifics it is done as a whole system. This is where you would look for such failure points.
However, depending on the industry, code coverage is a valuable tool to gauge the maturity of the software baseline and burning down software execution risk. One example of this is Airworthiness or Safety Critical Code.