It clearly makes a difference for me. My servers which have SSH listening on port 22 have countless failed login attempts. Those running it on another random port have almost none.
On the other 20+ boxen with ssh on port xxx22 the logs and f2b rules are much smaller - which means less hassle for the admin.
And on the boxen with services behind WG there is zero noise.
Naturally we're using keys only with all this. The reduced noise in the logs/rules/firewall are very handy.
[0] https://www.youtube.com/watch?v=xkrMsPiqG6M&feature=youtu.be...