I changed my port from 22 to a higher one and the user attempts are completely gone.

Only nuisance is that the higher ports may be blocked, for example my uni blocks my new ssh port so I can't connect to the vps when I'm on campus.