As I understand his argument, it’s “Changing port number add security, therefore it’s a good idea.” I think nobody argues that it adds security. The problem is that:

1. It adds very little security: 16 bits is not much, and the result is not 256 bits (say) of SSH key plus 16 bits equals 272 bits, but instead effectively still 256 bits, or 256+8×10⁻⁷³ bits.

2. The security it adds is itself bad (sent in cleartext, easily brute-forced)

3. These problems stand against the many drawbacks of this previously discussed (complexity, confusion, etc.).

And the final argument: If increased security is what you want, simply increase your key lengths and/or password lengths, and you will get much more than 8×10⁻⁷³ bits of security, without any of the above problems.