What are you doing here? SSH generally runs on port 22. You move it to an alternate port to thwart a naive intruder's attempt to establish a connection to a service running on that port.
You are still running SSH with the same old setup. If it was insecure before it remains insecure. The only thing you've done is "obscure" how to connect to it by using an uncommonly used port.
Why all the hate on "Security by Obscurity" It's a perfectly valid thing to do. It will definitely defend against a host of intrusion attempts. Its only cast in a negative light when it's the ONLY thing you do or if you consider it sufficient in terms of security.
FWIW the key analogy is broken. Hiding a key to your house in a plant is a form of "Security by Obscurity". Anyone that finds the key can open the door. Thats not very secure. So we hide the key so no only people that know where it is can find it. If an intruder found the key they'd immediately have access to the house. We rely on the obscurity of its location to provide whatever security it has to offer.
You could achieve the same affect by having a hidden door and leaving the key out in the open or hiding both. What use is the key if the location of the door it opens is "obscure". This works because access is defined by knowing the location of the key and the door simultaneously.
Both are examples of "Security by Obscurity"