Exactly.
I can't speak for Log4J as I have no experience, but I do know that anyone sensible agrees that OpenSSL is a pile of bloated dog poo.
OpenSSH, as I've already said, is written by the OpenBSD crew. They have a track record of having OCD about writing secure and correct software.
As I said mate, go look at CVE.
Its been YEARS/DECADES since the last serious vulnerability in OpenSSH.
That's despite being one of the most widely deployed pieces of software on the internet.
When an OpenSSH zero‐day gets released, you can bet your bottom dollar people will be scanning full port ranges for SSH servers. And only one of them has to find you.
If that’s the scenario you’re worried about, don’t rely on obscure ports. Run your sshd behind a VPN.