More context: the server in question is running gitlab in a docker container, and users should also be able to SSH into the server. Now I have to run either gitlab or the host sshd on a different port, which confuses users and often enough leads to problems with git clone (the syntax is [git@git.my.tld:22222]:path/torepo.git, which is accepted by bash but people insist on zsh and other shells which break on the brackets)...
git clone ssh://git@gitlab.example.com:2222/path/torepo.git
Alternatively, each user could put into their ~/.ssh/config:
Host gitlab
Hostname gitlab.example.com
Port 2222
User git
(In this particular scenario a bastion host probably isn't useful, since you'd have to copy the SSH keys out of gitlab into the bastion and make sure you get the management/security right. A web search for "ssh forward git user" turns up a number of people struggling with your problem; none of them seem to have entirely satisfactory answers.)
The gist of it is that it looks like you probably have "git.my.tld:22222" set somewhere in your config file, which should be removed and replaced with:
gitlab_rails['gitlab_shell_ssh_port'] = 22222
yep, exactly. ssh normally does not need a portnumber if you are running gitlab-ssh on standard tcp/22.