That other programm is knockd, which needs to listen to all traffic and look for the specified packets.
Granted, that program is really small and could be easily audited, but that same time could have been spent on trying apparmor/seclinux + a good vpn and 2fa