The main reason I'm not so concerned is if they have physical access to the server they have root access anyway. At least this way I can see them logging in.