Sorry, I'm confused, and I must admit that I haven't tried getting into China illegally (from Hong Kong or anywhere else for that matter), so I might be missing something.
I understand that HK and China share a land border. But AFAIK that is not a border you can just walk over - there's a river and a fence and guards and all. You can cross the border at designated border crossings, by foot or train, but you will have to go through HK immigration to leave HK and then mainland immigration to enter China.
To go from HK to Vietnam or Laos, you'd have to take a plane from HK, in which case you also have to go through HK immigration to leave HK.
It remains my impression that HK immigration, which one has to go through when departing HK (at the airport or the land border or the train to China or, before someone mentions that, the ferries to mainland or Macau), did not let him leave HK.
(As for pledging allegiance, again, the problem for that particular Chinese citizen was not entering China, the problem was leaving HK.)
Thus, I remain puzzled by the suggestion that he "just walk past to Shenzhen" or "get on a train". Swim over, sure, but walk or take a train?