That number is counted whenever citizens travel to/from Hong Kong and Macau. There are people making day trips to/from Hong Kong every day.
For this reason people in Hong Kong have not passed the stricter scrutiny applied to people who visit the mainland, so they have to be re-checked at the border.