I think the university must be risking something bad (such as legal consequences) if they felt that this type of obvious bad publicity was worth it. If they tell people to leave campus that’s as much as they can reasonably do.

The next step before wondering why someone used their keycard to access a building they shouldn’t be near could be just removing the access?

The best way to get people to leave has to be removing reasons to stay, for example access to campus resources.

Accusing students of lying does not avoid covid exposure, though? It just creates bad will (and possibly legal issue).