Courts navigate this fine line already even for their own courtrooms. There are restrictions on rights like time/place restrictions, for instance.