Did you try something else before? We are all sitting in a Jitsi Meet conference all day and it works fine so far. We even added a "private call" button on the video tiles in order to quickly initiate a 1-on-1. How is the "select person to talk" feature working for you?