Good catch, @bsldld. When it was just two of us experimenting with this, that used to be the case. It's taken care of now.
Regardless of how many different sets of people are watching the same youtube video, it would still be a private screen where only people with your family's unique co-watching URL can join or watch it.
That way the co-watching experience remains private. It will be a separate private screen for your family and theirs.
Thank you for bringing this up and letting me answer this for others too :)