My understanding is that you are correct. If the repo and all of its forks stay private then the only people that would be able to view them are people who have permissions to access those repos.