Yes, from a technical perspective you are right. However, from a conceptual perspective, you can't see any of the content in a server without being specifically invited. There is no global search. You are clearly either "in" or "not in" a server unlike, say, a Tumblr tag.