This is a wild idea because it's false. It's hard to be a part of any culture if you don't speak the language. All you've said is that knowing a language doesn't make you part of a culture, which nobody would disagree with in the first place
If you say "any culture" then sure you can be always part of some (sub)culture if you narrow it down, it just matter how much you are going to narrow it, in the end you can be part of culture of foreigners living in specific country.
Have you ever lived in Switzerland? Because I grew up there and if there's ever a group of people who don't accept you if you don't speak their dialect it's the Swiss Germans.
Swiss languages are regionally segregated, meaning that if you want to settle in to Zurich, you'll need to speak German (and at least understand Swiss German), if you want to live in Geneva you'll have to know French and you won't get anywhere in Lugano without knowing Italian.