I lived in Germany for seven years and by the end I was fluent. B1 is a very low bar to pass, I know because I've done it.

Sorry to say, but the rule is fair. If you want to be a permanent resident, put in a little bit of effort to integrate into the country that you would like to call home.