On average, yes. But if you look on a voting map[0], you'll notice that the majority of those votes comes out of the former DDR. In many other areas, much less than 1 in 5 voted for the brownshirts.
[0]: https://www.tagesschau.de/inland/bundestagswahl/wahlkreiserg...