I've always considered sci-fi to be a genre where authors re-imagine how society works or what it means to be human - sometimes highlighted by comparisons to aliens and their societies, transplanting a culture to a new environment, or imagining the effect advancing technology. I think Heinlein did a pretty good job of those things. I agree it wasn't always well done though.
I do wish his plot lines flowed better. They sometimes make quick jumps as if he was in a hurry or didn't want to take the time to make events more fluid.
Heinlein was definitely weird, but he wrote science fiction. You should sort of expect it to stretch the Overton Window.