The really significant upside that I think people tend to gloss over is that my pronunciation is pretty much perfect, because I’d learn to say things by just repeating what I’d heard. On the phone people can’t tell I’m not a native speaker.
Additionally, for really foreign languages (e.g. English vs Chinese) knowing in what order to place words and whether words are needed at all is of great help.
Both parts are needed.
It isn't like you think about grammar forever.
And by the way: I had formal language training, for adults. Immersive, even: I am in the target country, spent 15 hours a week in class with other immigrants, and was taught by a native speaker. Most of the learning was introducing new word groups and practicing speaking and listening. A little bit was grammar, though, for specifically the reason I listed.