Documentation sucks because it's hard/impossible to do well. Every change to the code can trigger cascading changes in documentation and there's no compiler to tell you which things need to change.
If we can't even keep the README up to date, when that's in the same repo as the change, how on earth are we going to be disciplined enough to go update the docs.
And the occurrence of the word "disciplined" hints at the real reason: it's a lot less fun to write docs than write code, so we don't unless we're forced to by circumstances or by management.
That's not really what we're talking about here though. Like you said, it's the concepts and informal stuff that's the problem.
I've found autogenerated stuff to only be marginally more helpful than just reading the code in the first place.