It depends more on what platform you're targeting. On GitHub, cross-reference is easy enough with links (ala hyperlinks) using relative paths. On LeanPub, I can add notes and references with their small extensions.
For most documentation, though, the basics are good enough. For some projects, yes you may need more structure, and either building a custom docs site or more heavily customized documentation is a good idea.
But even for medium-sized doc sets, Markdown still works great (I maintain a Docs site for Drupal VM[1] using Markdown with Read the Docs).