I'll remain skeptical about OMB's ability to know about all the datasets from federal agencies, because as you said, there is a wealth of government data that is "public" but obscure.

Another example is the Department of Defense's budget justification data. The PDF documents that go into the annual President's budget are made public and are easily found on agency websites. They look like this:

http://www.saffm.hq.af.mil/shared/media/document/AFD-150130-...

What the public would have no way of knowing, though, is that attached to those documents are files with the extension of *.zzz. Those are really zip files, but need to be renamed because DoD computer systems tend to view anything with a zip extension as toxic. Inside those zip files are the xml files used to create the PDF document. The data is both clean and comprehensive.

I'm writing a set of scrapers that will download the publicly available PDF files and extract the XML, so if someone wanted to import them into their own database for querying, they could. I really wish that weren't necessary, though.

Edit: I should mention that those xml attachments only exist for the Procurement and RDT&E budgets, which account for about a third of total DoD spending.