We test human "reading comprehension" with the assumption that humans have an underlying world model.
This is starkly different from a machine whose only ability is information retrieval -- this program (and more advanced versions e.g Freebase) parse and restructure text to be more retrievable. But there is no comprehension because there is no world model.
You wouldn't call parsing a google search query "reading comprehension."