I wish all of you not to fall in the trap of ontologies. I worked very hard in this domain my conclusion is that all ontologies fail to scale eventually. I would recommend people in the field to go towards "perspectivism".
[1]: https://link.springer.com/article/10.1007/s11406-021-00371-1
See also his talk https://www.youtube.com/watch?v=ujMgQqp8YSY
Stuff like schemas and data dictionaries and reuse are chinese finger traps for us geeks. Exquisite problems we can't look away from.
I eventually decided to treat most data ingestion (ETL) as screen scraping. Honoring Postel's Law. Pull out the interesting relevant bits as needed. Ignore the rest.
There's still an internal model, natch. But it's the smallest, most obvious model to support my immediate use cases. Nothing more.
Any NLP application directly encoding knowledge (objects, perspectives, whatever) is going to have major scaling constraints, since it's impossible right now to encode human-level understanding automatically.
Perspectivism is the understanding that it's impossible to interpret semantic "knowledge" without knowing the limitations and implicit context carried with the fallible, partial transcription of the truth (set in a world that obeys quantum mechanics, for one thing) into words (which run to about 1MB tops before the author gets bored.)
Quantum mechanics actually says there's less information needed to describe X area of space than classical physics implies there is.
what does this mean in this context?
The reason for that is twofold:
1. Many of tools created for reasoning are research-first tools. Some papers were published about the tool and it really was a petter and more scalable tool than anything before it. But every PhD student graduates and needs to find a job or move to the next hyped research area 2. Tools are designed under the assumption that the whole ontology, all the instance data and all results fit in main memory (RAM). This assumption is de-facto necessary for more powerful entailment regimes of OWL.
Reason 2 as a secondary sub-reason that OWL ontologies use URIs (actually IRIs), which are really inneficient identifiers compared to 32/64-bit integers. HDT is a format that fixes this inneficiency for RDF (and thus is applicable to ontologies) but since it came about nearly all reasoners where already abandoned as per reason #1 above.
Newer reasoners that actually scale quite a bit are RDFox [1] and VLog [2]. They use compact representations and try to be nice with the CPU cache and pipeline. However, they are limited to a single shared memory (even if NUMA).
There is a lot of mostly academic distributed reasoners designed to scale horizontally instead of vertically. These systems technically scale, but vertically scaling the centralized aforementioned systems will be more efficient. The intrinsic problem with distributing is that (i) it is hard to partition the input aiming at a fair distribution of work and (ii) inferred facts derived at one node often are evidence that multiple other nodes need to known.
loose from modern single-node However, the problem of computing all inferred edges from a knowledge graph involves a great deal of communication, since one inference found by one node is evidence required by another processing node.
[1]: https://www.oxfordsemantic.tech/product [2]: https://github.com/karmaresearch/vlog/