A Review of the Semantic Web Field

Over and over, we see that the real world problem is gated on human effort to negotiate about the models and to do data cleaning and transformation. And, the best results almost always require that modeling, cleaning, and transformation be done with an eye towards a specific downstream consumer or analysis. We get tired of having to steer leadership back to reality after they buy into the snake oil suggestion that integration costs can be avoided and unknown applications solved.

The claim that RDF solves federation more than any other serialization format for structured data and models is about the same as fixating on JSON versus XML or YAML or Lisp s-expressions.

My thoughts on ontologies in general are that they can certainly be powerful, and I've seen them used in the past in rules engines that powered fraud detection applications.

In the SemWeb community, in the late 2000s, they successfully convinced a bunch of CIOs of massive organizations, especially in the US Federal Government, that a key to being able to centralize and federate all of their data, and save money on duplicative systems, they could simply have semantic mappings on top of every IT system's databases, and query this semantic layer. Ideally, they could eliminate duplicated data, so that all systems would get data from the "Authoritative Data Source" system instead of duplicating it locally in the application's database.

I'm sure you can immediately see why this is wildly stupid and unrealistic. Imagine what it would look like if every single piece of data that I can technically obtain from another source has to remain in that source, and that storing that data locally with my application specific data is forbidden..... Suddenly, there is a massive increase in I/O, drop in performance, etc.

The whole project taught me a lesson about the politics of academia, and how there is a segment of the population that is highly educated, and has learned how to manufacture work for themselves outside of academia by pushing for high-level government officials to implement programs based on their theories..... MITRE was a big part of this particular project.

My take with ontologies is building consensus is hard.

Tree Notation offers a solution to the problem of: what should we agree on for the encoding? I assume that simpler is better, all else being equal. Then Tree Notation is the simplest, in terms of the thing with the fewest pieces(tokens).

To get to Tree Notation, nothing was added, only stripped. I started with an existing notation and stripped away each visible syntax token that wasn't needed. Surprisingly, not one is needed. Not one quote, parens, bracket, colon, etc.

So now if we can get consensus around going with the simplest thing, we have got a way to agree on an whether we should use XML, JSON-LD, turtle, etc. The simplest thing works (which would be Tree Notation, or a close relative—someone can rebrand the notation but the idea is largely the same). This does not suffer from the 927 problem, as there are a few classes of things where we do have 1 new language that is mathematically superior and of a different kind than others (binary notation, for example).

So after you have agreement on that encoding, versioning and forking and merging schemas is dead simple (just use Git—in Tree Notation all changes are semantic and noise free).

So now we've solved what encoding to use for our ontologies, and we have a very fast and efficient way to collaborate on them (it's just plain text and git).

That brings us to a third advantage which is more theoretical. Tree Notation maps words/nodes to a 3-D representation. This means that there would be an X-Y-Z isomorphism with an ontology and the real world. I don't really know where we go from there, but at least by this point we've moved the semantic web idea a lot further and can start looking at the next realm of possibilities.

Talking about RDF is absolutely meaningless without talking about Serialisation (and that includes ...URGH.. XML serialisation), XML Schema data-types, localisations, skolemisation, and the ongoing blank-node war.

The semantic web ecosystem is the prime example of "the devils in the detail". Of course you can explain to somebody who knows what a graph is, the general idea of RDF: "It's like a graph, but the edges are also reified as nodes." But that omits basically everything.

It doesn't matter if SparQL is learnable or not, it matters if its implementable, let alone in a performant way. And thats really really questionable.

Jena is okay-ish, but it's neither pleasant to use, nor bug free, although java has the best RDF libs generally (I think thats got something to do with academic selection bias). RDF4J has 300 open issues, but they also contain a lot of refactoring noise, which isn't a bad thing.

C'mon, rdflib is a joke. It has a ridiculous 200 issues / 1 commit a month ratio, buggy as hell, and is for all intents and purposes abandonware.

rdflib.js is in memory only, so nothing you could use in production for anything beyond simple stuff. Also there's essentially ZERO documentation.

And none of those except for Jena even step into the realm of OWL.

> What are the alternatives?

Good question.

SIMPLICITY!

We have an RDF replacement running in production that's twice as fast, and 100 times simpler. Our implementation clocks in at 2.5kloc, and that includes everything from storage to queries, with zero dependencies.

By having something that's so simple to implement, it's super easy to port it to various programming languages, experiment with implementations, and exterminate bugs.

We don't have triples, we have tribles (binary triples, get it, nudge nudge, wink wink). 64 Byte in total, fits into exactly one cache line on the majority of Architectures.

16byte subject/entity | 16 byte predicate/attribute | 32 byte object/value

These tribles are stored in knowledge bases with grow-set semantics, so you can only ever append (on a meta level knowledge bases do support non-monotonic set operations), which is the only way you can get consistency with open world-semantics, which is something that the OWL people apparently forgot to tell pretty much everybody who wrote RDF stores, as they all have some form of non-mononic delete operation. Even SparQL is non-monotonic with it's optional operator...

Having a fixed size binary representation makes this compatible with most existing databases, and almost trivial to implement covering indices and multiway joins for.

By choosing UUIDs (or ULIDs, or TimeFlakes, or whatever, the 16byte don't care) for subject and predicate we completely circumnavigate the issues of naming, and schema evolution. I've seen so many hours wasted by ontologists arguing about what something should be called. In our case, it doesn't matter, both consumers of the schema can choose their own name in their code. And if you want to upgrade your schema, simply create a new attribute id, and change the name in your code to point to it instead.

If a value is larger than 32 byte, we store a 256bit hash in the trible, and store the data itself in a a separate blob store (in our production case S3, but for tests it's the file stystem, we're eyeing a IPFS adapter but that's only useful if we open-sourced it). Which means that it's also working nicely with binary data, which RDF never managed to do well. (We use it to mix machine learning models with symbolic knowledge).

We stole the context approach from jsonLD, so that you can define your own serialisers and deserialisers depending on the context they are used in. So you might have a "legacyTimestamp" attribute which returns a util.datetime, and a "timestamp" which returns a JodaTime Object. However unlinke jsonLD these are not static transformations on the graph, but done just in time through the interface that exposes the graph.

We have two interfaces. One based on conjunctive queries which looks like this (JS as an example):

```

  // define a schema
  const knightsCtx = ctx({
    ns: {
      [id]: { ...types.uuid },
      name: { id: nameId, ...types.shortstring },
      loves: { id: lovesId },
      lovedBy: { id: lovesId, isInverse: true },
      titles: { id: titlesId, ...types.shortstring },
    },
    ids: {
      [nameId]: { isUnique: true },
      [lovesId]: { isLink: true, isUnique: true },
      [titlesId]: {},
    },
  });

  // add some data
  const knightskb = memkb.with(
    knightsCtx,
    (
      [romeo, juliet],
    ) => [
      {
        [id]: romeo,
        name: "Romeo",
        titles: ["fool", "prince"],
        loves: juliet,
      },
      {
        [id]: juliet,
        name: "Juliet",
        titles: ["the lady", "princess"],
        loves: romeo,
      },
    ],
  );

  // Query some data.
  const results = [
    ...knightskb.find(knightsCtx, (
      { name, title },
    ) => [{ name: name.at(0).ascend().walk(), titles: [title] }]),
  ];

and the other based on tree walking, where you get a proxy object that you can treat as any other object graph in your programming language, and you can just navigate it by traversing it's properties, lazily creating a tree unfolding.

Our schema description is also heavily simplified. We only have property restrictions and no classes. For classes there's ALWAYS a counter example of something that intuitively is in that class, but which is excluded by the class definition. At the same time, classes are the source of pretty much all computational complexity. (Can't count if you don't have fingers.)

We do have cardinality restrictions, but restrict the range of attributes to be limited to one type. That way you can statically type check queries and walks in statically typed languages. And remember, attributes are UUIDs and thus essentially free, simply create one attribute per type.

In the above example you'll notice that queries are tree queries with variables. They're what's most common, and also what's compatible with the data-structures and tools available in most programming languages (except for maybe prolog). However we do support full conjunctive queries over triples, and it's what these queries get compiled to. We just don't want to step into the same impedance mismatch trap datalog steps into.

Our query "engine" (much simpler, no optimiser for example), performs a lazy depth first walk over the variables and performs a multiway set intersection for each, which generalises the join of conjunctive queries, to arbitrary constraints (like, I want only attributes that also occur in this list). Because it's lazy you get limit queries for free. And because no intermediary query results are materialised, you can implement aggregates with a simple reduction of the result sequence.

The "generic constraint resolution" approach to joins also gives us queries that can span multiple knowledge bases (without federation, but we're working on something like that based on differential dataflow).

Multi-kb queries are especially useful since our default in-memory knowledge base is actually an immutable persistent data-structure, so it's trivial and cheap to work with many different variants at the same time. They efficiently support all set operations, so you can do functional logic programming a la "out of the tar pit", in pretty much any programming language.

Another cool thing is that our on-disk storage format is really resilient through it's simplicity. Because the semantics are append only, we can store everything in a log file. Each transaction is prefixed with a hash of the transaction and followed by the tribles of the transaction, and because of their constant size, framing is trivial.

We can loose arbitrary chunks of our database and still retain the data that was unaffected. Try that with your RDMBS, you will loose everything. It also makes merging multiple databases super easy (remember UUIDs to prevent naming collisions, monotonic open world semantics keep consistency, fixed size tribles make framing trivial), you simply `cat db1 db2 > outdb` them.

Again, all of this in 2.5kloc with zero dependencies (we do have one on S3 in the S3 blob store adapter).

Is this the way to go? I don't know, it serves us well. But the great thing about it is that there could be dozens of equally simple systems and standards, and we could actually see which approaches are best, from usage. The semantic web community is currently sitting on a pile of ivory, contemplating on how to best steer the titanics that are protege, and OWLAPI through the waters of computational complexity. Without anybody every stopping to ask if that's REALLY been the big problem all along.

"I'd really love to use OWL and RDF, if only the algorithms were in a different complexity class!"

RDF4J is okay for RDF, but completely ignores OWL.

RDFLib is a bug ridden mess, have you ever used it, or checked their issue tracker, and commit history? With that amount of production breaking bugs that haven't been resolved for years, it might as well be unmaintained.

Redland has last been updated years ago. Sure there's software that's finished, but with the complexity of RDF, OWL and friends, my hypothesis would be "it's dead Jim".

I haven't used dotNetRDF, but it looks okay at first glance. So at least you can do semantic web on windows...

YES! THEY'VE BEEN TESTED FOR 20+ YEARS AND ARE STILL HALVE BAKED, BUG RIDDEN, REASEARCH PROJECTS. MY POINT EXACTLY. These are all smart, dedicated people. The semantic web ecosystem is too complex to get right, no matter how many hours and $$$ you throw at it.

I have nothing to gain from talking about the shortcomings of RDF and it's related standards, except maybe inspire people to come up with something better, and to save themselves some pain and suffering.

The pot calling the cattle black. Your motives seem more questionable, with the whole "username matches the topic talked about", and has a Semantic Web consultancy business, shtick.