Why is RDF so old, complicated, unpopular and still not discarded?

Now, there are two perspectives on what RDF is.

To an idealist, RDF is the universal data format. There are no semantics baked-in, and you can write arbitrary subject -> predicate -> object triplets to express any possible relationship. To an idealist, it's the perfect format for exposing all the structured data on the web in a machine readable form. The dream has always been for automatic agents to crawl the semantic web for you, understanding the meanings of the RDF triplets, and using them to reason out the solution to your query.

To a pragmatist, that dream has always sounded like a bunch of bull. Absent the presence of strong AI, it's a complete pipe dream that a piece of software will ever be able to infer the "semantic meaning" of interlinked RDF, just because it happens to be defined by triples. At the end of the day, you're going to have a programmer writing rules against specific terms in RDF, and if that's the case, than RDF is nothing more than an extremely awkward API.

Fortunately for the web, the pragmatists won. APIs are everywhere, and RDF is nowhere.

Unless strong AI happens to be right around the corner, the web dodged a real bullet there. Personally, I'm of the opinion that any web agent that could possibly puzzle through RDF triplets should have no problem understanding our APIs, in any case.

Let's look at RDF like a startup: The old RDF marketing from, say, 2003 was hopelessly out-of-touch with reality. Users were never going publish their metadata as RDF, and even if they did, you'd need strong AI to use it. Here are two classic articles spelling out why classic RDF wouldn't work:

http://www.well.com/~doctorow/metacrap.htm http://www.shirky.com/writings/semantic_syllogism.html

But things have been looking up in the RDF market lately. The complicated RDF XML serialization is mostly ignored in favor of simple n-triples. Google is making heavy use of RDFa metadata when searching for products, and something like 3.5% of web pages now contain RDFa. The RDF conferences are booming. There are cool projects like dbpedia that are organizing publicly-available information as RDF.

So if the RDF tool vendors are going to succeed, they need to pivot (and many of them are). They need to drop the AI hype, and focus on what their early users are telling them. Some possible sales pitches:

1) RDF is useful as a distributed, schema-free graph database. Competition: Neo4J, other NoSQL databases. There's a couple of very good sales pitches here, including the fact that RDF databases are available from multiple vendors, and that RDF inference can be used to normalize schemas between different data sources.

2) RDF is useful for embedding small amounts of data in web pages. Competition: Microformats.

But the RDF model is a wonderful thing. The use of URIs as identifiers for objects and properties makes possible for the first time to reuse knowledge and share data, linking APIs in the same way we are alreay linking web pages.

RDF semantics are maybe harder, but most people can start using RDF without caring about things like entailment.

I really think RDF has a future, specially since the steady growth of the LOD initiative. The revision of the standard is also a good opportunity to polish some aspects of RDF, for example, the use of named graphs.

Is that a failure of the technology? Because most people don't understand what they would use it for or how they would apply it? I don't think so. I think the claims that it would change the web were high-flown. I also think its creators did a bad job of explaining it. However, you'll find the people that do understand it and have a domain in which it is clearly applicable - love it.

One of my friends works for the library at UCSD and they use RDF, RDFS, and OWL-DL extensively - I couldn't even imagine doing what she does with the library's book ontology using JSON (as some have proposed replace XML and it's vocabularies, even with a JSON "schema" language). I have another friend working for a biotech company - and he uses it there, extensively. These are only two examples and it excludes the other web projects and companies out there that also use it and it's higher level vocabularies/ontologies (UMBEL, etc...).

Is it a failure for the web? (a topic in another thread a few days ago) I don't think it is, I think it is a failure on the part of developers to understand it and apply it (Drupal has applied it).

There's work being done in this space that is really exciting and I think we'll soon see how much potential the semantic web really has.

I really like the approach Freebase takes. It’s a proprietary format, basically. They use JSON rather than XML and they have their own query language MQL (also expressed using JSON). However their graph of entities maps to RDF as well, so they use RDF (along with common ontologies) as an export format. I think while their system is still complex under the hood, it’s less verbose, less scientific, and more user-friendly w.r.t. the public interface. And most important, thanks to MQL it's super easy to build applications on top of it.

---

For me modeling data as a graph (as RDF proposes) is a really great idea! What I always wanted to do is building client applications (single-page web apps) that can operate on a graph of data directly (instead of talking to a REST service). That's why I'm putting efforts in the creation of Data.js, which features a Data.Graph that can be manipulated in JavaScript environments (like the browser or Node.js). Such Data.Graphs can be persisted (synced) at any time. There's support for CouchDB as a backend.

Well, in the README I also pointed out why I decided not to use RDF and instead took inspiration from the Metaweb Object Model (that Freebase uses).

https://github.com/michael/data

I'd enjoy some feedback btw. The lib is actually working, but the examples are out of date. Have a look at the source or ping me if you want to try it out.

There is a very frequent problem people suffer from, which is mistaking goals for results. When you start looking for it, you'll see it a lot. A new open source NoSQL database will pop up, post a long list of goals ("Fastest performance, maintain some integrity, transactions, trivial sharding, consistent available and partition tolerant, and able to run on a TI-83 at web scale!"), put up a benchmark that shows that if you have no features and have no code written to ensure you don't fall down under real load you can put up way bigger numbers than the products with features, and suddenly you have some set of very excited people. Why are they excited? It's not the code; the code is worthless, the only thing it can do is run that benchmark. It's the promises.

You can see it in graphical programming. Graphical programming has a few modest successes, but the promises are about changing how everybody programs and how even Granny will be able to program. The fact that it has never happened despite immense effort for tons of smart people doesn't stop a certain segment of people from still being True Believers.

And, today, we talk about RDF. It promises to organize the web, it promises glorious wonderful search engines, it promises the world. It can't deliver, because merely sticking URIs on some graph nodes is only the beginning of the solution, not even remotely the end, you still have issues of agreement and accuracy and all kinds of other things. But the promise is so beguiling that some people just can't give it up, if we just try harder it'll happen, it's just that nobody has done it right yet, I'm smart enough to see what the previous hundreds of smart people haven't and I'll get it right, oh, it'll be glorious when everybody gets their heads out of their ass and listen to me and just start doing it right.

But RDF can't get us there. It's so general it's nothing at all.

There are all kinds of places where people become excessively bedazzled by promises and never notice the concrete reality before them. Another interesting example is Object Orientation. This has proved useful, IMHO, if not the be-all end-all of development methodologies, but it is interesting to contrast the promises made by OO back in, say, the late 1980s, with OO reality today. The promises were about how objects can represent things in the real world and you can model the real world with them. This turned out to be bunk. The real world has some place in OO but only carefully layered and wrapped and mixed in with a lot of other not-real-world things, iterators and factoryfactories and facades and data structs and ORMs and so on. The old promises were interesting and wrong, but also so beguiling that even today you will still hear this nonsense spouted about how this is the purpose of OO, even though it is now well understood that writing your programs with an excessively-strong tie to physical reality is asking for problems. Even as the reality is actually useful the old beguiling promises are still around screwing young developers up to this day.

(It is a tricky balance maintaining the proper level of skepticism because conditions change and sometimes wild promises become practical, and sometimes someone really does manage to pull off one of these things. The latest example would be the commercial success of the iPad, because for a long time smart money was on there being no market for tablets after numerous and repeated failures in creating the market. But in general, "show me the code" or appropriate manifestation is still the best way to avoid being trapped in one of these marketing traps, you will miss out on a few hits but pass on dozens of losers.)

(Also, I am aware there are still some True Believers using RDF. My point here is not disproved by a couple people using it, even using it in a big way. My point will only be disproved if someone brings about RDF Utopia, the actual promises. Of course you can bash RDF into submission, but that doesn't prove it was the best solution for your problem.)