Comment by andrewstellman | Hacker News Reader

Comment by andrewstellman | Hacker News Reader

Parent Full thread

I've been using RDF and triplestores / RDF databases for the last half-decade, developing both front-end and back-end systems, and training many developers to work in RDF. If you're used to either relational databases or object-oriented design, it's a really different way of thinking about data. Just like OOP is really good for certain kinds of problems and models, and RDBMS is good for other kinds of problems and models, RDF is great for specific kinds of problems. For example, if you need to combine several data from several somewhat incongruent sources into a single coherent database (e.g. dozens of data feeds that are almost the same, but you need to preserve the differences while combining the parts that are the same), you might end up with headaches trying to come up with a good RDBMS design; RDF is really well-suited for that kind of problem.

While a lot of the work I do is covered by NDA, one problem that I've applied it that I can talk about is analyzing basketball play-by-plays. I've spent some time talking to the analytics team at an NBA franchise, and it turns out doing interesting analytics on play-by-plays can be a surprisingly tough nut to crack. RDF was a great tool for tackling this. Here's the source (written in Scala), for anyone interested at having a look: https://github.com/andrewstellman/pbprdf

This is interesting, I always wanted to find a good use case for RDF but in the end RDBMS worked out fine.

Sports events seem a good example. What made it easier for you in your example of basketball play-by-plays with RDF?

Taking the first example https://github.com/andrewstellman/pbprdf#example-analyze-a-s... and translating it into an RDBMS approach seems rather straight forward:

  GameEvent (PersonA, EventType, PersonB, Game, Time)
  Roster (Person, Game, Team)

To get the fouls drawn you then take

  SELECT Team, COUNT(Team)
  FROM GameEvent
  JOIN Roster ON GameEvent.Game = Roster.Game AND GameEvent.PersonB = Roster.Person
  WHERE GameEvent.EventType = "foul"
  GROUP BY Roster.Team

Is it because of easier schema changes later on like introducing "secondsLeftInPeriod"? I suppose in a normalized relational scenario that could be something like:

  Period (Number, Game, StartTime, EndTime)

And doing

  .. WHERE Period.EndTime - GameEvent.Time < 5 ..

> Is it because of easier schema changes later on like introducing "secondsLeftInPeriod"?

That's definitely an important benefit. RDF makes it really easy to introduce changes that not only don't break the existing schema, but can be entirely isolated or combined in queries.

Another thing that RDF makes easy is analysis that takes advantage of a graph -- using relationships with other players, shots, etc. What players have the highest percentage making 3-point shots in possessions immediately after a player on the other team missed a 3-point shot? Building queries you compare previous possessions, shots, quarters; players' relations to each other (e.g. performance players who were subbed in after previous teammates went scoreless for 3 possessions) -- these things are a lot easier to do in RDF than in SQL.

Obviously, there are many things that are easier to model in RDBMS and query in SQL than with RDF/SPARQL. Every tool has its uses.

Do you know if there's off-the-shelf software (GUI) to create/edit/explore your own RDF dataset? Or does it always involve building your own front-end?

I like WebVOWL for visualizing the RDF ontology. Here's the pbprdf ontology displayed in it: http://www.visualdataweb.de/webvowl/#iri=https://raw.githubu...

A few years ago I put together a quick GUI in C# to make it easier to run SPARQL queries: https://github.com/andrewstellman/sparql-explorer

I haven't found an RDF editor or visual tool that I like. Some people like Topbraid Composer: https://www.topquadrant.com/tools/modeling-topbraid-composer... (commercial, closed source)

May be you can have a look at Datao: http://datao.net.

PS: I am the maintainer, and we currently have some certificate issues. Drop me a mail to datao@datao.net if you want to be notified when the problem is solved.

Take a look at http://ontodia.org

Is there a particular RDF store which you would recommend? It seems scaling is a bit of an issue with Apache Jena. It's very easy to bring to entire system to a crawl with certain sparql queries and enough data in the store.

Semi-related, is there a good in-browser RDF store? Say something like what PouchDB does for CouchDB (and similar) JSON document stores.

For as important as RDF has been for the web, it feels increasingly less "web native" today as most efforts still seem to be highly Java-focused and Browsers mostly don't run Java anymore.

(I ask because a silly project idea I have some tiny amount of notes for is something of a Twine competitor. I realized that while the language ideas I'm exploring don't look like SPARQL or other graph languages, there is a bit of an overlap conceptually under the hood and an RDF store might make sense as a bootstrap tech, but brief searches didn't turn up anything useful.)

At the moment, I use N3.js for its N3.parse() function. It builds a graph of Javascript objects in memory from a RDF string, just like a JSON.parse() builds a tree of Javascript objects in memory from a JSON string.

Note: I have never understood the need for a DB on the client-side.

In the case of a Twine-like system to make offline capable games: everything is just HTML/CSS/JS, it often gets packed into a single HTML file, and there is no server side at all. A client-side DB would be important in that case so that you can query/update game state.

In another thread GUN is mentioned, at that may be the closest client-side DB to what I think I'm looking for, if I ever get around to that side project.

This is probably the closest: http://linkeddatafragments.org/

A team I've been working with for years has had a lot of success with Blazegraph: https://www.blazegraph.com/ -- the marketing materials call it "ultra-scalable," and that actually turns out to be true. I haven't seen too many cases where specific queries will cause serious performance with the system.

That said, we've done some work to prevent runaway queries (e.g. strict query timeouts, downstream systems that handle that situation gracefully).

I believe BlazeGraph is the basis for Amazon Neptune, the graph DB AWS announced at ReInvent last year. https://aws.amazon.com/neptune/

That's great! I've had a chance to talk to some of the engineering and management folks at BlazeGraph over the years, and they're a really solid group. It's really nice to hear about a deserving team get success.

Reply on news.ycombinator.com