Trinity: Microsoft Research’s Hypergraph Database

You can make a nice, simple (and surprisingly fast) graph database from the file system.

I had to do this back in 2003 to create dynamic sites for users of a build-your-own-website app (who needed to have their data stored in their local filesystem - and they wouldn't let me use sqlite).

It turns out to be really good for expressing websites structures. The downside is that, as you can't create on-the-fly relationships, you need to either store your data in a tree that describes its use, e.g.:

  blogs/2009/03/05

or you need to add links that point to the same data to create the relationships, e.g.:

  blogs/2009/03/05 -> stories/a_blog_post

Obviously you can add these relationships later in much the same way as you might add indexes to a traditional database.

My system also had a notion of arrays. An index to the next auto_increment was held in a file named '.#'.

I also has a file called '.class' which would tell my object system how to process a node like this as if it was an object (each class had a a set of scripts that acted as methods).

It all worked amazingly well.

There were a couple of issues: old Solaris, I recall, used a linear search of directories so the system would crawl if a node got too big (I overcame that first with some simple a/b/c style hashing and second by upgrading Solaris) and directories were 4k each so users (who had file-usage limits) might have had grounds for complaint.

MS focusing on a GraphDB makes a lot of sense to me. Unlike document database or key=>value stores, GraphDBs aren't as mainstream yet (amongst the early adopters). Also, the typical GraphDB use case wouldn't compete with SQL Server.

I think it's a problem space that we've only see the tip of the iceberg and Microsoft recognizes this. I also think this is one area where the MS stack doesn't scale well, hence why they'll address it. Sadly, I wouldn't be surprised to see this research get packaged into SQL Server (or worse, Azure).

I was thinking... Will every datastructure eventually gets it's own database to facilitate distributed persistence of mass collections?

Why, or why not?

I find companies talking about their internal products with no release pretty much the antithesis of hackeryness.

AIUI, graph databases have much less impedance mismatch to objects than SQL databases do. So why are most people still using SQL? Tradition, or efficiency?

Seems to me that if a node can have multiple edges pointing to it, and an edge can point to multiple nodes, nodes and edges are effectively the same thing, or at the very least have identical properties. Am I missing something? I suppose it's a useful convention to still partition a hypergraph into nodes and edges.

See also http://www.semantic-web.at/index.php?id=1&subid=57&a... which is not reference and seems to predate other efforts noted herein.

Microsoft is making Trinity after Neo database (http://neo4j.org) was made.

they even have the name derived!