Hacker News Reader

Comment by jeffbee | Hacker News Reader

I'm interested in industry history, so I wonder if you can comment on how much of this, if any, came into Google from Metaweb. There are no Metaweb-connected people among the authors (that I can tell) but Warren Harris is acknowledged. Was it a language that was in development before the acquisition, or was it needed mainly after?

View on HN

Warren was certainly more familiar with the guts of MQL than anybody else, but PathQuery was designed for Google. PathQuery was actually the second system he built for extracting protobufs from graphs. He was also a longtime functional languages aficionado and the prototype implementation of PathQuery was written in Haskell. It took many smart-person-hours to get Warren's interpreter to work at Google scale and speed.

I think what differentiates PathQuery from many "query languages" is that it takes on the task of transforming that data into somebody else's schema (aka "turning protos into other protos"). Having a DSL for this is particularly attractive in a company that will otherwise expect you to do string formatting in C++. But even under normal circumstances there are wins from having data transformation vertically integrated with your query language.

Mainly after. Metaweb brought MQL with them, which ran on their own single server graphd, which was eventually shutdown.

Warren started working on PathQuery as a replacement for MQL. PathQuery was originally executed on Pregel and was far from a realtime query language.

A realtime query engine was later developed for PathQuery in the search stack so interesting search queries could be answered a la minute from the Knowledge Graph, rather than just pre-generating results for a limited class of queries.

This was all done in the 2012-2013 timeframe. Most of the people involved are long gone.

I've forgotten the details: by real-time query engine are you referring to DGraph, Livegraph or something else?

Definitely not graphd. Maybe Livegraph? I forget what the project was called, but it was the first realtime query engine built for the Knowledge Graph and it used PathQuery as the query language.

PathQuery was started by Warren Harris. At Metaweb, Warren did a rewrite of MQL in OCaml which we never shipped because of the Google acquisition. He spent much of the first year at Google working on a prototype of PathQuery in Haskell. The prototype was rewritten in C++ and used in the first KG-serving infrastructure sometime around 2012/2013.

As someone who worked on the Metaweb/KG team during that period, it's safe to say the development of PathQuery was a direct result of the acquisition. The lack of Metaweb-linked authors relates to the fact that PathQuery sits closer to 'serving knowledge' than 'representing knowledge'. But lessons learned from the development of MQL surely made their way into PathQuery, through Warren and others.

Reply on news.ycombinator.com