Hacker News Reader

The state of the art at Google appears to be F1 [0], although that takes a pretty different approach than Presto/Dremel and may be complementary. This area (interactive SQL on big data) has become very active lately. In addition to those you list, there's Shark [1] and BlinkDB [2] from Berkeley's AMP Lab.

[0] http://research.google.com/pubs/pub41344.html [1] http://shark.cs.berkeley.edu [2] http://blinkdb.org/

View on HN

As sprizzle says, F1 and Dremel are not competitors; Dremel is a read-only system supporting large analysis queries, while F1 is a read-write system designed for large numbers of smaller queries. Last year Google published a paper on PowerDrill (http://vldb.org/pvldb/vol5/p1436_alexanderhall_vldb2012.pdf), which sounds somewhat like Dremel but is designed to use large amounts of RAM, which in turn enables some very powerful optimizations.

I believe F1 is the SQL DB that powers ads (AdWords, AdSense, etc.) and is used in production for consumer-facing apps. Dremel is more of a data/log analysis tool, but doesn't typically interact directly with consumer-facing apps. Dremel (externally: Google BigQuery) is still widely used at Google across all product areas; F1 is used in a few products but not many.

Reply on news.ycombinator.com