Hacker News Reader

Comment by temuze | Hacker News Reader

How does this compare to PipelineDB? I'm pretty sure the cofounders of Pipeline are ex-AdRollers, so there should be a lot of overlap since it was made with the same problem in mind.

View on HN

It's not quite the same thing, PipelineDB is a database to execute streaming queries at massive volumes. That is definitely one of the challenges at AdRoll.

TrailDB is more about a different way of grouping and events, and granularly querying and analysing each trail of data. TrailDBs are materialized and stored in S3 typically.

I'd also add that PipelineDB excels at SQL-based workloads where you know the queries you want to run in advance and want to stream large volumes of data into PipelineDB's continuous query engine and store ONLY the results of the continuous queries in PipelineDB's underlying relational database. The main value is in continuous computation and distillation for realtime reporting and realtime monitoring and alerting use cases. PipelineDB isn't designed to do ad hoc, exploratory queries, although it can to the same extent that PostgreSQL 9.5 can. It's designed for scenarios where you know the analytic queries you want to run in advance and where a SQL-based approach to streaming analytics with integrated storage provides value by being simpler than building a custom system in Java / Scala using frameworks like Storm, Spark Streaming, Druid, Cassandra / HBase, etc.

Pipelinedb.com is a great product and very complementary to TrailDB.

PipelineDB is all about querying streaming, real-time data in SQL whereas TrailDB is great for computationally intensive analysis of historical data using any programming language.

Reply on news.ycombinator.com