Hacker News Reader

Comment by pufuwozu | Hacker News Reader

I worked on Precog. Here's my explanation:

It's an analytics platform for big data. Basically, you upload lots of JSON, use the custom language (http://quirrel-lang.org/) for analytical queries and the platform will handle efficient storage and querying of your data.

There are also has extras like a library for charting and reporting query responses using JavaScript and an IDE for writing Quirrel and uploading data.

View on HN

Thanks for the explanation. What do you actually upload the JSON into - is it a proprietary NoSQL technology, something built on top of Akka, a wrapper around HDFS or something else?

How does this compare to rethinkdb (sharding, map-reduce, indexes, queries and joins on JSON data)?

columnar storage, compression etc

Interesting, so precog breaks JSON objects into columns dot-named by their path. I wonder how rethink does it, internally?

How is this different/better than using something like Elastic Search? Does it handle real time upload/searching or do you have to bulk upload, process the data and then run your searches?

No processing necessary. Just upload JSON.

The Quirrel language is much more high level than the API for ElasticSearch. It comes with things like machine learning (e.g. clustering and linear regression):

http://precog.com/developers/how-tos/machine-learning/

Even without libraries like machine learning, you can write somewhat readable queries which do some impressive things:

http://precog.com/developers/how-tos/common-analytics/

Great, now I'm gonna be up for another three hours! Thanks for your feedback.

Reply on news.ycombinator.com