HNHacker News
TopNewBestAskShowJobs

kishoreg

2 karma · joined October 10, 2012

submissionscomments
kishoreg··on Creating a full-text search engine in Apache Pinot
Pinot is built to answer OLAP queries at high throughput while maintaining low latency. It powers many customer-facing analytics apps such as LinkedIn's who viewed my profile, Publisher Analytics, etc (50+).

At LinkedIn, it serves 100k+ queries per sec with 10-1000 ms latency while ingesting millions of events/sec from Kafka.

This is achieved by various indexing techniques - sorted index, bitmap index, range index, star-tree index, bloom filter, partitioning, etc and a flexible query execution planner that can dynamically pick the right plan based on the query and data profile.

https://www.youtube.com/watch?v=luMLCDANxiU should give you more info on why we built Pinot at LinkedIn.

Disclaimer: pinot committer