person_id is a foreign ID for a "person table" that fits well in the relational model. A "person" has various attributes (name, DOB, email) as well as a one-to-many relation to "groups". Groups are a collection of users with additional attributes (e.g. group name).
Now, what if I want to answer questions like:
- how many total events for all persons in group X
- who are the top 10 users in number of events in group X
- which are the top 10 groups in number of events
In this case, the person/group tables are part of the core business logic, they aren't specific to the events table. It doesn't make sense to store it in Clickhouse. Also, this person/group data gets updated sparsely, but "freshness" should be kept at a minimum (< 30secs).The simple approach to the first question would be
- Get all the user IDs in group X
- Filter events by those user IDs
But what if there are tens of thousands of users in group X? And hundreds of groups? Are megabyte-long queries supported in Clickhouse?