Hacker News Reader

Everything Distributed Masonry does is very interesting. Wish I had more excuses to use your stuff at work.

Storing all data forever in a single source of truth is awesome until regulation like GDPR comes along. Do you have plans to support excision or is your guidance on personal data to avoid putting it into a system like Kafka/Pyrostore?

View on HN

You might enjoy reading Greg Young's https://leanpub.com/esversioning, which covers this topic.

It covers several strategies, three of which are:

* Encrypt it and then throw away the key to forget it

* Store private data outside the event with the event just pointing to it

* Delete events (on systems that support this)

We will be launching support for native excision and data anonymization soon, as these are extremely important to storing streaming data for the long term.

Workarounds for excision in Kafka, such as key compaction, are often not possible to use as they depend on the key scheme used.

Reply on news.ycombinator.com