That's a great example ^ that demonstrates exactly how this vulnerability can be easily abused
82 karma · joined May 6, 2015
Lenses a leader in DataOps is looking for an ex-engineer who can now look beyond the UI, and design excellence in the user experience.
We are a high growth startup based in London with a distributed team building something really cool
Contact us at info@lenses.io and meet us in Old Street
If we think around the options, you can have either:
i) eventual deletion (log retention policy) ii) compacted topics (and push null values) iii) expensive re-processing of the entire log iv) expensive segment re-write operation
with each option bringing in a new set of challenges
HDFS, S3 and other systems have immutability in-built. Immutability is not bad per-se, as it give (some) assurance that data has not been tampered with, and although it could be implemented, the system cost could be significant.
Stricking the right balance is the challenge
Are you interested in data streaming systems ? Are you familiar with Apache Kafka and Kubernetes ?