Bit of an annoying title, assumes tokenisation only means one thing. The toy compiler course I did explained that tokenisation was breaking a string into parts "tokens". Probably other meanings too I'm guessing
Yeah, synonyms exist. If you get annoyed by technical terms having different meanings in different fields, you are going to have a frustrating ride through life.
Yup, thought it was about text tokenization, i.e. for nlp/machine learning purposes.
Yep, overloaded term. Anyone got a better idea for what to call it in the context of sensitive data within applications?
Hashing, perhaps. It's probably what they're doing in background.
We call it PII (Personal Identifiable Information) at my workplace
For some time I though it was all about the security implications of strtok().