Great article, until the end.
Who uses PII in test data derived from real customers? That's just an absurd practice to begin with, and no one who takes security seriously would even consider doing this.
Who uses PII in test data derived from real customers? That's just an absurd practice to begin with, and no one who takes security seriously would even consider doing this.
As others have said, we've found a lot of smaller companies will test with production data because of their need/desire to move quickly. But we've also seen much much larger companies use production data in their dev/staging environments. Sometimes there will be production-like safeguards and security measures in place but not always. People shy away from practices that slow down development and testing.
We think synthetic data is the right solution for a few reasons. Most importantly, we believe it provides the right level security, while still allowing your team to be productive, i.e., your business logic and test cases still work. It also allows you to scale really easily since you effectively have a ruleset for generating data of any size. Finally, it’s a great way to share data throughout your organization and can help facilitate sales and partnerships. If you’re curious about scaling, check this post out: https://www.tonic.ai/blog/condenser-a-database-subsetting-to...
https://www.dagensmedisin.no/artikler/2019/01/27/brukte-virk...
In summary, when doctors were testing a new electronic patient journal system, they used real social security numbers (our version of them). And just for kicks they tested in production, so the persons used got all kinds of prescriptions for stuff they didn't need etc.
I think culturally there may be a difference since I'm in a place where some data (addresses, phone numbers, ...) is public info, i.e. given your name I can get your address and phone number from a public DB anyway.
The flip side of the coin is dev databases not being representative of production, this can cause performance issues. "It works with 10 rows on localhost, why doesn't it work with a million in production?".
Some small companies will refuse to use generated data if it takes even a minute more to generate it vs import it from production. In the consulting world I’ve seen multiple examples of companies complaining bitterly about other security minded consultants efforts to improve security and privacy through even small amounts of additional development time.
Making a copy is probably more effort than most developers out in the wild are going to make.
As a result, you get shockingly mature companies that do exactly this obviously absurd thing because it's a ton of work to stop. Work with no obvious reason to do this instead of feature work.
Since this involves computers nothing above is a hard rule, but it goes along with my experience.
The other two categories specifically target companies that really should comply with this law - I assume the $25mil clause is there to make sure large companies can't loop hole themselves out of this somehow (offload PII responsibility onto a subsidiary or a "third party" that is incorporated in Bermuda by the owner of the company)
Throwaway for obvious reasons.