NZ’s biggest data breach shows retention is the sleeping giant of data security
privacy.org.nz
privacy.org.nz
Equifax's breach in 2017 was essentially every valuable piece of PII they had, on more than half the households in America, and they settled for $300 million in a civil suit. Their net income (after all expenses) every year is in the $500-700 million range.
They should have been fined directly, an amount intentionally set to force complete liquidation, shareholders should have been completely wiped out, and all proceeds from the liquidation after court costs should have been distributed to every victim of the breach.
If that was the case, 100% of US companies would now treat PII with the respect it deserves. As it stands, nah, it's nbd here.
Even Google of all places "respects" it now (after pressure). It becomes noticeably better every year.
GDPR may have had good outcomes too, and I am neutral on all other aspects, but whatever part of EU and California regulation led directly to cookie banners is a colossal failure which has benefitted no one (except possibly the dozens of snake oil cookie banner products which pretend to comply).
Understanding the data you collect, why you are collecting it, what you are using it for and what the risks are if the data is leaked is unsurprisingly a useful thing to do as a business.
In the UK, current news cycle is about sewage being dumped on our beaches by water companies that were previously privitised.
I think GDPR is generally good for individuals and the internet but if someone hates cookie banners, isn’t it fair to place the blame on GDPR?
Why can’t websites accept a special header which automatically accepts all cookies? I would enable it and handle clearing/retaining cookies myself through a browser feature/extension.
No, blame companies that set cookies for merely reading a website and then bothering users about it. They have a choice, they choose to make it obnoxious.
I could be wrong but IIRC cookie banners predate gdpr.
It did not specify cookies, and did not actually specify any technical means. The ePrivacy Directive requires that companies get consent from users before storing information or gaining access to information stored on end user devices. This includes every kind of cookie you can think of, including LocalStorage. There is an exception for cookies necessary for the service requested, which typically includes things like auth cookies or shopping cart cookies, so long as that data is not used for anything else.
The GDPR doesn’t specify the technical means, only that permission must be explicit and freely given, with the default assumption being “no permission granted”. I think these conditions are entirely reasonable, and a header that could be set by somebody other than the user, then sent by the browser on behalf of the user, does not satisfy these conditions.
Most entrepreneurs believe that visibility over how your visitors are using your website is “strictly necessary” for running a functional/secure/performant website and surviving as a business, but GDPR disagrees. Hence, cookie banners everywhere.
Not deemed “strictly necessary” > “Statistics cookies — Also known as “performance cookies,” these cookies collect information about how you use a website, like which pages you visited and which links you clicked on. None of this information can be used to identify you. It is all aggregated and, therefore, anonymized. Their sole purpose is to improve website functions.”
So much could have been simplified if the GDPR rules, instead of imposing burdens on a million websites, required the 3-4 browser vendors to have a toggle for preserving first-party cookies on sites where the user submits a form with a password field, and simply cleared all others at session end or periodically.
But by regulating browser vendors, they could have made it so that it doesn't matter what cookies they sent you. If the user hadn't consented in a browser UI, the browser would forget the cookies. Easy to verify compliance.
It's just like the ol' pathetic "Do Not Track" header. Same flaw. Asking "please don't give me a cookie that I'll have to keep and send back to you anytime you see me" instead of saying nothing, and just dropping the cookies you don't need on the ground.
This is not something that can be solved client-side other than obfuscation etc. They can track you with other means than cookies. Even worse, you might have an account on their site. Having an account and using the site (and logged in) makes it trivial to follow you, but that does not give them the right to abuse that information for other purposes. You might have an unique IP and can't reasonably expect to do anything about it.
GDPR covers all of that.
"Just delete your cookies/session" is not relevant.
So if you want to use cookies to link a user’s sessions on your own website together (without actually identifying them) so every request doesn’t look like a totally anonymous, opaque request, then you must show a cookie banner.
You could (presumably) do this through browser fingerprinting and not require consent (since you don’t actually enrich/link the browser fingerprint to be become user data) but you need a cookie banner if you do it with a cookie.
> So if you want to use cookies to link a user’s sessions on your own website together (without actually identifying them) so every request doesn’t look like a totally anonymous, opaque request, then you must show a cookie banner.
Wrong. The ePrivacy directive has an exception for strictly necessary cookies (Article 5.3), which is applicable for user sessions.
The ePrivacy directive: https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CEL... Search for “strictly necessary”. More details in this opinion from WP29, see section 3.2: https://ec.europa.eu/justice/article-29/documentation/opinio...
> You could (presumably) do this through browser fingerprinting and not require consent (since you don’t actually enrich/link the browser fingerprint to be become user data) but you need a cookie banner if you do it with a cookie.
Are you able to identify someone from the fingerprint of their browser? Then the fingerprint is PII. Consent (or any other legal basis from GDPR Article 6) is therefore required if the exemption from the ePrivacy directive is not applicable.
GDPR Article 6: https://www.privacy-regulation.eu/en/6.htm
It also isn't necessarily a requirement that such permanent records are digital, this depends on the country.
"He who controls the past controls the future. He who controls the present controls the past."
How else will future generations be able to put your residence on Paeroa St in connection with the consumption of fish in that area during that time, to reveal the political stance of you and your offspring against the ruling party of the Tilapia...
Which reminds me, I really should just write those down somewhere safe.
For example you don't need to know exactly who voted for Trump or Biden in the last election, you just need to know the result
Deleting data is hard work that requires a lot of preparation and has to be done without the safety net of backups. In many cases it will require changes to proprietary software. Keeping data happens automatically.
Not having data that you're supposed to have becomes obvious as soon as someone asks for it. Holding on to data for longer than necessary only becomes a problem if there's a data breach (that cannot be covered up).
The added complexity is regulation saying you need to keep data on your customers. In a way the early somewhat anonymous internet was better in this regard.
An edit to add, if you encrypt user data at rest (big ask currently). You can destroy the keys past a point and then the data in backups is safe etc.
Anyway it'll take a while for this view of the world to shake through.
Company A that leverages data will outcompete company B.
Lots of layers to this analogy, given the damage our use of oil is causing.
Companies are going to learn that using LLMs without paying proper consideration to data governance is a recipe for large fines and worse (such as being required to throw out entire models because their provenance violates data subjects' privacy rights).
many direct analogies indeed
I think we'll see a national privacy law in the United States at some point in the next five years. There's appetite for it in both major parties (Democrats to protect bodily autonomy, Republicans to stick it to Big Tech), and I think the targets of the regulations themselves will at some point lobby for a consistent national law rather than the patchwork of state laws that we have now.
Everyone should watch it https://www.youtube.com/watch?v=GAXLHM-1Psk
I work at a megacorp, and we have an entire group that goes around auditing projects to ensure compliance with data rentention policies, which include mandatory deletion of different types of data at different timeframes.
Financial transaction records have a very long mandatory retention, IP addresses for logins have a fairly short mandatory deletion. Product telemetry has strict rules about avoiding PII and very very quick deletion (aggregate data can be kept longer).
I’m sure the policies could be improved, but I suspect every large company is intensely aware of the importance of data deletion, even if competing priorities sometimes lead to longer than technically/legally required retention.
What is very, very quick? And why?
If I was to walk into an ad tech space in 2016 and tell people about my new found talent of scheduling data scrambling jobs against PII and rambling on about migrating all user data to be encrypted at rest they would have called the police.
See also: GDPR transition
This is about the bigget teeth the privacy commissioner has. Suprised to see such a heavy handed response from them given it was only 20% of the population.
Really, the NZ privacy commissioner's office is a joke. You can see this when they say things like:
> The Office of the Privacy Commissioner also encourages individuals to challenge hard why an agency needs to collect and retain their personal information.
I've talked to people who work in govt that have had full cctv surveillance at city wide (Hamilton) level Ok'd by them. Like no worries mate, they're in public so you can use casino face detection technology on them to track wherever anyone within the inner city is detected for 40 days of collection capacity.
Well, the privacy act has no teeth, so all they can do is put their hand in their pocket and pretend they have a pistol.
The act is about 30 years old now, it’s totally unsuited to the modern age and it desperately needs an overhaul.
A question about NZ English... How is that last clause (the final five syllables) used/interpreted?
To my ear, it sounds comical. "Our office wishes to formally advise millions of citizens that your trust has once again been violated, and in our official role we declare this to be a not okay event."
https://www.centrix.co.nz/my-credit-score/suppress-your-cred...
Two years ago they made you submit a manual request then prove with a police file or something that you have real proof that you're in real danger of identity theft. Now it seems it's more automated.
Equifax: https://www.equifax.co.nz/credit-file-suppression
Illion: https://dbcsprodaecdn.azureedge.net/web/themes/cycnz/assets/...
No actual authentication to sign up - presumably you can do so if you know a few personal details and somebodies driver’s license number or passport details (neither of which are secure information).
Thanks G
That is, no evidence in favour nor evidence against you.
> That is, no evidence in favour nor evidence against you.
It's because this isn't a court of law. You're trying to convince other people to give you an unsecured loan. The default will never be an assumption of trustworthiness.
Spy agencies: we need everyone to collect data to prevent terrorism!
Politicians: We need to do that to protect children!
Politicians: Also if you ever leak any of the data we forced you to collect and store data whether you want it or not, you will be liable.
If we would just start with all their data first for their in office part, as that is of real public interest again...
I didn't get that impression. Drivers licences and passports aren't things banks generally care about. It sounds more like the "know you customer" and "anti money laundering" hoops all financial institutions are forced to jump through now.
The rest of your comment rings true, although "spy agencies" is too narrow. It's spy agencies, police, the tax office, the social welfare agencies (like Centre Link in Australia), the financial agencies like Australia's ASIC.
I don't think it's politically possible to stop the data collection. A few real terrorist plots in Australia have been caught with this data, banks have been fined billions for allowing what is effectively money washing. Regardless of what I personally think, the Law Enforcement Agencies will say not collecting this information is the equivalent of "let the terrorists/paedophiles win" making any argument to stop it a near impossible sell.
But that doesn't matter. You can insist this data is collected without creating a wide open barn door like we have currently using cryptographic protocols. As an example, the government could you an app on your phone, lets say a car rental agency requires proof you have a valid drivers licence. You present the phone to an NFC reader, the phone says "XYZ Business has an authority to verify you have a current licence and it's expiry date", you click "hand it over", the phone provides the required data to the rental company. It's signed to prove to the rental company the the government has certified you have a licence expiring on some date. It also contains a lot of details about you the police can use to chase you down if you are in a road accident - but that's all encrypted, so no one can see it, including the car rental company. So it doesn't matter if it leaks. Barn door closed.
Well sort of. The car rental is probably going to collect a lot of information about you anyway, like a name, contact phone and address. But that's because they want it, not because the government demands it. Hopefully those details won't be sufficient to pull off identity fraud. The government ID's they are collecting now definitely are ID fraud material.
This way of handling sensitive data isn't novel. We already do something like this for credit card details. When a web site offers to remember your credit card, it's likely they've never seen it, let alone remember it. The only people who have seen it is the payment processor - Stripe maybe. What they are remembering is a token Stripe gives to them, which they can use to charge your card again. But the token reveals nothing about you. What's more it is useless to everyone bar them - Stripe won't accept it from anyone else.
I initially thought that it was gov mandated data, but I felt that given the available info in the article meant I couldn't back up that as a statement so backed off.
I would guess it's probably a mix of government mandated and business purposes, at the same time I suspect there's a bunch of info they want but that happens to overlap the gov requirements.
Legal tends not to understand the subtleties of data models, so they blanket prohibit any data deletion from anywhere.
Yes, sometimes even archiving or vaulting of old data is not allowed as it might compromise ongoing litigation.
It was built to directly mitigate events like the Latitude breach whereby the service gives an answer to an identity question rather than spraying PII across the economy. Answers were formed by pulling data from disparate authoritative sources in real time, a set of tokens were created and an audit record created and shared with PII owner consent. No personal information was ever intended to be shared or stored. It was an elegant solution for New Zealand, though we were mindful of a potential scaling issues in larger jurisdictions.
The financial sector was the initial target to help with AML/KYC flows. The banks in particular lobbied for access to the PII rather than an answer to the question so the service was devalued from the get go. If we’d won that answer I believe that digital identity and personal information sharing would be very different today.
Forest and trees, and lack of actual political leadership.
Of course it was never going to be adopted en masse by the private sector since part of the "get approval to use RealMe on your website" was "get Parliament to pass an Order in Council adding you to the authorized users schedule".
How does an organization that has leaked PI even resolve a 'privacy harm'. Once that information is out it's out. there's no taking it back.
Counter terrorism legislation requires than financial services companies store customer identification.
The issue was not adequately restricting the third party communications into Latitude’s network.
> Counter terrorism legislation requires than financial services companies store customer identification.
I was very much not their customer.
The third party was https://dxc.com
But more often I don't think third parties don't need full access to the master database via web access ... if they do then surely the customer needs to be informed who the company's partner is, and what that company's policy is to guarding any personal data loss/ misuse / retention.
From what I see lately (and there has been some massive data loss here in Australia in just the last year) there's a very care free lax attitude with a few shrugs after data is lost - with the hope naughty hackers can be blamed.
In principle. But in practice this is quite hard, especially in smaller organizations, or in large organizations that have a relatively small tech department. And that's before you get into companies that use outsourcing.
Some stats show the majority of all breaches have an insider component, either wittingly or unwittingly.
Agencies should not be collecting or retaining personal information unless it is necessary for a lawful purpose connected with their function or activity. All agencies should have a personal information retention schedule that they review regularly. The simple discipline of deciding how long information will be retained as you collect it and acting on these decisions will save you and your customers a lot of pain.”
So many panels, surveys, commissions, and watchdog groups state the obvious and yet little action on those recommendations is ever taken. In fact, many times the actions that are taken exacerbate the problem. The recently proposed bills to try and deal with TikTok and the social media data collection practices is a great example of this: propose sweeping new laws that violate privacy and free speech in order to combat privacy issues.
Everyone wants soft delete deleted_at instead. Everywhere. Many ORMs even force this behavior.
Why? Why can't we just delete records anymore? I'm sure some have specific reasons for this, but I'm trying to find the broader reason that everyone has jumped on this bandwagon.
Maybe I'm too YAGNI, but I prefer delete by default, unless justified. Not the inverse...
The ability to change your interface is what's alluring, not the mere fact that you have a record of changes. Though the audit trail is nice too.
Funnily enough you don't get to keep an audit trail while also losing the data. It's a trade off and businesses prefer the model that suits their business.
Also managing “on delete” is complicated to hook everywhere, much easier to hook a flag and touch nothing else implicitly.
FK cascading and DELETE, when done poorly, can... well, cascade, and delete lots of things the end user did not foresee. Like nuking half the database levels of bad.
`deleted_at' flags avoid that problem; combine it with row-level security in your DB (like postgres) and you can hide the rows permanently as well: from orms and non-superuser roles alike with a simple policy not to show any row where `deleted_at` is not null.
Someone accidentally deletes their company profile as they thought they were deleting the duplicate. They expect to be able to get it back.
Someone wipes out all the config files to get revenge on their employer. That employer is going to get in touch with support to get them back and they expect them back, even if you have to dig through backups and will make support's life difficult if they do not.
I think there's a certain amount of "you must keep records pertaining to financial transactions for N years" going on combined with (more recently, I suppose) abuse/spam protection.
Old-school solution - keep the records, in off-line hard drives in a fire-proof safe, with each one labeled "physically destroy on $Date".
Now, what do you do with standard backups such as the ones which mingle data which is 1 day, 1 year, and 5, 10, and 20 years old?
So what happens if a bank employee accidentally deletes an email in Microsoft Exchange? This is can not happen due to the policies in place. All emails are stored in a hidden folder for as long as necessary, ensuring that no data is lost.
Imagine you run a shop with a list of products. People can order and receive products from you. View past orders and get invoices and recepts.
One day you have a product that has reached end of life, or isn't selling well. So naturally you delete it from the store.
If you actually deleted this item, all the old invoices wouldn't be able to refer to it. Old customers couldn't claim warranty on it. You couldn't create old sales reports that made sense.
Turns out that this problem extends to most tables. Old clients? Users? Products? Invoices? In most cases outright deletion makes many other things invalid that you want to keep. So it's nice when the ORM makes meeting this requirement easy for you.
All this in addition to the ability to recover from an accidental or temporary deletion.
Corporations and governments have an obsession with tracking individuals, although probably for different reasons (profits and tax collection being the main goals, I imagine, with 'social credit score' as a newer more dystopian concept). This is well-documented in the US and no doubt applies to NZ as well:
> "As we have documented at The Privacy Issue, pacts with U.S. intelligence are a mix of cooperation with, and infiltration by, three-letter agencies. Though nearly every major Big Tech company has cooperated with the NSA's surveillance programs in some way, the Google cloud was notably backdoored via clandestine means that were likely unknown to the company, with NSA agents gleefully bragging about the infiltration in an internal presentation."
https://theprivacyissue.com/government-surveillance/battle-c...
New Zealand is a member of the so-called Five Eyes Collective so the NSA likely has full access to all personal data of New Zealanders, let alone everyone else in the region:
https://www.theguardian.com/us-news/2015/mar/05/new-zealand-...
Isn't it different?
At worst, companies like Walmart, Google, and Amazon are creepy in using any information they can collect to try to sell you useless products.
At worst, the government can use any data they collect about you to throw you in a metal cage for life, take your kids away from you, drone strike you, steal your money, inject you with drugs without your consent, execute you, etc.
One is a mere annoyance. The other is a direct threat to your life, liberty, and property.
Yes, but these are not compartmentalized how they should be.
The government has some limits on what data they can gather/store on you (mostly it seems these limits are respected less and less, but at least they exist and you have some chance to fight for them in court on constitutional grounds).
Private industry has no such limits. So they gather everything. But there are also no limits to how they can resell your private data, so a lot of it is resold to.. the government. Which gives a clean backdoor to bypassing those pesky limits.
Unless proven otherwise, you should assume that any of your private personal data gathered by private industry is also likely being passed on to the government.
I was sent a shipment recently and the sender didn't put my last name. So the courier asked me to email them a copy of my passport or driver's licence. I rang them directly and was told that this is the only way of providing my identity. I said hell no and the package was sent back (they ended up resending it with my full name).
I find the worst culprit to be healthcare providers. I've had two recently ask me to send my credit card information in the regular mail. I drove down to one of them to do it in person, and they put my details in a filing cabinet. I sure hope that lock is good!
(Pro tip: if you ever need to send CC details via mail and you cannot arrange an alternative, use registered post with signature so you can confirm it made it to the recipient)
I bought an iMac on interest free finance direct from Apple in 2014 via Gem (which is now lattitude). So I imagine everyone who has bought a Apple product on finance is going to be caught up in this. Also looked at the Aussie Apple site and they still use Lattitude for finance payments.
The breach vector? Legally-mandated tax records. Someone cracked the padlock on the warehouse they were stored in and made off with the hard copies.
It is entirely possible that this sort of data exposure is, in the limit, impossible to guard against. Perhaps the better approach is to ask whether there's any way to make that data less valuable.
But the real risk, I think, is the collection of every packet going over the wire. There are entities storing every packet in the hopes of eventually using quantum decryption to decrypt it all. The question has moved from "should it be secret" to "how long must it be secret?" because the longer you need to keep it secret, the more computationally expensive the encryption is going to be.
To be fair, encryption strenght has always been rated on the basis of how long it should stay secret.
For example a site like https://www.keylength.com/en/compare/ (which has been around a very long time) guides you to pick key strenght based on how many years you'd want the data to be protected.
You can be charged for storing more data than is actually needed. For regulated entities like banks, it is important to have processes in place and demonstrate them during your annual audit. In case of non-compliance, you may face fines or even lose your license temporarily until the issue is fixed.
In my job (infrastructure management) I am constantly fighting against people wanting to throw data away. "We know the condition of the bridge" they say - "so we don't need the past 20 years of maintenance records. It's too expensive to keep".
What they lose is the ability to understand how things change over the long term - how bridge supports and bearings degrade, so they can take proactive steps in the future and replace things in a planned way, rather than waiting for them to break.*
*Actually, what we do is a bit more complicated than this, but hopefully you get the point.
Such data can be kept forever, maybe should even be public because of public interest, and likely even publicly owned.
Personal data / PII, history and also current cases again and again show should be minimized. Germany had it pretty right with its https://de.m.wikipedia.org/wiki/Datenvermeidung_und_Datenspa... though it gets attacked and holed out recently everyday again.. for the public interest they say, but for the profit interest of some corps they mean.. if not even for spying on citizens...