Unsealed court documents reveal data anarchy at Meta | Hacker News Reader

Unsealed court documents reveal data anarchy at Meta

iccl.ie

42 pointsdonohoe14 comments

Uh, horse poo. Prod data is guarded like Fort Knox. No one can get access to anything on the graph without explicit approvals for a business purpose. I tried it. I can't even see who are my coworkers from my own FBID in internal apps.

> people responsible for data systems are unaware of how other people in the company use their system. In some cases even the engineers using a system may not be able to understand what is happening because, according to a Meta engineer, ‘it is not possible for humans to understand’.

Sounds like a good way to hide a "Room 641A" situation from your own engineers...

https://en.wikipedia.org/wiki/Room_641A

It sounds like someone asked a vague question about a neural network during a deposition and didn't understand the response.

> The documents describe data anarchy within the company, where people responsible for data systems are unaware of how other people in the company use their system. In some cases even the engineers using a system may not be able to understand what is happening because, according to a Meta engineer, ‘it is not possible for humans to understand’.

Honestly, I think a lot of big orgs with deep legacy tech and a bunch of shadow IT are in a similar position.

I work in an old school bank. Can confirm. You're lucky if people even know what the data they themselves use actually means. Usually there's all sorts of edge cases that make them go "ohh no, it can't do that, that would make it completely wrong", but it does do that and it is indeed wrong.

> When ordered to produce information about what 149 different data systems within Meta do, and what parts of Meta’s business use them, the company was unable to respond.

This feels to me more like a "tantrum-compliance" (I made it up) with a court order. But what do I know about it (nothing), maybe there is data anarchy at Meta.

This means that most European banks, big companies and government departments will soon be fined by the EU... The following could be said of almost all large corporations and institutions, including the year long project that failed to catalog data sources effectively:

"When ordered to produce information about what 149 different data systems within Meta do, and what parts of Meta’s business use them, the company was unable to respond. This was despite having conducted a year-long investigation of those systems."

There is no point having a law which essentially no company can be compliant with. It means that the law will almost certainly be applied only by exception and hence capriciously. The likes of Facebook may be able to use that as a defense despite their breaches being egregious.

As a side-note, the idea that Meta only has 149 in-scope data systems sounds ridiculous.

Lots of organizations and government agencies have already been fined because of these laws and lots of organizations comply with these laws. I work for a company that makes healthcare software solutions and we're fully compliant with the local GDPR laws.

There's the possibility that nobody at Meta genuinely knows how the '149 different data systems' interact and interoperate.

I wonder if the EU has the legal authority to force them to understand via some means? Otherwise it seems like a stretch of a demand.

I think it’s fair (if biased) to say that Meta’s incompetence is no excuse to break the law.

No individual needs to know all of it. But any team owning a system that reads data should know what they're using it for. Likewise anyone providing data to other teams should at least know which other systems are reading that data; even if they don't know for certain what the data is used for they know who to ask about it. So the process of understanding can easily be sharded, and the results merged in a fairly mechanical manner.

The timeline here seems to be that they were asked for this in December and required to produce it in January. It probably is unreasonable to get this done from scratch in that time. But if they haven't started a project to do it by now, then it does feel a bit like they prefer being able to say "we don't know".

Do you think a single person or team could up a coherent flowchart/diagram of all data processes at least, given enough time?

Speaking as a data practitioner - data lineage is a profoundly hard and still unsolved problem.

In the last few years, I feel like we've begun to crack it - dbt's data lineage graph is a huge advance over anything I've ever used before:

https://docs.getdbt.com/terms/data-lineage

But in general - as an industry, we aren't there yet. All judgments about Meta's practices aside - no large organization likely has the ability to comply with a law like this without years or decades of engineering work, and the law seems pretty clearly intended for selective enforcement.

This is clearly data chaos by design as a way to skirt privacy laws that may impact business performance and profits. The EU won’t fall for it!

Reply on news.ycombinator.com