Hacker News Reader

Comment by freedude | Hacker News Reader

45GB of data could be like a dozen employees' or less Outlook PST files. For this to be astounding we would need to know the quality of the data. Otherwise it is a bunch of hype and hoopla.

View on HN

I'm not sure about the legality and ethics of training models on stolen data, but for reference, but so far as I can tell the Enron email data set is about 1.5GB (much lower than I expected to be honest!).

And I believe some of the more interesting things found in that data set (outside of the fraud) were people cheating on their partners.

https://www.kaggle.com/datasets/wcukierski/enron-email-datas...

I do believe that 1.5GB is tarred and gzipped though, so it is a fair bit bigger. That's also supposedly half a million emails, so 45gb is quite a bit.

Sure, but that was leaked in 2004. A very different time for email, and where attachments were generally smaller.

The Enron dataset was not leaked - it was made public by the US government as evidence from an investigation by the Federal Energy Regulatory Commission.

https://www.ferc.gov/electric/industry-activities/addressing...

Yes my understanding is that this is why it's such a common training set - it's interesting, large(ish) and free and legal with no licence requirements. I think it's public domain? But I'm not from the US, so don't quote me.

It depends upon the size of the individual email. In the early 2000's email attachments were less common and on average smaller in size. Today, 20 years later, it is common in my line of work to see email attachments in the 5-10MB range. Pass them back and forth a couple of dozen times for changes and approvals and it doesn't take long for it to balloon. Yesterday, I worked on an outlook email issue on a PST file that contained 40,000 emails and was over 10 GB in size. His PST was new less than two years ago.

It really depends upon how it is used, misused or abused.

You're absolutely right. That's one thing that I didn't follow up on, I only briefly looked at the csv in the kaggle page and didn't see anything that looked like images.

You better pay up or we'll delete all of Marge and Victors email backups!

I was just gonna say, there’s probably single CFD simulation files that are larger than 45GB.

Could be the porn stash of the Boeing directors. That could make some of them pretty nervous. ;)

or it could be a single finite element analysis file

Reply on news.ycombinator.com