Scanning 7.6 Petabytes of HuggingFace Training Data for Secrets | Hacker News Reader

Scanning 7.6 Petabytes of HuggingFace Training Data for Secrets

trufflesecurity.com

35 points882542F3884314B10 comments

Open articleView on HN

In principle interesting, but I can't stand the Claude writing.

> Keys with real blast radius

> Here is what they unlock.

> This is a floor, not an estimate of actual balances or unauthorized usage. The keys were verified but never used.

> We cloned the public dataset hub end to end: every repository, every branch, every large-file object

> The size is only half the story. These are the training sets behind models people actually use. The worst-hit ones are named, card-documented pretraining corpora that open models were built on. We verified every credential we cite against its provider, so they were live when we looked.

The whole post looks like a Claude artifact with random little cards.

It's also just too long, which is a side effect of using LLMs, it's just too easy to create walls of text.

I similarly find it interesting; I do not understand why it is not the default to just generate the thing as a draft, research anything you aren’t clear on, and re-write it in your own voice.

That takes a level of taste and craft that many in this industry do not possess.

It takes a level of taste and craft to write good content.

It only bit of effort to write in your own voice , anyone with high school level of writing skills should be able to muster if they were inclined to do so .

Ever read a book by Yuval Noah Harari? He seems to have been writing like this since before LLMs. Perhaps that’s where they got it from.

reply with ASD-STE100 -> Yuval

I think "7.6 PB" is more informative than "4.4x Empire State Building heights worth of DVDs", but that's just me.

Wouldn’t it just be easier to crawl the net? Not sure what huggingface has to do with anything here.

I guess Huggingface datasets are easy to crawl - those datasets are hosted to be crawled. In contrast to the web at large which will be behind Cloudflare etc

This is fine. All of this is fine. :’)

Reply on news.ycombinator.com