Hacker News Reader

Is it still “scraping” when the purpose of these transparency logs is to be used for this purpose?

The ostensible purpose of the certificate transparency logs is to allow validation of a certificate you're looking at - I browse to https://poormathskills.com and want to figure out the details of when its cert was issued.

The (presumably) unintended, unexpected purpose of the logs is to provide public notification of a website coming online for scrapers, search engines, and script kiddies to attack it: I could register https://verylongrandomdomainnameyoucantguess7184058382940052... and unwisely expect it to be unguessable, but as it turns out OpenAI is going to scrape it seconds after the certificate is issued.

Unintended: agreed. Unexpected: plenty of us called out this inevitability when the CT proposal was circulated.

The main thing isn't validating the cert you're looking at, per se, it's to validate the activities of the issuers. Mainly that they aren't issuing certificates they aren't supposed to (i.e. you can monitor the logs for your domain to check some random CA you've never done business hasn't issued a cert for it). This is mainly enforced by any violations (i.e. any certificates found that don't show up in the logs) being grounds for being removed from browser's trusted list.

Reply on news.ycombinator.com