Hacker News Reader

Comment by mgdlbp | Hacker News Reader

Google refuses to search for hash-like strings that aren't well-known across the web.[0] That's why this search gives no actual results. (It certainly will in the future as this submission gets reposted, breaking the googlewhack)

[0] https://news.ycombinator.com/item?id=31638976

(I think Bing also blocks hashes but has an exemption for UUIDs; right now it gives me results for random UUIDs but not random MD5s)

Bing and Yandex are both capable of locating OP's Minecraft account in online databases:

https://web.archive.org/web/0/https://www.google.com/search?...

https://web.archive.org/web/0/https://www.bing.com/search?q=...

https://web.archive.org/web/0/https://yandex.com/search/?tex...

PS: example of an uncommon hash that Yandex finds but not the other two: the MD5 of 'Hacker News' (f3e4...), which actually occurs on the web on one site (http://archive.ph/oLgUe http://archive.today/lkyNN http://archive.ph/v3TY5)

View on HN

What's the reasoning behind the design decision 'refusing to search for hashes', what grounds would they have to block such searches?

I also would like to know. My guess is maybe password dumps?

This was my first guess

This is as much a stab in the dark as any, but as someone that builds/rebuilds ROM sets pretty often (leading me to do a lot of hash-specific searches) I've noticed that there's a lot of crypto-trading related hashes in the indices they maintain.

I'm assuming what happens is that the hashes for various blockchain transactions have a lot of common sub-strings within that collide with non-crypto stuff, hence why they get offered.

Maybe the folks that manage the search engine were finding that these crawls were adding a lot of useless info and just blanket blocked their inclusion?

Might just be to reduce the size of the index.

Not sure if they still do, but they used to map keywords to integer identifiers, instead of using the actual string value (string indices get very big). Page and Brin themselves explain it here[1]. I do the same in my search engine.

Problem is there are a lot of junk identifiers, so there's a point to reducing the scope by eliminating probable noise-keywords that are unlikely to ever be relevant to any search. UUIDs and hashes would probably fall into that scope, since they have a very large namespace that can very easily gunk up the lexicon with words that are never ever going to be relevant. You'd probably want to keep the word identifier 32 bits if you can get away with it, but maybe 64 bits for a global search engine like Google.

[1] http://infolab.stanford.edu/~backrub/google.html (section 4.2.4)

I really enjoy reading your comments on HN. I found your search engine on Gemini then see you post about search engines every now and then.

You have a great ability to break things down in a way that makes sense.

Hey, thanks man.

I don't feel like I do a very good job at explaining things for the most part, but maybe that's not a very reliable indicator of whether what I write makes sense. :P

The top Comment of [0] of the OP has a likely scenario: malware offers periodically searching for hashes of their malware to see if it's detected on the web, without having to submit it to virus scanning websites (as most online virus scan engines will automatically submit the executable for automated malware analysis).

Ha, the search in the submission now points to the version of itself that I saved in archive.ph, which didn't get the equation solver feature. But archive.org did; maybe change it for posterity to https://web.archive.org/web/20220806203457if_/https://www.go...

meta: apparently HN penalizes comments that have been edited to be too different from what was originally posted -- I tried adding this to my parent comment, which sent it far down the thread, which was then reversed upon editing it back

Reply on news.ycombinator.com