Show HN: I have written a cloud native dark web crawler in Go

Can you share some examples of what it finds? As long as it is safe to post that.

I have an instance running for about 5 days+ with 1,300,000 docs. It's mostly huge amount of porns, some weird forums & lot of marketplaces.

There's still some good finds like a 'sysadmin' forum with huge amount of good posts & tips.

I'll certainly publish an index of interesting (legal) hostnames on my GitHub account.

Release 1.0.0 will be out very soon. Check it out guys!

nice, what is actually needed to crawl onion sites ?

i am not a go dev, would try it with scala/java

Just a tor proxy (socks) instance, and a HTTP client configured to use the proxy. Other than that nothing more than what a web crawler should have.

cool i look into that some time.