Hacker News Reader

Comment by vortico | Hacker News Reader

What if someone were to maintain an unofficial list with one domain per line, freely available as a daily torrent or served directly? Would there be a rights problem with mirroring and filtering ICANN data?

View on HN

A lot of tlds don't provide zone files unless you are a registrar. They would probably not be happy if someone put those out to the pubic. For com and the likes they would probably not care as much.

The best public list of domains I have found is the Project Sonar DNS (ANY) scans. I don't know how they do it, but their scans are pretty complete, at least for.dk domains, which are the ones I use.

https://scans.io/study/sonar.fdns

It's quite slow to download from that site, I guess there are a lot of HN readers consuming their bandwidth :)

Their download speeds are often bad sadly, they really should provide a torrent. But it's free and they have some really interesting datasets, so it's worth the wait.

I started to look at this one week ago, wish it provide some restful api for this useful database

Unfortunately, as part of the application you are compelled to sign forms promising that you won't make "significant" parts of the zone file publicly available in any way (at least this was my experience when applying to Verisign for .com and .net zone file access).

I see. Oh well.

Why not just upload it to Github? Then you only need to send the deltas for every update.

I was just thinking of creating a bot that will update the list(s) and push the changes back on GitHub every day.

But first, I definitely have to see if it's against GitHub's terms of service.

I think that GitHub would love the publicity personally. They strike me as being very savvy regards this sort of thing ;)

does GitHub need any publicity? honest question as I've found that anyone who would ever use the functionality GitHub provides is already very aware of git and GitHub.

people shift between services like github and bitbucket and alternatives all the time. Perhaps not often on an individual basis, but at any one time many people are deciding where to put their stuff.

Almost anything that gets the name of a particular service bumped up to the top of someone's consciousness for a little while will shift some of those decisions toward that service.

This is why even the world's most popular brands (Apple, Coke, etc) never stop spending money on marketing / PR :)

isn't it essentially public data though?

Same question here

It could be used to create a competing system. ICANN would never allow this. If someone tried to put this together, I think they would quickly find their access to the data revoked.

Reply on news.ycombinator.com