Find every domain someone owns automatically | Hacker News Reader

Find every domain someone owns automatically

securitytrails.com

231 pointstzury107 comments

The results page should always show what your search term was. Currently it only shows this when the search finds something. If nothing is found, all it does is tell you that nothing was found.

That leaves you with no way to check to make sure you entered the right term, other than typing it again. If you are typing it right but auto-correct is kicking in when you hit enter, even trying again might fail because your site responds fast enough that you might not have time to notice the change before the results page comes up.

Also, it might be helpful when someone searches for a domain at a TLD that you do not support to say on the results page that the TLD is not supported.

+1 on search term! And if you enter a domain that doesn't exist, it throws you back to the homepage with no way to fix it :(

Another feature request: When entering an IP, there's very little information available on it (only the hosting provider). Would be nice to get latlong, country etc (a la https://www.iplocation.net/).

Because seriously, that is an insanely cool database. I can even find all the sites that share my domain's Cloudflare IPs.

awesome. thanks for the feedback! great idea, we'll implement this week. Would love to stay in touch.

Awesome ideas. Will implement. If you'd like updates in the future drop me a line at chris at securitytrails.com - would love to have you keep testing.

I didn't look into what this is doing but it's not finding 1/2 of my domains (they are not private). I'm guessing it's not "Find every domain someone owns" it's "Find every domain that meets X criteria" which may more may not be every domain someone owns.

Yeah, I was fairly unimpressed when it didn't even manage to lookup the first domain I entered (it's not new and it's not private). I guess it might be useful for some domains, but there's no way I'd rely on it.

Same about this one. Any further info so we can enhance the user experience would be awesome. We have a tough time still with ccTLDs because registries locking down the zone files. Any more info or suggestions appreciated.

Hi! Would love to know more about the domains it's not finding. Are they Country Code domains or gTLDs? Any other info so we can debug would be awesome. Thanks for trying the service!

I have the opposite problem. It's finding 1300+ domains that I don't own. The perils of name matching against a common name.

Must say, I'm not a fan of yours or similar services, or whois databases for that matter. Privacy should be easier on the internet for people owning domain names.

Hi! The only way on the current Internet to remain anonymous is to obscure the domain registrant from right when the domain is initially registered. It's also important to use a service that pools a lot of domains together or have diversity in the hosting providers so they can't be easily correlated. We try and help people understand and make the problem visible. You bring up a good point that it should be easier.

A good option for people living in the EU is to register a .eu domain. For private individuals (like private non-commercial websites), the whois data is by default restricted to show an e-mail address only. (see section 2.4. in https://eurid.eu/en/other-infomation/whois-policy/)

Yeah, it's hard. If one has multiple domains from multiple tld's pointing to the same server, it's really easy to find out other domains and tlds with more detailed and easier to access records that will reveal the owner's identity.

Privately registered eu domains are quite nice. I couldn't even find two of such that I own in your database and whois only reveals email (which I made sure from the beginning to be of the same domain that I just registered, so that's useless for anything). People can probably pay for more info though.

Why do you think privacy should be easier for domain owners? Shouldn't it instead be easy for a visitor to find out who owns the domain and is responsible (content, technical or legal) for a certain site?

Not everyone needs to know my home address. Especially not for a domain I use for emails.

Way too many weird people out there.

Also, any content that people could take issue with. Any hobbies I might have that I wouldn't share on facebook.

This is why I lie in my whois data, to the horror of the goody two-shoes on HN I'm sure.

The whois data is a great place to start a social engineering hack. The address or any past address is often used for identity.

I bought an internet domain in order for people to be able to contact me via some stable handle on the internet, that only I control. That's all. I don't want my phone, my home address, my all other domains, etc. to be easilly accessible for no reason whatsoever by nosy types.

Also your question may apply to the other end too. Shouldn't it be easy for an owner of the server to find out who is behind an IP address that is connecting to his server - and I mean easy access to his/her phone number, home address and a complete history of those, too?

It's easy to concot legal, technical and content reasons why this should be.

Also think about online shops which turn out to be scams.

But perhaps there should be a possibility to remain anonymous (and the browser informing the user that this is the case).

This capability is already available to lots of large organisations.

Making it available to everybody just sheds light on the problem, it doesn't create the problem.

Security through obscurity.

You probably misread the word. It's not security, but privacy. And privacy is made by obscuring, usually.

How do you plan on handling the EU’s “right to be forgotten” (it’s pretty straightforward to make the argument you’re a search engine) and other components of the GDPR?

"The right to erasure" is not an absolute right for anyone to get all their data deleted. If the data owner (read: the registrars) still have a legal right to collect and maintain the data public and it has not been revoked one could argue that they (security trails) don't have to remove the data.

It's my understanding that the registrars are the ones with the burden here. They need to inform everyone of the data erasure and/or data updates on private information. Fun times when you have public information for anyone to gather on the internet. It could be that there are exemptions for these kind of services, I do not know, but would the exemption not also include the services that aggregate/collect historic information as well?

Disclaimer; I am not a lawyer. I am not well versed in GDPR. Anyone finding this interesting should go read up on GDPR.

>one could argue that they (security trails) don't have to remove the data.

It doesn't work that way. The "right to be forgotten" can be used to remove search results from Google, even if the original content stays up.

Interesting! From a quick google the following wikipedia citation seems to what you are referring to: Grounds for removal include cases where the search result(s) "appear to be inadequate, irrelevant or no longer relevant or excessive in the light of the time that had elapsed."[1]

Under GDPR, Security trails (company or person that operates it) could be classified as a "Data controller" [2] and then would of course be liable to delete information gathered about a person upon request and when the data is deemed to be "inadequate, irrelevant or no longer relevant or excessive". So for example, John Doe wants to remove the historic information that he used to own porn.com which he doesn't anymore.

However, I do not think it's clear that you have to delete the data for the current owner of porn.com due to his or hers need for privacy as long as they have collect the information lawfully.

As an actual advice to the people at security trails I would recommend they put up clear instructions on how to request a data erasure from their database. Like "Email erasure@securitytrails.com to request removal of your personal information" and what information they need to delete it.

[1] https://en.wikipedia.org/wiki/Google_Spain_v_AEPD_and_Mario_...

[2] https://en.wikipedia.org/wiki/General_Data_Protection_Regula...

Edit: formatting

Actually the first thing Security Trails have to do is to figure out under which legal basis they think they have the right to process personal data. This is fundamental to figuring out their duties. I strongly suspect they don't have a legal basis in GDPR terms and therefore would need to rely on consent. The much publicised "right to be forgotten" is the very least of their worries.

Haven't really explored this. Any feedback or ideas is appreciated!

You should read up on GDPR urgently. It applies to anyone anywhere in the world processing data of EU citizens and has some massive penalties for things like not asking for specific informed consent and not offering an opt out. Using the excuse that you're just aggregating public data does not cut it.

This can't be "The World's Largest Repository" when half my domains are "not a valid domain" according to the tool.

It doesn't work for privacy-protected whois (obviously), but it also doesn't work for .id domains (try http://every.id, http://awesome.id or http://player.id).

We'll add .id to the wish list. Tricky because they don't make the zone file avail, but we'll figure something out. Thanks for the note.

It's like Uber all over again. This is expressly forbidden with most registries, especially European ones.

It says I own 86 domains but is using my given names and not my email address. My name is not unique. Hardly a valuable service. I own less than 10 for the record.

suggestions on how to make it more effective for your use case?

Is this just a reverse indexed WHOIS database? If so, it's no surprise that my domains don't appear in yhe results: I signed up for Whois anonymization through my DNS provider. I was recently considering unsubscribing, so thank you to the creators of this for reminding me that my privacy is under attack at all times and I should do whatever I can to protect it.

That's what I think too, and I definitely don't own millions of domains.

Be careful about trusting privacy protection plans. A spam email came to me recently showing the unmasked info and I still have no idea how it acquired its data... https://www.thejach.com/view/2017/10/google_whois_protection...

I'm finding that sometimes it'll turn up the data correctly and other times it won't match what I can find by manually typing in addresses into domains.google

As in this service will claim all data is private when google is able to return the actual registrant email address and/or name. As well as valid phone numbers which don't match what dnstrails is outputting.

thanks for the note and testing. Would love to dig into any specific use cases so we can see whats going on. chris at securitytrails.com

Try damsteen.nl for instance, my blog, it shows no whois at all.

got it - thanks!

I think http://whoisology.com has been doing this for ages.

So has Domaintools (http://domaintools.com) but both are expensive (Domaintools in particular is very expensive) whereas DNSTrails appears to be free unless you want API access.

So has http://viewdns.info. Free too.

Great resource, thanks.

Hi ohashi, we do not only track the current and historical whois records, but also current and historical DNS records, even for subdomain, which technologies the website uses and even more data which we are currently working on. Maybe try our WHOIS aggregation feature and let us know what you think!

Can't even try it on one of my own domains (.IT) without paying up first...

Just tried it with some of my own details. Found a domain I forgot I owned on an old reseller I havent used in a while - lol.

On a more serious note - I'm very curious how you get such a long history of domains. i.e. I can see every DNS change and ownership for any domain - I didnt realise that was always available?

Hi! We acquired 4 companies that have been doing lots of cool data work. We also license and collect our own data to mix in. There's different granularity depending on the data (WHOIS history, Name Server history, DNS record history, technologies used) etc. We're constantly improving it.

The thesis we have is that if you get hacked, it often times is through an old server or satellite domain. We're building tools to help you find the extended surface area where you can be hacked or have downtime. The example of you finding an old domain is a prefect use case.

Just out of curiosity, was one of those companies originally named DeletedDomains.com? They had the full root zone (of the ones now managed by Verisign GRS anyway) since approximately 2001 (my involvement with them was 2003).

hey - no. wwws.io, dnstrails.com were the main ones for this data

I'm pretty sure this goes against European (or at least Dutch) privacy laws because it's not just company domains being searched. There isn't any privacy-overriding reason to keep a database with this kind of identifying information. Since these laws are currently barely enforced, nothing will happen of course.

More info in Dutch: https://blog.iusmentis.com/2017/11/08/internationale-domeinb...

\o/ I own 5 million domains... wait a minute do I pay for all of those?!

hey, can you give a little more info? Are you sure it's not picking up on some other part of the registrant?

Whoisguard variants.

Yah same for me on google domains

So, for .de it just returns either NULL, undefined, or empty for everything, for .eu it errors out entirely, and for the domains of me it does find, it has wrong data.

kuschku.de has, for the past 2 years, always pointed at 51.15.1.223 or 163.172.217.134, never at 204.236.227.242. Funnily, for other domains pointed at the same IPs, it has correct data – e.g. quasseldroid.info correctly shows the IP history.

The datasets used here seem of questionable quality, souring the taste of this awesome feature.

hey- would love it if you can drop me an email (in profile) so we can debug/improve.

.DE is particularly hard because they lock down the zone file. GTLDs like .INFO are easier to get because the zone files are open. We have 9 years daily granularity for the gTLDs.

Would love to clean up the U/X so it's clear what we have data for and what we don't to be completely transparent.

Thanks for the note.

Your profile doesn’t have any email address in it, but you can just check https://dnstrails.com/#/domain/domain/kuschku.de vs. https://dnstrails.com/#/domain/domain/quasseldroid.info

For example, WHOIS info https://i.imgur.com/WNpyvcl.png should maybe show something like "none available", or "no WHOIS info is available for .de", or "go to denic.de to see WHOIS info" (DENIC offers the WHOIS info, if you enter the captcha). On the other hand, http://whois.domaintools.com/kuschku.de (a competitor) correctly shows the WHOIS.

Second, with the domain – I have no idea how the wrong value ended up on there.

Hi kuschku, thanks for providing us with the samples with the outdated/missing data. We will definitely look into the case and continue to enhance the data we collect.

The UI enhancements you mention are excellent - we will implement this shortly (not only for .de but for any case where we can not output any values).

Feel free to contact us at the e-mail address given at the bottom on every dnstrails page!

What’s interesting is that it correctly finds Namecheap whois-guarded domains too. If I search for the guarded domain directly it shows the correct record as the owner’s address/etc being WhoisGuard, but then if I search for a non-guarded domain and click through from the identified name, it does list the guarded domains as well (!)

The OP said something about zone files earlier, so it's possible that they're getting their data from more than just Whois lookups.

can you give an example?

It said I owned 45 domains, and listed 10 that I actually own along with 35 I've never owned, heard of, or ever been associated with.

I have a very common english name.

Annoyingly, some of the domains it inaccurately says I own are NSFW. They need to put a big disclaimer on the results page pointing out that the results aren't necessarily accurate.

I know that GoDaddy blocks WHOIS requests like crazy. How are you able to 'bypass' that restriction at such scale?

Hi. There's some good info online.

https://www.icann.org/resources/pages/approved-with-specs-20...

Check out section 3.3.6

BinaryEdge does something like this but also for IP addresses and then security rates them https://blog.binaryedge.io/2017/11/23/organization-mapping/

searched google.com. turns out they own the url.. android.porn

Nice find bananamansion. Actually you will see this with many big cooperations which also register domains such as <companyname>.sucks, <companyname>.adult etc. ;)

They have that domain so that nobody else uses it.

I have a few of those. :)

Is it possible to include WHOIS data from way back when InterNIC was the only registrar?

I own a domain I registered way back when you would send an email to InterNIC and registration was free. The WHOIS data returned for the domain only starts in 2008 and skips about 12-14 years.

Hmm is there a technical write up of how you are pulling this data?

I tried one of our companies for fun and it’s only pulling 1.4million records in one place and then 65,000 in another. Doesn’t seem to have all our nameservers or relays either.

Based on the supported TLDs, I’m guessing they are pulling down the root zone tables from Verisign GRS. Verisign licenses the Whois data in bulk out to companies like theirs.

Would love to hear more so we can debug. chris at securitytrails.com if you have a few mins so we can chase things down.

We'll do a technical blog post soon - good idea.

Looks like I own around 289,000+ domains that I’m not aware of.

awesome. hopefully your creditcard isn't on file for the renewals :-) . we'd love to hear about any bugs- chris at securitytrails.com

GoDaddy is now masking _all_ DNS info whether you pay for their "privacy" service or not, so thats making a substantial black hole in this data

What do you mean they mask all DNS info? You mean whois info? Can you provide an example?

The cynic in me thinks they probably sell that data now.

Couldn’t get it to work with .rs domains.

That being said, cool tool!

<sigh> these country code domains. LOL. We bought and built up http://www.domainlists.io so we can add some more stuff like this in. I'll make sure we add .RS to the action items. Thanks for the feedback and enjoy!

This is great. Is there are a list of supported TLDs? For example, I'm not having much luck with .co.za (South African) domains.

.IT (Italy) neither.

i guess this doesn't work for private registration. E.g. Domains by Proxy's phone number has ~11 million domains.

We have a really fun blog post coming out for this next week or so. One of our team found a pretty big bug in certain private registrations. Stay tuned.

Can you give a gist of your findings?

I'll take a guess : customers are given unique whois protected email addresses allowing you to find all the domains owned by the person. Eg- If I own abc.com and xyz.com, both have the same public email address. The problem with such bugs is that there's no way to undo the damage since historical whois records are archived.

you nailed it. It goes a little further with what you can see and how easy it is but right on point! If you notice anything else people would find interesting, feel free to post it or email me (in profile).

You say fun, but in the era of swatting and doxxing, private registrations are often the first line of defense for those who don't have a USPS or ups mail address if they don't have a physical office/address. I would be very careful about how you responsibly disclose any such bugs.

Good point. Poor word choice on my end. Thanks for the note. The stuff we found is not quite that abrasive but very interesting. How would you recommend disclosing?

Let the providers fix it. Contact the people potentially affected if you can.

This is brilliant.

But a question: why is one website I know of (and that resolves, and that has valid DNS entries) comes up blank?

hey, thanks for checking. Shoot over what you're looking at to hello at dnstrails.com and we'll investigate.

Also, is your WHOIS data for each domain "live", meaning it's being queried in real time?

1. for the current record displayed on the domain results page, it's live but then has a short cache after the first time it's pulled on the public site.

2. For the whois registrant search, it's around 90 days old right now, but we're working on techniques to make it more current.

Very nice! Also... scary.

That is very handy.

Although it didn't work as I expected.

Hey, we're still kind of new and kicking the tires. Any UX feedback or data feedback is appreciated!

It's pretty much missing or empty data for the entire .au TLD (e.g. even google.com.au is not present).

Living here, I know that their whois server rate limits heavily and the ccTLD zonefile is not available, so I'm guessing those are probably contributing reasons. Do you guys do any crawling at all?

I think this is a fantastic concept though. Knowing that whois database and zonefile access is often protected for commercial motivations, it really irks me. Open it all up.

right now we're getting .au domains from Open Crawl. It's tricky because the zone file is not available like you mentioned. If anyone has any ideas on how to get more .au domains, we'll gladly implement.

There's some reverse engineered zone files for a lot of cctlds at http://viewdns.info/data/.

CT logs come to mind, if you aren't using them already.

DNS queries, if you have access to recursive resolvers.

we're using CT logs for getting hostnames (you can see it at the top of the page of any domain).

We have a great recursive resolver source but haven't been able to integrate it into the data pipeline yet.

thanks for the note and ideas!

woot- glad you like it so far. Any actionable feedback greatly appreciated.

It worked exactly as I hoped. I couldn't figure out why some domains I were expecting to show up weren't, but searching by email reminded me that I need up update some of my whois info due to a name change.

awesome! any feedback or ideas are appreciated, thanks for trying it out.

Scary, yet useful as fuck.

Reply on news.ycombinator.com