Thai Database Leaks 8.3B Internet Records | Hacker News Reader

Thai Database Leaks 8.3B Internet Records

rainbowtabl.es

166 pointsnexus755674 comments

> To be clear: DoH and/or DoT would have stopped the gathering of DNS query data in this case. It's simple to set up, and it's just a smart thing to do for anyone concerned about their privacy.

Actually, for most people that are not technically savvy this is definitely not an easy thing to set up, nor are they even aware that DoH/DoT exist.

Unless this feature starts being turned on by default in routers and popular software, the average user's DNS lookups will not be protected.

> Unless this feature starts being turned on by default in routers and popular software

Firefox has DoH turned on by default for US-based users. Unfortunately being US-only, it won't protect Thai users just yet. Especially considering the fact that Thai ISPs have known to be hijacking port 53 and NXDOMAIN since forever[1] with ISP in the article being one of the biggest offender.

It might also worth nothing that PDPA, Thailand's equivalent of GDPR, is to become in effective in two days (May 27th) so this incident will be... interesting.

[1]: Starting in early 2000s, with NIPA (that I never see anybody uses), who aim to provide IDN-enabled Thai domain names by providing NXDOMAIN-hijacking services to ISP

I recently set up a dns-over-https (doh) proxy on my router to forward dns requests to 5 resolvers, that also use dnssec.

I wish Firefox would expose the option from about:config in its user-friendly Preferences page so it will respect the "system default" (the advertised dns server).

I am - for no legitimate reasons - avoiding Cloudflare as a resolver. As far as I know Firefox uses Cloudflare.

Have you been able to find a trustworthy public DoT resolver?

I really want to use uncensoreddns.org, but availability has been a little flaky in the past. I'm not sure about Quad9. Google and CloudFlare are obviously out of the question. What else is there?

> Have you been able to find a trustworthy public DoT resolver?

It really depends what you view as being "trustworthy". Outside of the US good enough? Or do you want non 5/9/14-Eyes? (https://en.wikipedia.org/wiki/UKUSA_Agreement#9_Eyes,_14_Eye...)

Let's start with "not Google-levels of creepy" and work up from there. I want someone who's not harvesting data, or engaging in censorship.

Check out this list in privacytools.io: https://www.privacytools.io/providers/dns/

Yes, well i trust this one:

DoT: dns.digitale-gesellschaft.ch

DoH: dns.digitale-gesellschaft.ch/dns-query

Source:

https://de.wikipedia.org/wiki/DNS_over_HTTPS

https://de.wikipedia.org/wiki/DNS_over_TLS

https://www.digitale-gesellschaft.ch/dns/

Sorry, for not being available in the en-wiki

Hadn't seem them before, very interesting. Vielen dank.

Just a year old and located in Zurich,

OpenNIC has a few servers supporting either DoH or DoT or both.

DoH just moves the logging from Thai telecom and moves it to Cloudflare (or, whoever you set up as your DoH server, but most likely Cloudflare), no?

I trust CF much more than my ISP, but it makes the potential leak much worse...

edit: On the other hand, DoH makes DNS requests independent of ISP, which is nice. ISPs are often monopoly by nature.

Cloudflare has a privacy policy https://developers.cloudflare.com/1.1.1.1/privacy/public-dns...

Obviously, for Thai users this is a reasonable option, but I would understand why American users would not want to use Cloudflare, Google or even Quad9, as these are all US-based.

I thought quad9 was UK based?

Bottom of quad9.net says:

1442 A Walnut Street

Suite 501

Berkeley CA 94709

Aha, figured out I was confused because I read about their membership in the Global Cyber Alliance which includes members such as the London Police and the NYPD.

Bonus if you are interested: forum thread where Quad9's director of the board comments about DNS are surprisingly informational.

https://www.snbforums.com/threads/cloud9-dns.56918/

The netflow data from cloudflare, which is 0.05% of all traffic, is retained for 60 days according do the compliance report. 8.3B log entries is a lot, but I suspect that given how large market share cloudflare has, 0.05% over 60 days is also not a small data set. My intuition with probability calculus make me suspect that given normal internet usage over 60 days, a person is more likely than not to end up in cloudflares netflow log.

> a person is more likely than not to end up in cloudflares netflow log.

They are, but because this is all Cloudflare's 1.1.1.1 service does, that log only tells you that the IP address used Cloudflare's service.

So whereas this Thai data more or less says e.g. You watched Netflix between 18:40 and 19:26 and then again 21:33 to 22:09 the Cloudflare data says you own a device that uses Cloudflare's 1.1.1.1 DNS service. Maybe you can try to do some kind of activity estimate e.g. me idly browsing random web sites probably causes more DNS queries versus a SmartTV just periodically doing auto-updates when I'm asleep, but the 2000:1 dilution would make that more unreliable.

It isn't strictly nothing but it's damn close.

If the attacker also have access to the anonymized data set which is stored for 25hrs they should be able to deanonymize quite a bit of the data by just comparing time stamps and finding patterns.

It is much better than just having normal logs laying around, and since the data is split in two data set, there is a possibility that having access to one does not automatic result in having access to the other. The dilution is also helpful, through I am uncertain to the extent given the amount of traffic generated.

Is a privacy policy worth anything? Genuinely wondering, can they be sued, etc, if they violate it and claim "Oops, we made a mistake and our data got leaked".

For example Facebook probably said they would keep your data secure, but their system to prevent abuse from 3rd party "Quiz" developers was "Developer, by clicking here you agree not to abuse the data you can access."...

Could be interesting to use a list of DoH servers and pick one at random for each DNS query.

dnscrypt-proxy can do that for you, downside slow DNS server means slow DNS responce time

Well in Firefox you can turn it on: about:preferences#general >> Network Settings >> Settings...

Where is the setting in Chrome? It's not on by default in the latest version, and not available in the flags/settings...

Most malicious ISPs disable DoH (ex: Verizon) so it likely wouldn’t have solved this.

If you want it solved find a protocol that can be used in the libc resolver and make it ubiquitous rather than goofing around with browser defaults.

How do they disable DoH? They block TLS over that port?

Thats DoT, DoH ist just a dns query over https(443)

Alright, I repeat my question, since it's almost identical: how do they block that? HTTPS starts with opening a TLS socket, how do they reliably determine that they can drop that traffic?

I don't know if Verizon is actively doing it, but since most providers that offer DoH have well-known IPs, like 8.8.8.8, 1.1.1.1, 9.9.9.9, they could easily just block traffic heading in that direction.

This follows along with what I read, at first it’s easy to block but it’s a losing proposition because anyone can stand up a doh server (which is also a network security nightmare)

Not sure what has changed security wise. Any one could also stand a DNS resolver. Many people use hosts file or used a local resolver for dev domains for example, security minded folks setup up resolvers on openWRT stack router for example or a use a piHole.

Blocking popular DNS providers is a common tactic deployed by ISPs. It is technically easy enough to bypass depending on your skill level and interest in doing so. Their strategy is make it difficult for majority of their users, who won't know or care, users doing all this will not make substantial impact on revenue generated from selling this data, or from showing adds etc so they don't make the effort.

DoT/DoH is not going to change this, Firefox's market share is not enough for that. I don't see Chrome or Safari implementing this functionality at all.

Okay that’s great for my developers now what happens when they download some malicious script modified their doh queries to use their servers instead of my corporate wins?

I don't think they do that.

Mozilla provides instructions for network operators that want to block it.

How can they disable DoH? That's Dns over HTTPS.

> Most malicious ISPs disable DoH (ex: Verizon) so it likely wouldn’t have solved this.

Is this confirmed? I couldn't find a source for this statement.

This is something that the average user fails to understand. One thing is saying I don't care they check on what I visit but once you aggregate enough information, it can become something of a "Big Brother".

With enough DNS data I can assure you I can see when you leave to work, get back, determine the moment when you leave for vacation and no one is home, etc.

You wouldn't even need DNS data, just how much Internet traffic a specific connection or device is using is enough to determine these things.

(Disclosure: I'm Thai)

Especially in Thailand, where free speech is almost non-existent.

Few months ago there were Twitter user who goes by the name "Anonymous" ("นิรนาม" in Thai) who have been arrested for spreading fake news and being a threat to the country. The Twitter user mainly tweets about topics subjected to lèse-majesté law. He never leave any traces, which leaves question on how officials managed to track him down if Twitter claims they didn't received any requests from our government.

My small group of friend came up with one scenario where official sent a honeypot URL via Twitter DM, then trace him via DNS query logs. This is assuming the scenario where he don't click on random links and using a browser that performs DNS prefetching of sorts. Everyone thought it was unlikely at the time, partly because nobody thought ISP would actually logging all DNS queries.

Apparently, all of us were wrong, at least on the latter.

But aren't all the URLs in the messages/notifications "shortened" to a t.co/. So he would have had to click on the link.

This was also why everyone believed it's unlikely. Also I don't think Twitter even has DNS prefetching turned on. However now it's revealed that logging is real, us Thais should be worried.

Just for my understanding: this wouldn't have happened if the user in question would've used a VPN and/or TOR right?

Don't get me wrong, I really don't like this in Thailand and it's absurd that you would even need something like that. As a foreigner visiting Thailand I don't feel that comfortable with my browsing habits. Usually I trust a local provider enough to just browse and not care about what I'm looking up, Thailand is not one of those places and I always use a VPN. (Mostly routed to Singapore)

Yes, it probably wouldn't happened if the user uses VPN or Tor. If VPN or Tor's setup doesn't leak DNS, at least.

Related: there's no such thing as "anonymized data", there's only "anonymized until correlated with enough other datasets".

I hope someone takes the opportunity to download the entire database and serve it up as a torrent, because it would make a great source for studying the pictures that can be painted with contemporary ISP surveillance.

> Interestingly enough AWN had this DNS dashboard saved with a filter specifically looking at Facebook traffic. It's unclear why they would be particularly interested in who was going to Facebook.

One likely non-malicious explanation is that the telco is offering some plan with data caps based on social media such as instagram, facebook, etc. Searching around, I found the offering below for unlimited data on 9 social media apps http://www.ais.co.th/one-2-call/simcard/en/super_social.html...

I'm guessing one way the telco implements the selective cap is by tracking user's DNS, and is probably interested to know traffic to facebook

AIS is mobile operator hence assign you random IP from the pool every time you reconnect to the network. IP address could be used by many different users during a day, definitely not a household as author states. Looks like useless data for me.

Sessions on mobile networks can last for many days and even weeks. The data is far from useless and could be used to enhance or append to other data sources.

They also have residential internet (AIS Fibre) and also own another ISP (CS Loxinfo). CS Loxinfo used to use their own DNS servers and such, but have switched to use the same infrastructure as AIS Fibre (sharing IP address pool and all) since 2019 or so.

Also AIS mobile is IPv6 (2001:44c8:4400::/44) with CG-NAT since 2017. IIRC they were giving out /64 to every mobile client, but I'm not sure how long does /64 assignment lasts.

also they probably, like every ISP nowadays know when what IP to who was assigned so dynamic IP is not guarantee for anonymity

Can someone enlighten me who and what is rainbowtabl.es?

It's a blog run by the Head of Trust & Safety of Cloudflare.

waves author here.

To be clear: I do this research in my free time. This is 100% independent of my $dayjob at Cloudflare.

Solution for this is to tunnel the traffic through encrypted connection to servers in countries that respect persons privacy(if that is true nowadays). The easiest way is to use WireGuard, easy to set up uses only one port and have clients for many devices.

If you trust your vps dns, easiest way would be autossh -D<port> <user@host> and set your browser's socks5 proxy to localhost:<port> and tell it to use remote dns when resolving domains. This requires no wireguard setup, no certificate generation or anything.

I've been doing both and have to say Wireguard is much more performant and stable than an ssh tunnel. Besides, it shouldn't be too hard to set it up on a VPS.

As a counter point about reliability, I've been tunnelling my HTTP traffic (and DNS) through SSH (to get around corporate restrictions and monitoring) for 10 years or so - I don't think I've ever had any reliability issues.

I've had a lot of problems: latency, ssh tcp connections dropping packets and whole connection becoming unstable, manually configure proxy / browser each time & also sometimes you may forget to start the tunnel. You also need to start a new ssh connection for each port you want to forward, so you end up managing a bunch of ssh connections if you want to expose some services for example. Wireguard is more deeper down the layers and just works without jumping through hoops - none of the apps are aware of it and when it's on, it just stays on). Of course, when all you have is ssh to get around pesky restrictions, then I guess that will do fine too! ;-)

A good point about switching proxies and apps that don't support SOCKS natively.

I guess I've been dealing with those issues for so long they don't bother me anymore!

Also, I use a great extension for Firefox, so I can switch to/from the proxy in 2 clicks, "Proxy Switcher and Manager".

WireGuard doesn't use certificates it works similar way to SSH with keys, also they have open source clients for Android and iOS a few clicks configuration

You don't generate a key pair to share with the client? How does that even work when you want to disable a key or set a password?

What i meant generating keys is not equal to generating certificate in the common sense of this word, it only works with randomly generated keys , passwords are not save way to encrypt data unless you can remember random sequence of characters for every client you have. If look at WireGuard protocol will get all the answares.

I got my answers, it is pretty much openvpn with a different wireprotocol and there are key pairs.

> ... countries that respect persons privacy

May be a long time ago, in a galaxy far away, such a thing once existed. It's a sweet thought though.

You don't even need that to be useful though.

In my tinpot banana republic (Australia) ISP metadata retention is required by law, and warrantless access to that is granted to organisations involved in fighting terrorism, child abuse, and other serious crimes - and those agencies include local councils, animal control, the taxi commission, and various horse racing oversight organisations... :sigh:

Even moving your meta data to a different legal jurisdiction makes it less likely to be abused. My local nosy dog catcher is unlikely to attempt to get hold of any useful internet meta data when my ISP hands over their records and say "Ahhh, yes - bigiain's metadata here shows about 2TB of bandwidth for May, all to the ip address of a VPN endpoint in <checks ip geolocation> Belize... I can look up the Belize police phone number for you, do you speak creole?"

Isn't the official language in Belize is English? Aren't South America's routers accessible by the agencies collaborating with the US Govt.?

Maybe, and quite likely. I considered using Moldovia as my example jurisdiction instead, but Belize has some nice cachet and backstory to add appropriate colour and context to a rant.

Still gonna put off the local dog catcher who's trying to work out if I'm video chatting with his ex girlfriend...

(If _actual_ FVEY or equivalent national security agencies are curious about me, I'm pragmatic enough to know none of my tradecraft live action role playing is gonna make any difference at all. I could buy some magical amulets, fake my own death, and live in a submarine. I am still gonna be Mossad'ed upon... I'll avoid running shipping containers full of drugs/weapons/children across international borders, and try to keep my harshest criticism of the Saudi/Trump Royal families to myself...)

Oh. It's quite enough to express your opinions.

Thailand has some new and strong privacy laws:

https://www.insideprivacy.com/data-privacy/thailand-passes-p...

Which have been delayed for a year due to C19.

All Thai constitutions have had strong privacy requirements, but that has never been important for what actually happens.

It's really not clear what compliance will be like. If it's anything like most things here then it'll only be if the government gets annoyed that a company will be in any danger of prosecution.

There is a special place in hell for software developers who write server software with no authentication by default.

Some of them might respond with something similar to "if you know what you are doing". Probably, security is someone else's job.

Every single one of their customer is a customer that one day installed it for the first time and didn't know what they were doing because they didn't know the product.

And with the complexity of modern software, imagine if the defaults in the whole software stack all the way down to the OS and hardware were open by default. You would need to be an expert in security to set up anything. Thanks god everyone else goes secure by default.

I think you will find that place quite empty, today and in the future.

The other place, where "product owners" and pointy haired managers mingle - that place will be quite crowded when their TTL expires.

Reply on news.ycombinator.com