JackWritesCode

You are correct. Fathom Analytics is the only globally distributed provider that offers EU Isolation (keeping EU data completely away from US cloud providers). https://usefathom.com/features/eu-isolation

Love it, Aaron. I've always felt more confident using Laravel knowing that it's backed by a competent entrepreneur. These paid services ensure the longevity of Laravel. We've paid for Spark and Nova, and we have Vapor & Forge subscriptions.

Thanks so much for supporting us. We have spent so much money on getting our compliance right :)

A lot of companies claim to be GDPR compliant, and they’re not. We’ll be blogging more about this in the future as we don’t like folks being lied to.

Woah this thread is packed. Most of these companies didn’t exist when we started Fathom back in 2018, so it’s great to see how the privacy-first analytics space is thriving.

I’m Jack Ellis, the cofounder of https://usefathom.com. We’re a two person, self-funded company from Canada. Our software is used in projects by companies like IBM, GitHub, Tuple, Tailwind and lots of other awesome companies.

You shouldn’t use Fathom if you flinch at $14/m for a highly available service. We’re self-funded, priced to be sustainable long term, and we don’t guess on privacy law.

Recently, we launched a feature called EU Isolation following the Schrems II ruling (where Max Schrems sued Facebook). Long story short, if you’re using analytics and are passing your EU website visitors personal data (IP & User agent) to US-controlled cloud servers (even if they’re located in the EU), you’re violating the GDPR. Well with Fathom, we use both EU infrastructure and US infrastructure, but we automatically route all your EU visitors through German-owned infrastructure, and hash all personal data there, meaning your EU website visitors personal data will NEVER touch our core US infrastructure. This approach was put together with our Canadian and EU lawyers, and hasn’t been seen before in the analytics space. Lots of companies run on “EU servers”, but they’re controlled by US cloud providers and subject to FISA. This means they’re not GDPR compliant.

We don’t guess on legal matters, and we don’t cheap out on infrastructure. The lawyers we invest in work with some of the world’s largest companies, and they’re a big budget item for us. We run our infrastructure across multiple availability zones and invest heavily in serverless infrastructure. If you compare Fathom against most privacy-first analytics providers, you’ll see our uptime is uncontested. It costs us more, yes, but keeping our customers’ analytics reliable is of crucial importance to us. We run only on managed services, as we want experts (like some of the DevOps folk on here) maintaining it for us, and we stick to our strength (building our application).

With our custom domain solution (to bypass adblockers), we handle automatic SSL for you and serve your website visitors from a CDN, keeping things really fast.

We’ve also recently launched multi-domain, which is super powerful for holding multiple domains under a single dashboard (something the OP was speaking about). Especially since you can also then utilize our API to generate custom reports.

We’re going from strength to strength, and we’re the best option for folks who need GDPR compliance. We are also based in Canada, so we have adequacy ruling under the GDPR.

Hope this post is helpful for anyone who is already wondering about Fathom :)

And if you’re serving those pop-ups from US-controlled servers (even if they’re in the EU), you’re violating the Schrems II ruling.

Sure :)

Haha I didn't know who he was. But the fact Alex Debrie follows him is certification enough for me. And I was wrong. Looking at his Twitter, it looks like he knows 2000x more than me about servers.

Never expected to see someone defending me on Hacker News. I appreciate it. To clear things up:

* I found AWS Shield Advanced organically

* It was a DDoS attack

* I am incredibly happy with the personalized service. It’s like hiring someone to handle it, except you have people on call 24x7

That sounds incredible. We’re used in a lot of universities right now and it would be great to see more

Dealing with AWS hasn’t been hassle but is expensive. However, the value proposition is well worth it for us.

Can’t talk about this publicly, sorry :(

The data was full of spam & was targeted at high profile customers. It was also run every hour for one period then ramped up to 500,000 concurrents. It was a DDoS :(

Yup with a minimum 1 year commitment.

> Layer 7 just means they are making ton of HTTP requests from lot of IPS however this is the nature of a web analytics.

Exactly! Which is why it was so hard. There's no path pattern. Everything hits "/". So the only way to fight back is to match IP / header patterns (but even then, we have to redact sensitive headers).

> How can the attackers know which websites have your analytics on? Crawling the web is super hard.

The attacker went after some of our more high profile customers. They're known via testimonials or from Twitter.

> The access log should store this `sid` allowing you to just block the account. (or ask them for ton of money if it's a legitimate use :) )

We could certainly temporarily block traffic to a site. The problem is, without some kind of firewall (e.g. WAF), our application has to absorb so much traffic, and that's the issue. We need to block it at the edge.

You and me both, fargle. When the attacks started, I thought someone was attacking us just so I would write a technical blog post.

I saw Kinesis stream today with DynamoDB and I will have to review it :)

They're still going to need to process IP addresses or some kind of PII though, that's the thing :(

It was 100% a layer 7 DDoS, the attack was targeted and malicious. I can't say too much but the AWS team confirmed it. But I appreciate how something like you describe could happen.

I like the idea of an MD5 hash. Although I'm not too certain why an IP would be a bad thing to log for 24 hours. From a privacy law perspective, the MD5 hash is considered PII. And if we see an IP address in an access log, we know that an IP visited one of the tens of thousands of websites Fathom runs on, but we don't know which one.

Edit: Something else, with MD5 there's no way of finding patterns with the IPs, so you'd have to play whack a mole. Whereas raw IPs allow no real privacy invasion whilst allowing pattern detection

Public access point. The server accepts pageviews, so they DDoS'd that.

Awesome, I'd love some advice if you're willing. So here's our situation.

* We're getting hit with a huge DDoS attack, repeatedly over 3 weeks, with no sign of stopping

* With zero access logs, there was no way to find patterns in the attack, and we had no way to block it

* Our service was going offline during these attacks

* We introduced access logs that are auto-deleted after 24 hours. We redacted all information about the site/page/activity etc. but keep IP & User Agent for pattern matching

* We were then able to identify a pattern and block the attack on Saturday

* Without access logs (even redacted ones), this wasn't possible

I was hoping a more senior engineer on Hacker News would comment and I can't wait to hear how you'd do it. I have no experience in DDoS protection at all, and this seems like the only possible way. Even rate limiting requires storing IP addresses. But if you know a more privacy-focused way to block these attacks, I'm sure I'll buy you a few beers when we hang out.

And to add on this, the plan moving forward is to default to writing data to the appropriate places (aim is < 100-200ms) but fall back to SQS when services are unavailable.

Vercel is good too. And I've heard great things about BunnyCDN. Cloudflare does a lot though.

Same with AWS Shield Advanced :)

I bet you could set up something way cheaper. And I'm certain you know 10x more than me regarding servers, hardening, configuration, etc. And I'm certain you enjoy servers!

For us, the cost works and we have appropriate margin for it. The cost savings aren't worth the extra "we have to monitor these servers" thoughts. Our approach is 100% emotional.

We are indeed. Neither of us enjoy DevOps so we pay a premium to not have to manage servers. It brings a huge mental health benefit (we are a two person company), we're profitable and our monthly Lambda cost isn't that significant. Honestly, the biggest inefficiency (in terms of cost) is our use of SQS/RDS, which we are ditching soon.

Lambda concurrency wasn't actually increased to 800,000 concurrents. It's a stupid idea that someone would think up after fighting attacks all day.

It's interesting to see a multi-billion dollar company step into the privacy-first analytics scene. Moving from one centralized, tech giant to another. I was so fired up that I wrote this: https://usefathom.com/blog/big-tech-vs-fathom

Auth for what? It's Laravel so we use the built in auth.

You shouldn’t keep centralized user data. I’m done here as it’s taking us both too much energy to explain our points. Because I run Fathom, I know about the data I keep, and there’s nothing user specific. Even in our queue, we never store user data...

You replied to a comment talking about:

> Fathom, SimpleAnalytics.com, Plausible.io, Matomo.org, GoatCounter.com, and (shameless plug but WIP) Chiffre.io

and you said :

> I think self-hosting is the solution. This way, there is no company that will store centralized user data, which is the biggest privacy issue for analytics.".

Your premise was that these companies store centralized user data. That is absolutely not true. So you presented a fake problem (that doesn't exist amongst privacy-focused analytics solutions) and gave a solution.