Hacker News Reader

Comment by scandox | Hacker News Reader

I'm still left not quite certain what would happen if they were hit with another L3 DDOS tomorrow.

That said I'm very happy to use Sourcehut and I think they'll overcome these challenges over time. They seem to have the staying power.

View on HN

I am really curious if the DDOS tried to follow them to the new infra and failed to cause an outage or not. Apparently the perpetrator noticed when they got Cogent to narrow the null route, but the blog post notes they still can't access the original subnet in that datacenter. Are they still trying to knock Sourcehut offline? Is the DDOS still pointing at now deprecated infra for some reason?

When they switched DNS over to point to the AMS datacenter, the DDOS attack followed it until it got smacked down by the OVH NAT.

> At about 06:30 UTC the following morning, the DDoS escalated and broadened its targets to include other parts of our PHL subnet. In response, our colocation provider null routed our subnet once again. This subnet has been unreachable ever since.

Right, that's expanding to the rest of the subnet in their old DC. They've since migrated to the new DC with new countermeasures. Did the DDOS follow and the countermeasures are working? Or if it didn't follow, why not?

There's also the question of whether the DDOS is still even trying the old infrastructure. The post says it's unreachable, but that would be true if the null route hadn't been removed yet.

Yes, the DDoS followed us to networks with countermeasures, and yes, the countermeasures worked. We don't want to disclose too much about that, though.

They’re on OVH now and should have protection from it by virtue of being on their network now.

> However, we found that OVH’s anti-DDoS protections were likely suitable: they are effective, and their cost is amortized across all OVH users, and therefore of marginal cost to us. To this end the network solution we deployed involved setting up an OVH box to NAT traffic through OVH’s DDoS-resistant network and direct it to our (secret) production subnet in AMS; this met our needs for end-to-end encryption as well as service over arbitrary TCP protocols.

I'd consider it mostly protected, because no their servers are not on OVH, just a single box performing front-facing NAT/proxy essentially. The attacker now just needs to find the "secret" production subnet and attack it directly instead of through the front-facing NAT addresses.

That is very easy to mitigate, because you null route the production subnet except for a VPN that only can be reached by the proxy. You can even VPN over a completely different IPv6 route.

They could still try to knock your entire datacenter offline but that is much harder.

That's not much harder, that's exactly what happened to them in the initial attack.

You're still depending on either a "secret" IPv6 network, or your upstream provider performing some source-based routing to only route packets from the VPN connection. I doubt that's available to a simple colo customer.

That's true. I guess it'd be more accurate to say they're on OVH's network, not necessarily their compute and other infrastructure.

My reading is that OVH would handle it.

Reply on news.ycombinator.com