Hetzner outage - UDP forwarding affected

statusgator.com

11 points·alcazar··7 comments

Open articleView on HN

There was no UDP connectivity from any of our servers for ~10 minutes, but TCP was working fine, I could SSH into them.

Even Hetzner's Robot interface was down.

Edit: https://status.hetzner.com/incident/5e901309-0873-42af-90c5-...

> We are currently experiencing a fault with UDP forwarding.

Exactly right.

From our logs, the main outage went from 21:01 to 21:10 UTC.

EDIT: Also https://status.hetzner.com/incident/affecfea-fd04-41a1-a680-... (Fault report on Cloud API and Hetzner Console)

Very interesting. That would suggest to me that all traffic is passing through a stateful device vs. direct connectivity from a router. Routers can give simple bandwidth metrics. A stateful device or mirrored ports to a netflow analyzer would be required to create detailed connection logs though mirrored ports wont break state such as UDP virtual tables being full. Are they doing detailed logging of traffic?

It seems to be back up. I can access https://console.hetzner.com/ now, and our VPS are back online.

Yeah first time in 6 years being with them that I've had an outage of this scale. Cloud VMs and Dedicated servers all down / intermittent

Everything went down but seems to have recovered now.

They sent out an email about raising their prices earlier today. Perhaps it was everyone running for the doors.

Reply on news.ycombinator.com