Smaug, the brand new OVHcloud backbone network infrastructure | Hacker News Reader

Smaug, the brand new OVHcloud backbone network infrastructure

ovh.com

59 pointssm2i31 comments

OVH Hardware, support and pricing is GREAT! Buuuuutttttt...

Their firewall situation is not. Guess what, if you use the supplied firewall, any server from any other customer in the local NOC that your server is in, can connect to your server. They seem to be all "safely" behind the OVH firewall product.

You have to protect each server individually with its own in-machine firewall.

I don't want to automatically trust all other OVH customers.

At first I thought I was doing something wrong (more than a decade of setting up firewalls). But I did put in a support ticket and they confirmed this.

Maybe I'm wrong, maybe something I did not understand, but damn... If I'm not.... :-(

That's not really even a firewall issue, just a very poorly configured network. Pretty shameful for a provider of their size.

Can this be mitigated by using something like ufw and restricting the IP access?

That is what the OP said to do in his post.

Unless you use docker, then it doesn't really matter. Really have to make sure your docker containers are not exposing ports they shouldnt be.

Is there a good tutorial you can recommend on how to audit this?

No, but I would start with a `docker ps` and see what ports it's forwarding.

> You have to protect each server individually with its own in-machine firewall.

That's the standard practice?

OVH's own firewall is for DDoS/DoS protection, not for fine-grained security, did I understand OVH's information incorrectly?

Ouch. That probably means there's a metric shit tonne of VMs running Docker with open ports in their data centres.

Saying that because (by default) Docker screws with firewall rules on the VM when it starts up, to allow other hosts to communicate with the containers.

In other hosting environments, the workaround is to apply firewall rules to your VMs using the hosting infrastructure capabilities. eg separate to the iptables (etc) rules on each host

Yes. There are two different products: VAC, which is for DDoS protection, and general SDN firewall/security groups which is only OVHcloud (not dedicated servers).

In the most general case, it seems other customers can actually send DDoS/volumetric traffic toward you from within OVH and it doesn't get picked up.

Because I was thinking about OVHCloud, they dont have an VPN with private IPs? And load balancers like AWS/Digital Ocean?

I had the worst experience from a vps/dedicated hosting provider with OVH few years back, long story short I had a dedicated server with software raid, after a month, one of the disks failed I gave them all the details SN of the disk at fault etc, but apparently the removed the good disk and I lost the server, I asked them to put it back and they told me they had destroyed it, luckily I had backup. Lastly I asked for a refund they didn’t give anything back.

I moved to hetzner immediately, and I haven’t had such issues till today.

I know that you can’t expect much from cheap providers but OVH is extremely unprofessional in my experience.

Amazon did about the same exact thing to me a while back, so you're not alone and it's not just cheap hosts that make that mistake as we spent 10k/mo on support alone. (AWS had EBS silently fail which is awful enough but then restored data from the 'bad leg' of the system and lost all. To this day I've never trusted them again - maybe I should get over it but, would you?

I can't imagine that any of the ebs recovery isn't fully automated now, if it wasn't then.

Guess you were unlucky; I host with them since 2004 and it has been good with 100s of servers.

Notice how there’s no info on IPv6. That’s because OVH has horrible support for IPv6 and requires non standard routes to be set because they don’t support router advertisements. They also rely on ND packets and not static routing for IPv6, and also block outgoing IPv6 packets if an incoming IPv6 address has not been established. I would avoid OVH.

You’re being downvoted, but a hosting provider redesigning their core network infrastructure in 2020 without proper IPv6 support is really bad imho.

It’s unacceptable that all these (cloud) hosting providers collectively make ISPs look good.

I miss OVH's old control panel. It was so nerdy and to-the-point, unlike their newer modern interface that heats up my CPU with boatloads of javascript, and adopts the 'flat' design pattern that has now permeated every site in existence.

Yeah it seems every time I log in something has changed ...

What they describe sounds to me exactly like standard architecture for combining PoPs and peering with backbone providers. What am I missing?

Nothing really, they were on worse designs for years and they came to a point that it couldn’t scale, so they had to come up with a new proper design

OVH is an absolutely shitty company. I've seen a tremendous uptick of spam from OVH that they're happy to simply ignore. The same kind of spam using the same content, the same registration patterns and the same template have existed on their networks for many months in spite of constant abuse complaints.

I can't imagine why anyone would want to run anything on the same networks that OVH uses to host spammers and scammers.

And good luck talking to an actual human at OVH if something goes wrong.

and their APAC backbone: http://weathermap.ovh.net/#apac with Singapore somewhat migrated to the new architecture

Just a reminder, from Wikipedia (https://en.wikipedia.org/wiki/OVH#Email_spam):

  As of November 2019, OVH is listed by The Spamhaus Project as the world's
  second worst Internet service provider for the proliferation of unsolicited bulk E-Mail

https://www.spamhaus.org/sbl/listings/ovh.net

Looking at the same list now, it recently seems to have added fraud, and many malware distribution entries too.

This is like saying Google is the search engine with the most links to malware pages. 36 IPs is nothing given how big ovh is.

Those entries aren't all singular IP addresses. Some are ranges (etc).

Picking one at random:

https://www.spamhaus.org/sbl/query/SBL492369

That's showing a fair number of IPs.

Is this the list you are referring to? https://www.spamhaus.org/statistics/networks/

Seems like OVH is not on it anymore.

Cool. Sounds like they've put good effort into cleaning things up then. :)

It's the biggest European cloud provider.

That’s good, it means they won’t bother me about whatever I choose to host. Thanks for your recommendation!

I can't speak to reports but I can testify about server logs. OVH IPs have been a top source of spam and attacks against my (US) (<10) servers for most of a decade. I'd add Digital Ocean to round out the top 2 list.

If a firewall goes offline for 60 seconds, I will get hammered from OVH/DO networks. Not exclusively but they're the standout kings. Just think Psychz networks, but scaled up.

I know OVH's size plays into that. But size here is less about the number of net blocks and more about their bureaucratic disinterest in abuse (common to larger hosts, inc hosts I like).

There are comparable sized hosts in the US (AWS, Azure) but when it comes to crapty traffic, OVH makes them look small and insignificant[1].

Unlike moderation at scale, known attacks are often qualifiable, detectable patterns. Can we please care enough to notice & maybe eventually, one day interrupt them?

[1]disclaimer: Spam from Google/Azure & malicious SMTP traffic from AWS totally dominated the first ½ of this year. IDK why. It's since died off - which differentiates them from OVH/DO.

Reply on news.ycombinator.com