OVH Security Incident | Hacker News Reader

OVH Security Incident

status.ovh.net

123 pointsbjonathan54 comments

OVH has come a long way. They used to be cheap and bad at service and totally incommunicado about any issues. Then a few years back something changed and they started to work on their image. Their still cheap, but their service is good and getting better and they seem to have nailed the communications angle. Good for them. Between OVH, Hetzner and Leaseweb the EU hosting space is doing fine.

Wasn't there a recent thing with their CEO hating Github because of "Githubs agenda of poaching developers" or some shit like that?

Oh yeah: http://www.ovh.com/fr/a1136.interview-github-octave-klaba-ov...

Google translate: http://translate.google.com/translate?hl=fr&sl=fr&tl=en&u=ht...

I've always been a huge fan of OVH in my previous company. I had never seen this article. I was planning on moving my company's infrastructure from Rackspace to OVH. Reading this article makes me reconsider my decision.

Note to future CEOs: your opinions matter when it comes to getting/keeping customers. Customers not sharing your opinions are likely to leave, if they can, or not subscribe to your services if they're not already subscribed. Bashing github is certainly the best way to alienate the developer community from OVH. Moreover, stating that the reason was because one of their developer got hired by another company makes me wonder what must be the working conditions at OVH. I had thought one day applying there, I won't think about it anymore.

Their CEO is not the brightest guy in the universe. This security notice is well written and signed by him... but obviously not written by him.

I might add that OVH has a US arm as well - prices are competitive with Hetzner at least.

I've been wanting to try them. Does this security breach have any implications for the US data center, or was this pretty localized to the European arm?

Did you RTFA?

We tried a number of hosting companies before settling on OVH including the premium dedicated companies. They do have the occasional hiccup but overall we have been very impressed with the pricing and the support both reactive and proactive. The day we signed up for our 50th server we received a nice personal email from the head of their support in Montreal.

We have over 25 machines with them. How do you cope with having to pay individually for every single machine, every single month? Their billing system is non-sense, unless I'm missing something?

It is annoying but the cost of having someone sit there and click renew for 4 hours once a month is still less than the $35k extra we would have to pay to host elsewhere.

The level of transparency in this report is great. Especially compared to things like the Linode incidents.

OVH is always very good with transparency with its client. That's part of why I use them for my server, and my ADSL2+ (soon to be VDSL2) at home: when there is a problem, I know what it is and they report their progress solving it in live. Also their customer support is great and doesn't take you for a foul, if they feel you can take technical details, they will gave them to you and really discuss with you about the solution.

Your interactions are pretty much opposite of mine. OVH is constantly doing stupid things to their network (like updating all their BGP reflectors at the same time last week and then falling out of BGP because of it). What they have in transparency and they lose in being a bunch of asshats to deal with. They've taken a week to replace a blown PSU (caused by their own inability to wire a rack); they've dumped my servers because their own internal billing checks weren't aligned with my actual billing details; ffs they essentially root all their servers with their default install image.

I won't even go into their blatant breaches of contract...They toss a ToS at you, but you can only read it when signed into your manager and you can only sign into your manager if you accept the ToS. They won't email you the new ToS either, you have to accept it without being able to read it.

> a blown PSU (caused by their own inability to wire a rack);

Please explain. I spotted the BGP failure but this happens from time to time with basically everyone.

A set of dedi's a bought as soon as BHS opened were on a rack that wasn't wired properly, an electrical short blew a PSU.

I believe we also had a bunch of servers in that same rack. Downtime was around 5 minutes from memory.

Yeah, as great as Github... Oh wait!

This is how you do a security incident disclosure.

I hope Apple is taking notes.

Are you referring to the recent disclosure, or just in general how they do it?

I'm just referring to this specific post from OVH (who I hadn't heard of until today). In reading it, I get the impression that they've handled the situation extremely professionally, and are setting a great example.

"The encryption password is "Salted" and based on SHA-512, to avoid brute-force attacks. It takes a lot of technical means to find the word password clearly"

"clearly?" OVH is wrong. Based on this information alone, it is not sufficient to say how costly it is to recover the password. SHA-512 needs to be iterated to make it costly to brute force.

For example, a raw SHA-512 hash, even salted, is not iterated and is easy to brute force. But multiple passes, as in crypt-SHA-512, are iterated and very costly to brute force.

If one's password is "password", the choice of hash algorithms and salts is irrelevant anyway.

Yes and no. If you salted a plaintext using any hash algorithm, the biggest advantage is on the population, not an individual password; so yes, it is irrelevant for any single password. If you have comprised a database, and you are interested in a single user, then you can easily recompute months. But if you are interested in compromising a large user population, because you want to find the weakest (like find all the users using 'password' or '123456'), then you have a hard time because with salt you cannot find occurrence. People think salt saves everything, that is not true. It only adds hardness to secure a population, not an individual password (because the salt is appended and stored!). Now when it comes to iteration, sure, adaptive hashes like bcrypt or scrypt can make computation longer. That's all. Using sha512 is still wrong: it's a hash designed for fast hashing, not for securing password. bcrypt is slowly broken. Slowly because workfactor is exponential.

I think you may be reacting to a mis-parse. The sentence is awkwardly phrased either way, but "clearly" could mean "it is clear that it takes a lot of technical means," or it could mean "to find a clear version (i.e. plaintext) of the password."

It's an ESL issue. He meant to say "in clear" as in "plaintext" rather than "clearly", clearly.

Definitely: the french for plaintext would be "en clair"

Hacked again?

https://bitcointalk.org/index.php?topic=186902.msg1936161#ms...

Quite a different incident, I think.

If I was a customers, I'd be asking if "based on SHA-512" means some kind of iterated algorithm, or if have they lost my password?

Maybe SHA-512, salted?

Yep, in the french version of the message they specify that it's salted.

http://travaux.ovh.net/?do=details&id=8998

Salted SHAx passwords are basically the entire reason GPU John The Ripper and oclHashcat exist, although SHA2-512 is significantly slower than SHA2-256, so if you're going to use a terrible SHA-based password hash, SHA2-512 is your best bet.

SHA512 is slower on most (all?) current GPUs, but there is plenty of hardware on which it is faster than SHA256.

Hm. Example?

It'll (theoretically) be faster on any hardware that supports 64 bit operations, as SHA512 ends up doing fewer block operations.

This assumes you're hashing something at least 8 bytes long, and that your hash implementation is smart enough to use the 64 bit capabilities of your platform.

That jumped out at me as well. People need to understand that even cryptographic hashes cannot be considered irreversible for short inputs, and that you need to use something specifically designed as a password hash to do it properly.

http://throwingfire.com/storing-passwords-securely/#notpassw...

http://codahale.com/how-to-safely-store-a-password/

It doesn't matter; you should assume your password is lost.

Follow their advice: "we advise you to change the password for your user name."

It matters a great deal (well, not to me, I don't use the service). Password reuse is common, and the way to fix that is good quality password hashing. If I used OVH, I'd need to change my password everywhere now.

If I used someone competent (i.e. they still have bugs but they use bcrypt), they'd have lost: $2a$10$NkYCXBjWeVP0rJUlfl0VL.d66EvJjbVUA/YEsmBSyTZOnbY0/anxa which is a bcrypt hash of my most secure password.

I'm happy to publish that. The salted SHA-512? Not so much!

Password reuse is a really bad idea, regardless of how the password is stored. Depending on how the server is comprised, your password could be intercepted before it is hashed. Don't reuse passwords for sensitive stuff.

And yet.

Password reuse is still bad, your password doesn't have to be breached through a database dump. You could be accessing a compromised site, or get caught by any number of network attacks from stripping to mixed scripting to dumping a stream that doesn't have forward secrecy. https://www.imperialviolet.org/2012/07/19/hope9talk.html

My point was, if they hash properly, a simple db dump is not a big deal (although obviously considering the password compromised is sensible, and password reuse is a bad idea which is not a proof people don't do it).

I'm signing out of this thread as my point seems to have been lost somewhere. Thanks for the interesting link though.

> I'm happy to publish that

foolhardiness is not bravery ;)

No, the password should now be considered compromised regardless of how it was hashed and regardless of how many other sites you used it on and how annoying it would be to change them all.

$2a$10$NkYCXBjWeVP0rJUlfl0VL.d66EvJjbVUA/YEsmBSyTZOnbY0/anxa == hunter2

"based on" could literally mean "some shit our lead developer cooked up that's like SHA but not really."

I would hope "based on" doesn't mean that but why use the phrasing at all? Either it's SHA-512 or it isn't and if it isn't, confidence is definitely not instilled.

Not everyone is native english speaker. He means the password is salted and SHA-512 hashed...

"After internal investigations, it appeared that a hacker was able to obtain access to an email account of one of our system administrators."

That translates to password reuse, or an insecure password.

...or a client attack, or XSS, or poorly secured tokens, or whatever. If we always blame the user first, we're bound to miss something. Even if the fault were an insecure password, the admin site would still be to blame for not throttling and locking down the account in response to repeated attack.

Fair point Jess.

  An email will be sent today with the new password

Password in plain-text? I understand the convenience factor but doesn't sound very secure...

If someone can read your email, they don't need your password. They can just initiate a password reset.

  https://www.ovh.co.uk/cgi-bin/nic/nicPassword.cgi

But yes, in general, email isn't a very secure method of sending passwords.

Reply on news.ycombinator.com