Stay away from Digital Ocean | Hacker News Reader

Stay away from Digital Ocean

gsundeep.com

34 pointsgsundeep30 comments

I work for large host here is my take on this

1) They shut down my servers and suspended my account without notifying me. If your server is doing 1Gbs outbound I would expect them to shut it down ASAP for many reasons. Likely a more senior person detected the issue and shut down the server, then passed it on to 1st line support to contact you.

2)After contacting them, they immediately accused me of "launching an attack from their platform". You launched an attack vs a sever you are responsible for launched the attack, it's semantics but the support agent could certainly have worded it better, but not a big deal in my view.

3)After they performed their "investigation", they refused to give me any details and kept my account locked and servers offline. The investigation probably wasn't any more than their networks team detecting very high outbound traffic originating from your VPS IP that was clearly malicious. In that case the only explanations are that you are deliberately breaking their terms or your server is compromised, there is nothing else to discuss. They should really have sent you some standard text helping realise the reality of the situation but it wouldn't have hanged anything.

4)They refused to give me a copy of my data which delayed getting it hosted elsewhere (had to get ssl certs reissued etc). This is not great but if your server is compromised you have to accept that you will need to work from backups. First line support probably don't have a way of getting the data without starting the machine which they can't do in the circumstances. I'm sure one of their sysadmins could mount the virtual machine as a disk image to get the data for you, but that's not part of the service they offer or you're paying for. They can probably get your data given time and/or if you pay them but it's not reasonable for them to do this as a priority, especially as they are a budget host.

This was the poster's fault. They failed to secure their box and then neglected to take responsibility when it was compromised.

How was OP supposed to take responsibility for a compromised server without access to the server?

Why would Digital Ocean give them access to the server ?

If the server is compromised then it should be permanently shut down immediately.

Can somebody please comment on what the risks (security, legal or other) are for a service provider that prevent them from at least giving the customer his/her data back?

Most likely they just don't have any easy way to do it without reactivating the droplet. If so, there probably is some higher-level admin who could do it, but $5 a month doesn't buy you that support plan.

The reality is that unless you are a really big enterprise customer with your own legal team who has negotiated custom contracts and SLA's then the host can destroy your data at any time. You have no comeback unless they admit liability or negligence, that's one of the reasons the support agent could not discuss details.

Are you saying it is impossible for a secure server to be compromised?

This post was really insightful, they shut down my servers and suspended my account too without notification. I thought that it only happened to me, but it looks like it's a communal problem. Thanks!

Sometimes I wonder if people act hostile to support staff from the get go in order to get a response so that they can build up an interesting communication history for a blog post.

Seriously though, why not be nice first, and ask for the things you need, before saying you are gonna post publicly about something. You have to consider that maybe the person communicating with you is just a support flunky that doesn't know how to grab the data you need off of your disk images.

Think of the amount of skilled labor they would need to expend to get your data off the disk images. Even if its just an hour, most engineers cost $80-100/hour. If you are paying $5/month for your basic Digital Ocean instance, that means more than a year worth of revenue on your account needs to be spent to make you whole, because you fucked something up and allowed a third party to exploit your server.

If you don't make backups and don't administer your websites correctly, I think Digital Ocean will probably be happy to see you go, since you are probably a negative revenue customer in the long run. They send you to Amazon or Google with their blessings.

edit: they should definitely have emailed you to let you know that they were suspending your account though.

Coming next month: this guy gets banned from Google and Google refuses to explain why.

This seems pretty much the same pattern as my issue with DO last month. Better to move your service to AWS or Linode.

In Case anyone wants to check.

The Blog Post : http://serdardogruyol.com/?p=122 The HN Thread : https://news.ycombinator.com/item?id=6438761

I agree with "you get what you pay for" sentiment. With their margins, I can't see them having the support for any more than what they did. I think the takeaway here should be:

1. Backup your data offsite.

2. Monitor your servers or at the very least, your sites. If a site is important enough to pay for hosting, I would think it would be important enough to monitor for uptime. Bonus points for monitoring server resources as well.

3. Trust noone. Who's to save Linode, Amazon, or Google would act any differently if they detected a compromised server on their network.

These things should be done regardless of how much you are paying for your hosting.

As with so many other computer disaster stories, the moral of this one is, always keep backups. Even if your data lives in the cloud, you still need off-line backups.

Couldn't agree more.

Coincidentally, early this morning, I received a notification email from a droplet i own. It(my security agent process) had noticed repeated attempts to login and that it had blocked the offending IP address. When i traced the IP address, it was from Digital Ocean's network!

I am getting unnerved with all of this. Looks like DO is now a popular target for hackers. I would think the vast majority of droplets (despite their owners trying to keep things secure to the best of their knowledge) don't have much of a chance against a sustained attack by a capable and determined hacker.

So yes, I agree that keeping as near to real-time backup as possible seems to be the only takeaway from all of this (assuming that we would want to continue with such a discount hosting provider).

Sundeep was almost immediately hostile to the support people. While DO clearly could do a better job communicating to him, he's not helping himself with his attitude. Support folks often do not have the privilege to go off-script and make a positive difference for a customer, but they can always stick closely to the script and "regret any inconvenience" while being unhelpful, which is what I see here.

Did they provided more info about this ? I'm still very worried, I posted a question about this situation including a link to your website and this post and I think it's not available in their website. So, not sure if I should trust them or not, I use it for personal use and been advising my company and we have lot's of projects running there, client work, etc... : X

Even if Sundeep were at fault, at the bare minimum, D.O could've informed him that they are shutting him down. Also allowed him to get his files back. Neither of this is hard to do, even for a $5 host.

I have been thinking about using Digital Ocean (moving from Linode) but I have heard a few stories like this that really make me second guess that decision.

Do you really expect a $5 VPS to be allowed to continue a heavy DOS attack? I wouldn't want to use a provider that did allow that.

I would expect them to email me and let me know at least, particularly if I was paying them hundreds of dollars.

I think they're probably inundated with novices who don't do basic securing of their VPSes and experiencing a lot of abuse as a result.

I can see why it was brought down, but it would definitely be useful if the owner of the account and Digital Ocean could work together to resolve the issue.

At least some sort of notification should be in place.

Wouldn't the fact that Linode is a security nightmare that consistently lies to its customers a bigger reason to move?

What Digital Ocean did here seems completely reasonable.

Not sure what you expect for $5 a month.

Just curious, what were you actually trying to do that made them think you were launching a DOS attack?

Is that the last message they sent?

Yep, I haven't received a response to my last message. I've rehosted elsewhere though so I am not worried about not getting my files back.

<s> Guilty until proven otherwise! </s>

Very troubling. I will not be signing up with DO.

They work on razor thin margins so they shoot first and ask questions later. They don't have the resources to provide top notch customer service or investigate in depth, or prices would be much higher.

So you get what you pay for, I wouldn't move my money making sites to a service like that.

Reply on news.ycombinator.com