URGENT:
We have just identified a malicious activity on Cloud 66.
To protect your servers change your API keys and enable termination protection on AWS accounts you have.
We strongly recommend changing all cloud keys.
We have shut down the site and will keep you posted.
Essentially it looks like somebody on Cloud 66s side found a way to remotely destroy hundreds of instances.
---
Today we had a major service incident on our site. As a result of this incident some of our customers lost their virtual servers.
We are still investigating the cause of the issue and our service will be shut down until the investigation is over.
# Here is what we know #
- There hasn't been any signs of security breach or abnormal activity anywhere on our systems.
- All sensitive information is encrypted throughout the system, including cloud API keys.
- The affected stacks were across Digital Ocean, AWS and Rackspace.
# Here is what we are doing #
- We are working hard to find the root of the issue, but we need to keep the systems shut down until we are sure our customers are not exposed.
# Here is what you can do to restore your service #
- If you are not affected by this issue, you will not be able to redeploy until the service is restored. We will keep you posted.
- If you are affected by this issue, we can help you with your latest deployment Git SHA (if you don't have it), redirecting your traffic from our DNS.
- If you are affected and were running on Digital Ocean, they might be able to restore your server from an automatic pre-destroy snapshot they take.
We are very sorry about this and understand the disruption it has caused to all of our users, we are working hard to restore the service as soon as possible.
Let that be a lesson.
So, absolutely zero information coming from Cloud66. Time to find a new provider.
Yeah... that's an odd reaction.
Here's the original announcement that explains it quite well: https://news.ycombinator.com/item?id=5213862.