4800 Aussie sites evaporate after hack
smh.com.au
smh.com.au
I have multiple copies of my own personal websites in all kinds of different locations, stored both at home, as well as on remote servers, and backed up within the cloud.
Any eCommerce company needs to realise that their livelihood is at stake and that if they can't get back online within a day then maybe they are in the wrong business.
He never tells them about correct backup handling or much else, really.
I agree that if you don't know how to get your business back online within a day or so, that you don't deserve to be running a business and that you've taken that risk on by trying to cut costs/corners elsewhere. Still, my idealising doesn't stop people from doing it.
I'm sorry for people when their web sites are disrupted. I am astonished when it 'kills their business.' One would think that in this day and age one would 'restore from the last backup' and move on.
So is there a web opportunity to add a value "over" AWS/S3 which is the equivalent of managing a strip mall property in the real world?
Having worked for a hosting company in Australia, I can say that most of the customers who suffer these problems are 1) small business owners and 2) not tech savvy.
Most of their sites are pre-packaged e-commerce solutions, and the idea of doing a weekly DB dump + /www tar is just beyond them. They would probably be better off with something like Shopify, but many of them also want local support & like the idea of "DIY". Unfortunately for them, doing it yourself means you're handling all the risks.
I still think many people "DIY'ing" shouldn't. The sites often look horrible, have no way to track stats (conversions), etc.
What would be better is it if the hosted e-commerce players really pushed to market to these people who think DIY'ing is the better way.
Chuckle. Agreed 100% however folks want to be independent, even when they aren't as good at it as they might think they are. From the psychology perspective if you can make a system that "feels" DIY but really isn't so DIY that the final product looks DIY, that is the value proposition in this market. You sell people the notion they can "do it themselves, no need for techno-nerds telling you how you're doing it wrong!" and then you give them a design system on rails for which the 'exit' points are all decent web sites.
As mentioned earlier, bandwidth in Australia is prohibitively expensive so the choice wasn't as simple as "use both".
We not only back up all of our hosted websites every day, on behalf of our clients, and make those backups available to our clients going back 120 days so that they can restore any files they accidentally lose on their own, but we also back up all mail accounts every hour and can restore individual mail messages if necessary. All of the backups are stored in a private location separate from all of our servers, which are scattered across the country.
To have 4000 customers and not have even a whiff of that kind of architecture is, to me, completely and totally inexcusable.
Maybe this isn't viable for budget hosting. But even the host's website is unavailable, obviously that is critical for their business and it appears it itself wasn't securely backed up. And this is hardly a totally unexpected scenario, it should have come up pretty quickly in a "what could go wrong" stage of their backup planning.
All I'm saying is that from the outside it's easy to take a fairly simple view and propose a technical solution. This looks like a very malicious attack designed to take down the business, the sort of thing that doesn't happen without a reason. If this is the case here then the prolem changes from one that's purely technical to something bigger. Something that can't have a purely technical solution. (Note: I'm not part of distribute it as a may be suspected by the fact that I'm new to HN.)
IMO, anybody that decides to venture into hosting should behave as though they've just walked into a warzone with a huge red bullseye on their back, and take precautions accordingly. Yes, there are certain things that just aren't immediately feasible before you launch, but the goal should be to get basic redundancies online quickly while you're operating.
By the time you have 4,000 customers, if you don't have backups for your backups, you're being negligent. And I say that without any malice whatsoever ... my experience with a lot of hosting companies, both big and small, is that Distribute wasn't doing anything out of the ordinary.
The thing is, targeting backups isn't new at all. It's been done before, and made the news before; at this point, it's not something that should surprise a sysadmin. i.e., the thought process immediately after setting up your backups should be, "OK, now what happens if a hacker tries to hit them too?"
So, yes, this is armchair quarterbacking, and yes, this is common behavior in the industry. But that still doesn't make it excusable in the least.
EDIT: Just to expound a little more on this, the reason I have such a hard-line stance on this is that, as a hosting provider, you are effectively taking responsibility for your customers' data and, in some cases, their livelihood. Yes, ideally, every customer would have their own backups and could move themselves to another host within an hour, but the reality is that it doesn't happen that way. Customers often have websites whose only copy is on your systems, email that's stored only on your systems (because they habitually use webmail, a service that you provide which makes that problem possible). Having "not our responsibility" in your TOS is very much not enough; you must be taking every reasonable precaution to safeguard your users' data, and in this case, Distribute -- along with many, many other hosting providers -- was not, because they did not have secured backups.
"I am a apple fan, but you can be assured that I will NOT be using iCloud for this very reason. Always have a back up and NEVER rely on cyber space."
"Moral of the storyis GET YOUR HEAD AND EVERYTHING ELSE OUT OPF THE CLOUD(S)"
"Like others on here, I feel that the rush to the "latest and greatest" IT thing of the cloud has serious implications - when you are not in direct possession or responsibility of your data, and something happens to it, what do you do?"
"This is why cloud computing is destined to fail. 'Cloud Shocks' like this will force people to rethink the whole concept and write cloud computing off as just a 'fad'. This isn't the first cloud shock and will not be the last."
The company in the article wasn't even a cloud provider...and further more, if they were a cloud provider they would have more chance of getting their data back. Some people just don't know the difference between the internet and cloud computing..
This isn't helped by the current trend for their articles to be link bait or scraped off a real tech source, rather than anything approximating journalism.
The general level of reader commentary on newspaper websites is pretty pathetic, no matter what the subject matter is.
As much as I despise The Herald Sun and News Ltd., they do the whole online community thing better and the site is better for it.
Bingo
Does anyone know of a security auditing service that customers could look for when evaluating hosting?
I run a blog network (http://ozblogistan.com.au) on a pair of Linode VPSes out of their Fremont centre. I periodically reprice hosting in Australia, but basically bandwidth charges make it prohibitive to host locally for the amount of traffic I get.
I'm talking $100+/mth differences.
Still, independent backups are vital. I use tarsnap.
It was certainly true 5-10 years ago, and may still be true for older people, but my experience has been that you get better support overseas now anyway. The biggest issue for me has been that scheduled downtimes are usually in the middle of my day, but you can work around that.
It's not Telstra. There's plenty of non-Telstra international transit, and enough (but not plenty) of non-Telstra domestic transit.
The problem is that to run a hosting business in Australia, you don't get huge benefits from scale, and you're having to maintain a high staff member-to-customer ratio. Customers have a lot of protection (a good thing) through Fair Trading, and it seems that (from my experiences working in the industry) Australian users have a higher expectation of customer service.
Plus, there's also just the time-zone thing. Calling up mid-afternoon in Australia puts you at late-night in the US.
For example, from my work on a fibre connection, pinging a local website takes about 2ms, and a US website about 160ms. Double that for residential DSL.
Also, national bandwidth is essentially free, and bandwidth to the USA is $$$.
It's generally billed at the same rate for all traffic without distinction. By hosting in the USA you assign the cost for bandwidth to users.
The economics of our ISPs and hosts are driven entirely by the price of about 3 major pipes out of the country.
My reply was in line with the parent post that the (implied) reason that Aussie business are reluctant to host overseas due to ignorance of options, perceived issues around support etc. From my dealings with Aussie business (of the sort that would host at lower tier hosting firms like the one in the article), they are ecstatic just to have a website. Issues such as latency don't even come into consideration.
http://radar.oreilly.com/2009/07/velocity-making-your-site-f...
So potentially, the money you save by hosting overseas is lost in lower conversion.
At least hold the data for ransom or something.