Delusions that companies have about the cloud | Hacker News Reader

Delusions that companies have about the cloud

gigaom.com

61 pointsiProject65 comments

My comment is still awaiting moderation. Curious, since there are other posts after I commented. Oh well, here's the repeat:

tl;dr : Disagreement != insanity. Your unwillingness to accept possibilities you haven't encountered or avoided doesn't mean those possibilities don't exist or can always be avoided.

Point #1: Bandwidth + power is sometimes cheaper than bandwidth + CPU cycle costs in many cases. If you have a massive spike, you may not worry about down time, but you will worry about the bill. There are plenty of occasions where the cost is unacceptable by comparison. And this is even after initial investment.

Point #2: If you stick to a generic storage or CPU package that needs no modification, then yes, go with the cloud. But when, you need to switch priority for some CPU threads without jumping through hoops or dynamically allocate high-priority data to primary search spaces and low-priority data to archives, then you'll suddenly find it comforting to know you have physical access to the servers and primary control interfaces.

Point #3: Evaporates quite quickly when the U.S. government says cloud data isn't technically stored by you, so they can do whatever they please with it. Since you're not the cloud provider, and it's off-site, they can claim "it's not your data" and use it as evidence. Read : Megaupload.

Don't laugh, that's an actual argument used often by them during court cases.

You store your data in the cloud and if you want it to remain private, it better be encrypted.

Bottom line: Any extra "Stuff" you add to the stack is another layer that can potentially fail. More "stuff" doesn't make a better platform, but BETTER "stuff" does.

There's something very comforting about saying "Hey, Mary. We just lost power and we're running backups. Could you make sure the generators kick in?" and hearing back, "Sure, I'm on it."

--- In addition to this comment, I'd like to point out here (since I'm not gonna wait for that comment to be approved too), there were some comments saying pretty much IT folks are shaking in their boots because the "cloud" makes them obsolete. Who do they think run the "cloud's" IT? Magical pixies?

Granted, some local IT people might be worried their jobs may no longer be needed, but if you earn your keep, your job will be safe. Or you'll have no trouble finding work elsewhere.

> There's something very comforting about saying "Hey, Mary. We just lost power and we're running backups. Could you make sure the generators kick in?" and hearing back, "Sure, I'm on it."

What if instead you hear back "Sorry, I already tried turning on the generators but they are not working. What should I do now?"

> What if instead you hear back "Sorry, I already tried turning on the generators but they are not working.

Well I guess you quit in disgrace, because it is your job to ensure you have a viable fallback, and the personnel to make it happen.

One thing I can be very sure of is that I'll never, ever, ever, ever hear "What should I do now" ;) Hell nor high water, we'll have power. One of the perks of working with people who kick ass 24/7/365(6)

We did have a generator truck on site during hurricane Sandy though, since at the time, were weren't sure if the local generators actually would kick in. This is a real problem with standby generators since they're very rarely used (ideally never needed) so they need constant service or need to run occasionally. It's like a car; if you don't use it for too long, it may not start.

    To put it bluntly, Google has assembled the greatest collection of computer
    science talent in the world. Similarly Amazon has a multi-year lead in delivering
    compute power by the drop, with which it’s happy to provide to you with the
    single-digit gross margins of a successful retailer. Your IT organization simply
    doesn’t rate at this level.

Maybe not, but it doesn't need to. Creating, maintaining, and fixing giant cloud services are incredibly hard problems. I'd even venture to say that the ratio of difficulty in maintaining something like AWS compared to managing a handful of servers is greater than the ratio of technical skill of the ops team at Amazon to the average IT team. This is not to disparage the skill of the AWS team but to highlight the orders of magnitude in the difficulty of the tasks.

The point to consider is whether your IT staff can beat their uptime with your own servers.

Right. As long as you have moderately competent people and reasonable resources, and are trying to solve the relatively simple problem of a small to medium sized site, you should come out ahead of smarter people with more resources (but less per-user budget) trying to solve vastly harder problems.

Some good points but for me this misses the insider security risk of these organisations which scale doesn't help with. This could be the company acting against you or it could be an individual acting without authorisation.

Suppose you are a startup that may be of interest to Google in an acquisition, why even give them the opportunity to read your mail by using Gmail?

I also have some concerns that the stacked layers (SaaS tool, on Heroku on AWS) leaves you exposed at all these layers to a bigger pool of insiders and potential creates different opportunities for outsiders to attack although it may well still be more robust to external attack than self hosting.

His argument seems to boil down to 'give us all your data because any concerns you have are just insane!'. It really irks me to see someone dismiss privacy and security concerns with such derision. It's also not just companies who should be concerned. I think the growing unease in the EU about US access to cloud service data is only going to get more serious.

You haven't worked in corporate IT I gather? Security and Privacy are at the bottom of the list when it comes to most businesses IT expenditure.

Unless the law mandates it and provides harsh penalties. Health care and financial companies, to name two industries, spend billions every year on security and privacy.

Spending billions != importance, especially if as you state it's mandated by law any way.

I wouldn't disagree with you though in regards to a large proportion of companies in those two areas. I however was specifically speaking of the other industries, the majority of which still see IT as an unnecessary expense and anything more than "new password every 30 days" as an inconvenience.

I don't know how I can generalize my experience but the research facility I work at prohibits use of Gmail (as POP3 client), AppEngine is completely out of question. Maybe it is less important if the company itself is in the US...

The delusion I have, which he didn't address, is that the price/performance ratio is atrocious. Sticking with Heroku, the $3200/m Ronin DB could essentially be a 1-time purchase if you are collocating, or 1/10th the monthly price if you go dedicated.

Exactly, I have yet to find a cloud hosting service with a price that makes sense to me.

Does Heroku actually charge the face value prices for real customers with that level of need? I would think that discounts would readily be made available to keep the business, but I have trouble finding anecdotes from people who have spent a lot of money on Heroku.

The cost of the database to an enterprise is not dominated by how much the license for the software costs, the hardware costs, or the electricity costs. It is dominated by how much you have to pay in headcount to manage it. At many companies, that app would have two people whose full-time job would be "Do everything Heroku ops would do, except slower and suckier."

I get this same question from some potential customers for AR, too: "Why would I pay you $X a month when I could buy a competitor's dedicated appliance for $X and then just pay for phone calls?" "Who is going to maintain that appliance?" "What do you mean maintain?" "Like, if a security vulnerability is discovered in a technology AR uses, I stay up all night. Do you have a guy like that?" "... No." "Do you know what a guy like that costs?" "... Uh, plumber money?" "Doctor money." "Shoot."

To be fair I'm surprised how often I see a rack mounted solution sitting in an office somewhere with no patches applied since ~2005.

"Your email has always been slow you say? That's because it's been running a botnet C&C server for some considerable time"

The cost can be so out of whack that it can become cheaper to hire talent. Two things make this truer: the person might have spare time to do other things, and sensitivity to things that PaaS are poor at: raw processing speed, memory or SSDs.

I've seen it over and over again with SaaS: server monitoring, logs, statistics, ...

We had an internal fight recently about how to manage our logs. Group A wanted to spend thousands a month on Splunk. Group B spent a weekend setting up logstash + kibana and deploying it in production.

Server monitoring? Nagios. Stats? statsd+graphite. Even if you have to outsource setting it up, it'll probably be cheaper from the very first month.

Two key points:

1) Many companies already have most of these skills, or can acquire them cheaply and reliably via a service contract. The cloud isn't the only way to outsource operations.

2) The performance/reliability constraints imposed by certain popular cloud platforms can create very expensive development work, distracting developers from doing things that actually create value for customers.

I worked with a successful firm that was close to outgrowing their dedicated server and was considering going cloud. They sketched out an architecture that would get everything to fit into nice little EC2-shaped pieces. But there was a problem: it would take a ton of developer time.

Instead, they spent a fraction of the expected developer cost on some serious hardware, and they were done nigh on instantly. Their developers then used that saved time to do things that actually delivered more value to the customers.

The cloud is awesome when it fits, but it doesn't always fit.

If you look at Gmail though, one of the most successful SaaS apps, its great value for small businesses. Many don't pay for it.

Not free for anyone anymore: http://www.wired.com/business/2012/12/free-google-apps/

Free for all existing customers, which is a vast majority.

The author was speaking more about SaaS (like Google Apps), and not just IaaS hosting (like EC2) or PaaS (like Heroku). EC2 and Heroku don't compete primarily on price, but on the flexibility to scale up/down as needed. If you can offer more of a commitment, there are much cheaper options available.

It's a well written article and I agree with his points and have no issues with incorporating cloud services into the work I do.

There are still two outstanding issues that prevent for now, and will continue to prevent, 100% cloud operation going into the future.

1) For the most of the work I do any connection to the Internet is not a certainty and further, thanks to sods law, is most likely absent when most needed.

Rural farms (I'm Australian) can have all access and external utilities cut off by flood and bush fire and still need to function in isolation, ditto volunteer services, remote survey groups, small bands of folks on a jolly.

2) Reliance on the continued goodwill of third parties and their governments.

Like any external service just make sure you have alternative storage for your data to ensure ongoing operations are never held up or held to ransom by <insert circumstances>.

The only beef I have with cloud/VPS hosting is the price.

Conveniently, the data center I've worked with for years[1] supports a hybrid setup, where you can use their cloud hosting combined with coloed physical hardware all in the same VLAN. It's hugely effective, since I can have physical hardware there to deal with the high-load/high-memory stuff there (like database servers) affordably, while at the same time, using VPS for less processor intensive/more ephemeral-load type stuff like webservers.

The result is that I drive to Chicago to work on hardware significantly less frequently than I previously used to, and it's fully worth it. If their cloud system goes down for whatever reason, I know their staff is on it, and I'm not worried in the slightest.

When I was 100% hardware based, I lived in a bit of terror of being out of town. If something happens, I had to get there. Now if something happens while I'm out of town, as long as I have internet access, I'm okay with it. Eventually, when prices go down enough, I'll go 100% virtual, and I'm not looking back.

[1] Steadfast Networks in Chicago: http://www.steadfast.net

Are you considering you loss of time/freedom in that cost analysis? I get that cloud doesn't always make sense in every situation, but in a lot of cases people don't take into consideration these other factors (loss time managing/maintaining hardware, actual hardware costs, etc) when making the decision.

I would say that should always factor into these equations, and I further would say that most companies large enough to worry about these kinds of decisions are indeed factoring those things in.

The cloud vs colo discussions have been done before on HN, but I still say the decision of one versus the other basically boils down to a relatively simple if-else/switch statement: If size of company is small and no sysadmins, cloud. If size of company is medium and sysadmins then cloud or colo based on due-diligence. If size of company is large and sysadmins and you need very high uptime then colo (Unless you're willing to spend a gazillion dollars for several access zones on EC2). Obviously there are going to be lots of if/else chains but you get my drift I imagine.

Now if you're a small start-up and have a likely outcome of tons of scaling needed in short order, your best bet might very well be scaling with the cloud and moving to hardware when you're more established (and if it makes sense from a cost benefit analysis).

Insanity #4; Your data could be accessed by the country the cloud host resides in.

Oh, wait, that's not insane, it's the opposite. It would rather be insane to give a government that's known for wanting access to everything–legal or not– like the US your sensitive data.

This could also be a positive. If you lived in a country where you had sensitive data at risk you could store this data in a country you felt comfortable with.

Though if you'd rather trust your sensitive data to another country you should ask yourself if you shouldn't be in that other country.

> [...] If you care about the reliability, security, and the protection of your data, then you should entrust it to those who are most capable of managing it. If you believe you can match the capabilities and rigor of Google’s Security Operations team, I wish you well.

My previous employer was a mid-size life insurance company. That place didn't use any third party services mainly because it complicated all audits & compliance since data would remain on 3rd party server. I'm sure everyone trusts Google with security. The problem isn't with Google or Amazon or Heroku. The problem is with all these other cloud service providers that aren't Google, Amazon and Heroku, which are more likely to be featured on front-page news because of some rookie mistake which lead to a security breach. The author's view of this issue is for some reason limited to a few big boys in the industry.

"This thinking is backwards. If you care about the reliability, security, and the protection of your data, then you should entrust it to those who are most capable of managing it. If you believe you can match the capabilities and rigor of Google’s Security Operations team, I wish you well."

They may be good, but naturally introduces more attack vectors into many applications.

My nicely firewalled and audited application would become exposed to an undetermined number of people I have never met. e.g. http://www.pcmag.com/article2/0,2817,2369188,00.asp . It only takes one rogue person and you have a problem.

My application has 5 trusted people. It would be interesting to know the additional number of people you are implicitly trusting by using a cloud service. 100?

Unless you also run and operate your own datacenter, the number of people you implicitly trust is fewer but still roughly the same as with a cloud provider. There's all the datacenter security personal, network and hardware technicians that could all possibly be "rogue".

His arguments might stand for Amazon, Heroku and Google, but what about the smaller cloud vendors? Are they as trustworthy, reliant and cheap?

Even if you ignore privacy and security concerns, what do you really get for your money? The cloud ain't cheap. If the service is bad it may be easier to have a student install your web server, put in on the internet and forget about it the next few years.

And then there is there are all these cloud services you don't actually pay for, they are financed by advertising or something. You've got even less to expect there. Why not get your email account from some regional/smaller source? How can the centralization brought on by the few strong cloud vendors be better?

The tl;dr version: Cloud services have exogenous risks that you can't control but trying to avoid them is wrong.

Absolutely false! If your company stands to lose $10,000/hr for every hour it's not operating (and that's a LOT of businesses) then the idea that "oh we'll just trust Amazon for this, what could possibly go wrong?!" is fairly short-sighted. Yes Amazon will work to fix things as fast as they possibly can but they sure as shit will not write you a check for $50k for their 5 hour outage.

And how much des the alternative cost? I'm not even sure what the alternative is, building your own data center?

If you're e-commerce and rely on the kinds of up-time numbers that the post you are responding to references, that is generally the answer, yes (Well, not a full data-center for most guys, but at least your own platform which you know down to the motherboard jumper settings or whatever).

I really don't understand. Knowing the hardware doesn't stop it from failing. There seems to be two approaches here. One, buy really expensive HA hardware. Two, build software that assumes failure and can handle it.

Neither of these guarantee 100% uptime, but you can do the second one on a cloud without knowing the hardware.

Side Note: I've done e-commerce, and e-commerce typically makes all of their money around christmas. It seems crazy to me to buy super expensive HA hardware that can handle your peak xmas throughput, and leave it idle for 11 months of the year.

> building your own data center?

Or data centers, plural.

Now, seriously, if you can beat their reliability, you are in the wrong business. I'll pay you to host my services.

Amazon offers one nine of availability. I'm currently getting three on self-hosted equipment for a high-growth startup, with tens of thousands of concurrent users getting sub-second response times.

The key is having intelligent staff who have a deep understanding of your specific environment. This guy's argument completely discounts the value of that - he's essentially saying that using PHP over writing in C is better because the PHP devs know better than you do how to make stuff work.

My experience with EC2 support staff is that they are mostly concerned with their generic setup and are of no use when anything falls outside of that, and they especially can't provide any help for your specific problems.

of course EC2 has worse availability, but it has many other benefits. The question is whether those benefits are a good trade-off for you.

When you say "staff who have a deep understanding", I'm not sure what you mean. Did they pick the RAID controllers? The UPS?

For how long have you been getting 3 nines? Would you be able to survive a data-center outage? A disk crash? A switch malfunction? A failed load balancer? A failed NAS?

Buying colocation space with a negotiated SLA that provides for reimbursement in the case of outage. And building enough redundancy into your setup that even if a location goes completely offline you're still OK. It's REALLY hard to do that when you know nothing of the hardware and infrastructure that your app sits on. It's still difficult to do when you know all the details, but it's easier to control for risks that you are aware of.

if you have to prepare for enough redundancy for a location to go down, then the cloud works just as well.

I'm really not seeing the distinction here.

It's very difficult to buy a high bandwidth, low latency link from one AWS availability zone to another to ensure that my databases are synced properly such that if one of them goes down the other will stay up, and there will be minimal/no loss of data. But I can buy colo space from two different companies in two different but close cities and gain a lot of redundancy.

Furthermore some kind of systematic problem that affects one of the datacenters run by one organization in one city is very unlikely to affect another datacenter run by another organization in another city. They're very unlikely to both make the exact same mistake and have it bite me in exactly the same way. But the AWS architecture is largely preserved even through availability zones, datacenters, etc. Easier for that to happen.

http://www.zdnet.com/amazon-cloud-down-reddit-github-other-m... That's an example of precisely the kind of problem that I'm referring to that wouldn't happen if you were redundant across multiple independently run datacenters.

That makes sense, and I agree, but this seems orthogonal to cloud vs non-cloud. If there were two cloud providers in different cities, and they allowed you to buy a good link, then the cloud meets this need too.

Yes and no. They won't let me put a link in and have the cross-connect plug directly into MY switch(es) because a core tenent of "the cloud" is "we reserve the right to move your instance whenever we'd like to" which is anathema to the idea of me controlling my machines, switches, routers, internet access, etc.

The idea of the cloud is that "I just need a box that's SOMEHOW connected to the internet and I don't really care about the details of how precisely" which is fine for a great many use cases.

But there are plenty of use cases where the above assumption does not hold. When it does not hold, using the cloud isn't the no-brainer idea that the author of the article makes it out to be.

Here at Disney we both control our machines, switches, routers, internet access, etc; and we use CloudStack for an internal cloud.

I think your idea of "the cloud" is narrower than mine.

Yeah that's entirely possible. If you're involved in building or maintaining Disney's internal cloud you have a lot more information about how it works than some folks on the other coast who don't have the right phone numbers or email addresses to really figure out how things work. You can -- because you control the internal cloud -- set certain instances up on physical machines such that you have all the redundancy that you need.

But "the cloud" as the author describes isn't a thing that you can peer into and determine it's inner workings, it's more of a black box that you can run an instance on. AWS, Rackspace Cloud and all the folks who sell VM instances don't publish all the data necessary or give you "hooks" into the network infrastructure where you'd need to really make a reliable system.

That's the whole point of the cloud. I pay you a very small amount of money for a VM that I rent by the hour and I don't WANT to know the details. Once you start learning (or having opinions) about the underlying network topology and everything else you're not "cloud" anymore, you're something else. What's the right name for it? I don't really know. But I would argue that it's not "cloud"

I wonder how many banks and stock-brokers are buying into handing their net operations over to a third party?

Anyway, once the words "delusion" and "insanity" got dropped, I realized how desperate the clown-computing crowd is getting.

The cloud is a marketing buzzword. You can determine a lot for a product by who is pushing for it - it was coined and promoted by the big corporations that had resources to spare and sell them with a 3-400% markup

http://www.codinghorror.com/blog/2012/10/building-servers-fo...

While I see the value in making things more flexible and cheap like the open compute initiative, my cynicism has taught me to assume every time that what is heavily promoted this way by a entity that tries to position itself as the middleman - it may not be bad for you, but it rarely will be the best for you.

How many people think twice before handing sensitive info to google after the David Petraeus situation.

The cloud infrastructure is just a tool - it has its limited uses, its gotcha-s etc. It is good for prototyping, rapid deployment but after that - not so sure.

The cloud based MS Exchange product that's offered to Australian businesses uses servers based in Singapore.

Whether or not the risk is real or not, the perceived risk of your data being seized due to laws or actions outside your own country is a major point stopping take up in my opinion (at least in Australia anyway).

I have had people attempt to explain "the cloud" to me. Most people don't understand that it's still servers, just in a different place. It was the first time I understood why the term cloud is used, it just makes sense on a white board.

take a survey and ask the average person in America, they will tell you its some new technology in the sky like satellites.

All mentioned things aside, memory-intensive cloud hosted service, like memcached or redis, provider prices are just too damn high ($400/mo for 2GB?). I've found app server and/or worker pricing to be relatively okay.

well , they need to make money somewhere,and it's not on storage that they will , typically services like caching , db backups ,or even DNS,CNAMES for some are expensive.

True risks are largely unknown at this time. "The cloud" to me is akin to the derivative financial instruments. They said that nothing can ever go wrong with derivatives.

However as a professional I can envision many Black Swan type events that would bring down the cloud. Asking Google et al about the quality and safety of the cloud is akin to asking your drug dealer about the quality of the drugs you are buying.

That is not to say that the cloud and complex derivatives should be avoided. However on should always hedge their bets.

An alternative is to use the best computer science talent in the world to design a distributed program that doesn't require the best computer science talent in the world to install or keep operating.

The biggest reason I've seen for not using cloud providers is regulatory. In the UK, principle 7 of the Data Protection Act makes the collector of data responsible for ensuring downstream security. Unfortunately, most of the big cloud providers won't let themselves be audited, and won't accept strong terms in the contract. Until that happens, they can't safely be used for bulk personal data.

(where safely means the data controller is secure from regulatory fine).

Security of data is a moral concern in some circles (e.g. academics outside the USA are quite wary of the Patriot Act) and a legal one in others (e.g. privacy of medical and financial information). Some jurisdictions have specific laws about certain types of data not being hosted across a border. These things may be disagreeable to some, but they certainly are not delusions.

The widespread nature of these delusions is the exact reason why most 'cloud' companies are hopeless without a sales force.

If you're a software development team, setup multiple remotes and push to them regularly.

Totally fake, but setup a bash or whatever shell alias for this: `git push origin; git push codebase; git push github; git push gitlab-host1`

So I occupied an elite circle in Hell when our services failed to deliver.

Revrend!!!

The cult of the cloud is great business for me since I specialize in finding VMM and KVM bugs.

Since corporations want to cheap out and host their previously secure physical boxes all on one server with a wide open console and management system to get into this has made pentesting much easier.

I especially like projects like Whonnix, where fools construct this complex layer of virtual machines on top of virtual machines. It's like taking the shit sandwich of blobs and bugs that is x64 architecture and ethernet drivers, and building a whole mountain of shit right on top then calling it secure.

The cloud definitely has some benefits, but if you're handling finance or require serious privacy better shell out for some OpenBSD racks, virtualize the routing table and set up pf firewalls to isolate the network with real actual isolation and not pretend magic isolation. Best of all there's just one operating system, not three of them piled on top of each other.

derpmaster....working on delivering a cloud infrastructure (VMware 5.0)leveraging OpenBSD-based security appliances (GeNUA)for the security perimeter. Would appreciate any insights on securing cloud infrastructure with OpenBSD. Do you have a website and do you offer any services with respect to cloud infrastructure penetration/security design/security testing?

Would be interested in VMM bugs with respect to VMware.

Thanks.

Ron (ron.szpak@gmail.com)

Reply on news.ycombinator.com