Secret History of OpenStack, the Free Cloud Software That’s Changing Everything | Hacker News Reader

Secret History of OpenStack, the Free Cloud Software That’s Changing Everything

wired.com

67 pointsdaegloe22 comments

A nice read and informative about the background of OpenStack, but doesn't really give the reader any information about what state OpenStack is really in and what trying to use it is like.

I work for a small to medium sized web hosting company and we looked into using OpenStack as a solution for offering a Cloud product, and the problem is that OpenStack is very much in a dough-like state. It's great if you want to bake your own solution. But if what you need is bread to build a sandwich on, you're going to be starting a lot further back in the proces than you maybe want to.

I also fundamentally disagree with the whole direction of playing chase with Amazon. Linux didn't supplant Windows by copying it; it supplanted Windows by offering something that was better. If Linux had merely been free and Windows compatible, I don't think it would be the success it is today. It was not just free and open, but also technically superior.

I think OpenStack had -- and may still have -- an opportunity to provide a much better platform than what Amazon has. EC2 has a lot of design flaws and faults which result not from technical limitation but just bad design choices, and there's no need for OpenStack to follow in them.

The biggest of them in my mind is the flavor issue. In the EC2/OpenStack model, you cannot have customers spool up VMs with arbitrary specs; they must choose from available flavors. So that means if you want to offer a wide variety of options, you have to have many, many flavors on option. Which then leads to the customer having to pick one of them -- or to obfuscate this issue behind an interface which can match up the user's choices with the appropriate flavor. Neither of which is a good solution.

There's no reason a cloud VM platform should have this kind of restriction. Other competing platforms don't. I think OpenStack will eventually be supplanted by something much better. It's just that this is a very new industry, much more like competing computing platforms in the 1970's than the much more mature battle that happened in the 1990's. Where even though Linux was new, it was based on 25+ years of prior work and experience.

I agree that IaaS can be done much better. The networking and storage that's available is terrible; it's worse than ESX 1.5.

In the EC2/OpenStack model, you cannot have customers spool up VMs with arbitrary specs; they must choose from available flavors.

Are you talking about sizes? If you allow arbitrary sizes or CPU:RAM ratios you'll get fragmentation which you have to recoup by charging higher prices. If VMs are 1/2^Nth of a server, you can use a buddy allocator.

Yep talking about sizes (OpenStack calls them flavors, or at least did last year when I was looking into it.)

I realize it creates fragmentation, but it's one of the key features customers want -- and should be able to get -- out of a cloud system. And other platforms provide it.

Linux did not supplant Windows by copying it, but it did supplant proprietary Unix. Most (not all) really successful infrastructure open source projects tend to be implementations of standards (MySQL/SQL, Apache/HTTP, Bind/DNS, etc.) In this case the defacto standard is AWS and there is certainly a lot of value in offering a compatible alternative. For me, the main issue is that most people want to use a cloud, not run one. Obviously your case is different because you are a web hosting company that want to compete with Amazon. My take is that it is going to be incredibly difficult to match Amazon technology and pricing and that existing web hosting companies will get consolidated and disappear. Please note this is not what I wish and I am not arguing this would be good for the ecosystem either, just that what it looks to me it is heading to.

A couple of things here, I'd say:

- EC2/AWS may be the de facto standard but it's not an actual standard the way SQL/HTTP/DNS are so I don't think the comparison is totally accurate. I think there's plenty of room to supplant AWS by offering a better system.

- There's plenty of room to compete with Amazon. Despite what you might think their prices are actually pretty high with plenty of room for even small hosts to undercut them. There's a reason that it's been so successful for Amazon; they are making a mint on it.

The big problem for smaller hosts wanting to compete with Amazon is that there's currently no ready made cloud solution that's 100% end user ready. Every single solution on the market requires some amount of custom coding in order to bring it up to end user quality. Which is frustrating.

A de facto standard is still a standard. Windows was the de facto standard in desktop OS in the 90s and there was plenty of room for technical improvements. But because it became standard and the network effects kicked in, there was little incentive and a lot of cost for ISVs, system integrators, etc. to support alternative platforms. The problem for smaller hosts is not that they don't have a ready made cloud solution, it is that they are never, ever, going to be able to match Amazon in terms of scale or efficiency. ISVs are going to support the platform that allows them to reach more users, users are going to choose the platform that has the apps/functionality they want. People are going to need customized cloud solutions, but my take is that they will be provided by third parties on top of Amazon, not as an alternative to it.

Most (not all) really successful infrastructure open source projects tend to be implementations of standards (MySQL/SQL, Apache/HTTP, Bind/DNS, etc.) In this case the defacto standard is AWS...

But x86 HVM, Ethernet, IPv4, IPv6, NFS, SCSI, etc. are also standards... that EC2 doesn't support (fully). There is room in the market for standards-based IaaS IMO.

There are room for other solutions, my point is that they are going to be marginal at best if the only differentiation factor is that they support more standards.

I didn't really explain myself well. The standards I mentioned have features that EC2 doesn't have, and those features make apps cheaper, more reliable, and easier to develop. Also, standards compliance makes it easier to migrate between cloud and non-cloud. It's not just because.

I am not sure the standards mentioned help make the apps more reliable, cheaper, easier to develop, at least in the traditional "higher level of the stack". Most people associate offerings like Heroku to those qualities (and they build on top of EC2). But even assuming that is the case, once the ecosystem is built around a standard platform (Windows in the 90s in my example) then no amount of technical features can effectively compete with it. You need a different approach. Microsoft in the Desktop was not beaten by a better Linux Desktop, but it will be by a combination of Google apps, the iPad, etc.

“Whenever I would talk in Washington about this cloud technology enabling data centers to run without people, this was interpreted as jobs going away,” Kemp says. “There was a serious political challenge to the project…and I was called before the NASA administrator — of the whole agency — to explain it.”

This is disturbing to say the least.

Anecdotal counterpoint: I worked much of my graduate assistantship in C.S. at Shelby Hall, named for the senator cited in the article as leading the charge against OpenStack.

http://www.bama.ua.edu/~chem/facilities/buildings/shelby.htm...

How is that a counterpoint?

My graduate STEM education and paid job was partially funded by the guy. He's not entirely out to stifle innovation.

Interesting, but a big risk is that generic interfaces/software will limit functionality and lead to suboptimal performance. Amazon can roll out a new kind of hyper-advanced SSD-based database with some awkard custom interface any day of the week because they control the hardware, software, and API. You just cannot do that if your software runs in hundreds of different environments you cannot control.

Well written piece, as accurate as anything that has been published to date. The title is a bit sensationalist (though I suppose that is what gets readers) -- none of this has been kept secret, it just hasn't been told on a stage the size of Wired before.

I do wish you'd gotten props, Bret, for your work in creating the openstack case at Rackspace that we took to the board, and for your work in building the ecosystem! We couldn't have done it without you, no doubt about it.

As an OpenStack user with several PB in production, thank you!

We recently moved our app from AWS to Rackspace because of OpenStack. If we are successful, we know we want to run on our own hardware, but to get a sense of the market is too expensive using dedicated hardware. Rackspace and OpenStack gives us a way to test the waters using a cloud provider, then take ownership of the whole ball of wax later, without massive disruption to the software development effort.

You can do the same with Eucalyptus http://www.eucalyptus.com

Anyone have more info about these communal software houses? They sound pretty interesting.

Can anyone comment on the main difference between this and Cloud Foundry?

Reply on news.ycombinator.com