Announcing PoolParty, An Open Source tool for managing EC2 clusters | Hacker News Reader

Announcing PoolParty, An Open Source tool for managing EC2 clusters

blog.citrusbyte.com

15 pointsprogprog11 comments

PoolParty is an open source tool that automates deployment, monitoring, and load balancing of EC2 instances.

It's worth keeping an eye on this as well:

"Vertebra: A Next Generation Cloud Computing/Automation Framework"

http://brainspl.at/articles/2008/06/02/introducing-vertebra

There hasn't yet been a "first generation cloud computing automation framework", so it's kind of impossible to be building the "next generation". That first generation is pretty much what everybody in the field is working on (including my company).

But Vertebra sounds like an interesting approach for the applications where it fits. And repurposing an existing protocol for RPC is always interesting, they've chosen the surprising, and novel, XMPP (we use a light-weight RPC mechanism that runs over HTTPS that was already part of Webmin, with a fallback to SSH on instances that don't have Webmin or Virtualmin). And they're addressing a niche that no one else is addressing, which is Ruby on Rails in a cloud environment...it could be done with Amazon using SQS to detect rising "pressure" and ramping up new instances as the pressure rises and shutting them down as the pressure decreases. But, Vertebra being focused on RoR probably means that if you're building a RoR app, you'll need to spend a couple of days less time developing that aspect of your deployment. You still have to solve the database scaling issue, and the data sharing and state problems, just like any other large scale deployment...but one less thing to worry about is one less thing to worry about. (And nobody is going to solve all of those other problems automatically.)

Actually Vertebra is not tied to Ruby on Rails at all. Since the main communication is all XMPP based, any language that can speak XMPP that implements our protocol can become part of the XMPP cloud. Our first implementation of the agent framework is written in Ruby but I expect many more languages to be supported.We will be releasing at least python and erlang based agents along with our ruby agents. We will be releasing the whole thing open source once we have locked down the protocol and made it a XEP standard, so keep an eye on my blog if you are interested.

Also Vertebra is not just for server automation, it is a framework for distributed computation of many kinds as well as a real time application platform.

The discovery mechanism and Resource based dispatch will allow for applications to get a request, discover via a Vertebra call which backend is least loaded and offers the request processing you need and then dispatching said request to the proper backend. Also allowing for map/reduce and scatter/gather style programming.

I have many ideas of where to take this thing and I cannot wait to get it out in the open so I can see what other folks do with it.

"Actually Vertebra is not tied to Ruby on Rails at all."

Interesting. I just assumed because of the connection to Engine Yard, and you know what happens when one assumes.

"We will be releasing the whole thing open source once we have locked down the protocol and made it a XEP standard, so keep an eye on my blog if you are interested."

Will do. While we have our own implementation for solving quite a few of the common cloud infrastructure problems, we're also agnostic and configure dozens of other tools without feeling the least bit guilty about having multiple ways to do things (must be our Perlmonger background), and it sounds like you're addressing things that aren't in our current scope. (For example, we support Xen, VServers, Zones, and EC2, and S3 and ZFS...we're loco like that. We don't even care.)

This is probably the biggest disappointment EC2 brought: this kind of stuff should be built into the service :-/

So, thumbs down for Amazon for not providing this basic piece of functionality, and kudos for the developers who are behind this.

You are exactly incorrect. Amazon is providing the raw materials on which an innumerable combination of strategies and implementations can seek to fulfill these ends. the fact that they are not pigeonholing you into just one is great. it is taking more time to develop open source workable solutions, but will result in a much richer ecosystem overall.

Is this Rails specific?

No, it adheres to the principles of AWS very well. So you get: - distro-agnostic - language-agnostic - framework-agnostic

ah hell yeah! bout time someone made EC2 autoscale. thanks!

at least one company already has (rightscale.)

Cool!

Reply on news.ycombinator.com