meskyanichi

I haven't checked out DotCloud's workers. I have a beta account for their service though. But have not yet taken the time to fully check them out.

Well, let me point out that there already is an add-on called SimpleWorker which provides external worker hosting.

Basically what it comes down it is that you host your workers there, and not on Heroku. Same thing. Why you ask? I have no clue! :)

Heroku Workers are pro-rated to the second, just like Dyno's. So the second you shut it down, the timer stops ticking.

The time it takes to boot a worker mostly depends on your ruby application. The heavier it is, the longer it takes to boot up. Heroku spawns them fast, I think instantly because they are spawned from a compiled slug which is generated each time you push your app to Heroku.

The HireFire checkup interval takes at most 60 seconds, and less if more resources are available on HireFire's servers.

Really? I don't know if you've ever set up a VPS or dedicated server, but you should compare the cost of that to what Heroku charges for the same resources. You'll see a significant difference.

Let me put it this way. 1 Heroku Worker (roughly 80-100mb of ram) $36 a month. Now, what can you get for $36 a month? Go look at 6sync.com linode.com webbynode.com prgmr.com. You can easily fit 10 workers in any of them. With this I come to the conclusion you're paying 10 times more than with a quality VPS provider. Sure, you don't have to monitor these processes, and it scales up and down easily, but 10 times more expensive is reasonable? Please. :)

Let me tell you what I find reasonable: If you're charging me 10 times more for the same thing, I find it reasonable I shut down my resources when I don't need them so I only pay for what I actually use. Why else would you offer the ability to set the worker quantity via their API?

Or how about this: You park your car, you pay for a parking ticket that lasts until the next month. Reasonable? I think not.

You should also consider the fact that this could also potentially encourage more people to actually host on Heroku, rather than be scared off by ridiculous pricing at early stages. I'm also not trying to "disrupt" their business. If this would greatly affect their business then I'll be damned, and I think I'm not the only one. So don't count on it.

If the demand is high enough I may incorporate such functionality in HireFire. There used to be another service but they have been down for I think a couple of months.

If you feel this is an important feature, please submit it to the forums. I'll definitely keep it in mind and see what the possibilities are for also handling Dyno's!

Definitely. The open source solution is fine in some cases. Though because it does post-create api calls it tends to be slow. Something I had not thought about prior to developing it, ironically. :)

Good question! I have actually considered doing so! I haven't taken an in-depth look at to how their add-on platform works but it's on my short-term to-do list.

Would be nice to get an even more "plug-and-play" feel. Though, thankfully setting it up manually isn't hard at all, but it is indeed a nice touch to create an add-on for it.

Good pointers! Definitely will take that with me to my future posts! Also, thanks for the compliments! Hope it helps!

I'm sorry if I offended you. It wasn't my intention. Looks like my writing skills could use a little work! :)

Thanks for the heads up. Seems like a good idea to catch the 503 errors. Also, for now I'll leave all the configuration defined in pure Ruby. Maybe eventually I'll consider using ENV var's to handle that.

Thanks for your feedback!

Hey Chad, great to hear! I also (despite having created HireFire) still have use-cases for SimpleWorker, however in my case they will likely be for my VPS' rather than on Heroku. On Heroku (even though there isn't a UI, unless you use Resque), it's already a partially managed service to host workers, it runs from the same compiled slug as the dyno's use so all the credentials / env variables are all shared among dyno's and workers, no need to re-apply configuration and such.

I think for me the hardest part was getting it to work the first time, waiting quite a while for the jobs to run to see if I managed to get it to work. The merge_gem is a huge improvement, though, handling credentials and re-connecting databases and other things are quite a hassle to set up (imo). I have to say that once it's all set up, and if the 30-60 start-up issue is gone, that it does seems like a beast for processing a lot of jobs concurrently and is exciting.

I'm definitely still looking forward to seeing future progress of SimpleWorker. So please do keep up the good work!

Hey. The gem actually has a local environment option which makes it work on a VPS similar to how it works on Heroku. You could try that and see how it works.

https://github.com/meskyanichi/hirefire/blob/develop/lib/hir...

Also, as far as I know that DJ Admin Interface is the only one for DJ that currently exists. If you want to use Resque, I'll be implementing Resque for HireFire soon.

Interesting and light-weight approach. Though, does it support other ORM's or ODM's like Mongoid? Currently HireFire supports ActiveRecord and Mongoid with Delayed Job. Planning to add Resque with Redis support fairly soon.

Thanks. I like your approach of not having to deploy to change the min/max value of workers. Though it's hard to implement (I think) if you're working with a worker/ratio configuration like HireFire. Currently HireFire also supports the Mongoid ODM. I'm looking in to implementing the Resque worker as well so people can use HireFire with Resque with Redis on Heroku as well.

Thanks for your input!

Thanks! Looking at Resque and will probably be implementing it fairly soon.

Ha! And yeah, the ENV variables. :)

Exactly. :) The DB connection is also quite frustrating depending on the location of the DB. MongoDB with MongoHQ or MongoMachine is fine since it's external already. But otherwise you have to know how to configure for example your MySQL PostgreSQL instances to allow incoming connections from external locations, and also becomes potentially insecure depending on your sysadmin skills.

I find Heroku + DJ + HireFire so much simpler, either using that handle_asynchronous method, or just called delay.my_method(*args) is brilliant. No need to think about isolated code or credentials due to the environment variables that are already set on Heroku.

I should look in to integrating Navvy and Resque as well.

Hey. I was actually also using SimpleWorker prior to HireFire, but I find that setting it up is a bit hard / takes long since you have to set isolated code / credentials / etc to their servers in order to process background jobs. Also, the jobs aren't guaranteed to run immediately after they are queued, may take up to 10 minutes on the cheapest plan. When deployed to Heroku it's easy since Heroku will just use the same compiled slug it uses for Dyno's, no additional code/configuration required, it just works. I also found that on SimpleWorker a job that took only 3 seconds on Heroku, would run for 30-90 seconds on SimpleWorker (no idea why).

That said. There are some advantages when using SimpleWorker such as the web interface, scheduling recurring jobs, and other useful options. Especially when you aren't running on Heroku, and memory is an issue if you have a small VPS. In that case, offloading background jobs to SimpleWorker is ideal. But I find that HireFire will make Heroku Workers cheaper than SimpleWorker due to the fact that jobs appear to take (a lot) longer to finish on SimpleWorker than on Heroku (probably due to the start up time? I'm not sure why).

Negative Nancy. Ever heard of the "Backlog too deep" error? That's a scaling issue, not the error you mentioned.

I could do a comparison. However, I prefer to just talk about what this does, rather than what other projects do or don't do. I have a lot of content already as it.

If you read the main page, you will (hopefully) in most cases come to understand what this tool does. You will remember it, then look at other projects that also handle deployment and see if they satisfy your needs. You make a list of the tools you might use and try them out. Now you're doing the comparison and "experiencing" what works best for you yourself, rather than listening to my (perhaps in 3 months) outdated information which might be inaccurate.

Now, for convenience I agree that I could make a list to summarize what this is all about but I'm not sure if there's any need for it. The 3 main aspects are covered: "Core", "Hooks", "Modules" and a video demonstrating something very basic.

Regardless, as I already said, any input is welcome and I'll certainly take it into consideration. Thanks!

I really wonder if people just look at the title and see "Git deployment" and then assume it's exactly like all the other git deployment projects out there and then comment here. Or, if they actually _read_ the contents of the website and understand what it's all about.

If you read the website you will see that Git Pusshuten can do more than just do a simple "git push" like other solutions.

Git Deploy by Mislav makes the assumptions you work with: Passenger, Ruby on Rails, etc. Git Pusshuten doesn't make such assumptions. It allows you to deploy an application, regardless of whether you're deploying a Ruby, Python, PHP, etc application, to whatever web server by defining _your_ environment through modules.

Additionally, it allows you to provision and pre-configure your environment so you can be up and running with minimal effort and sysadmin knowledge. Other solutions will make you manually set up your environment by SSH'ing into the server to configure everything. Now, I am definitely not saying that that is a bad thing. I'm just saying that Git Pusshuten's _goal_ is entirely different from that of Git Deploy.

So what you're pointing out now would only work for people using Phusion Passenger and a Rack application. That's a limitation that Git Pusshuten is trying to avoid.

Thanks for the detailed information! Actually it was referring to what you said earlier:

"The nobasu character (ー) ordinarily lengthens the vowel sound directly proceeding it"

But did not know that it "had" to be directly followed by katakana. :)

ー is just a dash.

This is correct. If you rely on a lot of security then maybe you should be considering a different deployment method, or write a module that does exactly what you want to do. It is modular so there is the probability you could accomplish what you want.

Also, the project just released a few days ago, mainly to say it's there, if anyone wants to contribute or provide feedback that would be very much appreciated. It is still in a _very_ early stage, hence the version 0.0.x so it could really go in a lot of directions, mainly based on input from the community and guys like you. There is a lot of room for improvement.

That said, any piece of information, advice and feedback is most certainly welcome!

It is not _just_ about pushing though. It is also about the modularity and additional server provisioning that it adds. You define your environment through modules, and Git Pusshuten helps you set it up, at least enough to get up and running really quick. Of course, it isn't for everyone. Just like a VPS isn't for everyone, and just like Heroku isn't for everyone.

This also introduces perhaps a new workflow to many people, if you already work with Git. As you said, it isn't very special pushing any branch to any remote location. However, it has not yet been done by anyone else yet (from what I can see, I might be wrong).

For a lot of people, (again, not everyone), it is a joy to be able to spin up a VPS, being able to provision it with pre-configured settings and common dependencies with a few commands, and being able to have your application live in a matter of a few minutes without needing almost _any_ knowledge of Linux.

Today we released support for node.js application deployment on our Rapp deployment engine.

This engine is installed on your Webby (a VPS server) and then you can publish multiple node.js (along with rails, rack, php, ...) applications using git.

We would like to get feedback from HN and we've recorded a 2 minute screencast that walks you through the process.

Let us know what you think.

Thanks,

Webbynode Dev Team