Heroku clone for Django? | Hacker News Reader

Heroku clone for Django?

djangy.com

88 pointsroom60661 comments

So, this is my site. Yep.

I forgot to turn debug off. Yep. And i'm using sqlite3 to accept email addresses. Yep.

I can assure you the quality of the ACTUAL project is far better :-)

I have a working prototype and am working my ass off to get something scalable up and running asap. keep checking back for details!

Regardless of the criticisms, if you need another beta tester, you'll find my email in your sqlite database. ;-)

Seriously though, best of luck, and godspeed.

How seriously can you take a service if developers forgets to disable the DEBUG flag :)

http://djangy.com/admin

must to be fixed quickly, that's a serious error.

Yeah thats the problem with deploying "by hand". Maybe this python guys should use Capistrano for this things :)

Fabric is nice too and it's in python (http://docs.fabfile.org/0.9.1/).

I regularly use Fabric. After deploying a couple of projects with it, I have come up with a standard deployment layout and stack. Now its as simple as "fab deploy" and "fab revert". Thank you fab!

I'm looking to optimize my fab setup, feel like sharing your fabfile.py?

Same here, except I've set things up so that any changes I make to my project are automatically synced instantly to a live staging domain, and I don't have a revert command (although I do back up copies of previous deployments). I also keep apt and pypi dependencies in project-local config files and have a refresh command that updates those packages on both the remote and local machine along with some other stuff. I think I might have overengineered things a little.

would any of you share these files and your setup? I'd love to have a look into them.

Thanks!

Here you are:

http://dpaste.com/231226/

thanks a lot!

Sure. I doubt you'll get much out of it, but I'm still quite curious what your impression will be.

http://pastebin.com/Q970rYaH

If it helps, "li" is my current project's code name.

The vast majority of the code is concerned with configuring a fresh Linode, not pushing recent changes (for that, see the rf and stage functions, and the functions they call). I hate system administration. The only way I could make myself configure my server was by writing a script to configure it for me.

thanks!

My fabric setup is basically:

* push to <server>

* update the files

* compile sass to css

* minify javascript

* process the dependencies file (using pip) for updates (it does nothing if I haven't added new dependencies)

* restart/flush/whatever I specify

It works beautifully, I can deploy changes in a few seconds.

EDIT: Okay how do I make lists :(

I was literally just looking into this yesterday because I've been lazy and haven't set up a good deployment process. I found this article http://lethain.com/entry/2008/nov/04/deploying-django-with-f...

I'm wondering what the best practices are and since I have so many people around me who use Capistrano I'm thinking about just going with that. Any words of advice?

http://djangy.com/signup

Oh dear.

Use of SQLLite too - does that scale and perform well against something like postgres?

I think what we're seeing here is an email/hype-grabbing attempt with an interesting vision, using the Heroku name for leverage.

Would like to see it materialise, but they really should have hardened something as simple as a signup form. I've emailed the admin telling him to lock it down.

To be fair, you wouldn't need much more than SQLite to gather emails. It's more than sufficient (I would say it's absolutely amazing for most use cases, but the one thing I am not sure about is how well it handles concurrency, if at all).

I tried SQLite for a few things, and it has always come out a champ. I would not hesitate to use it for most of the semi-static websites I make (web apps would be pushing it), if I were sure it can handle more than one connection (I'm not, sadly).

I am very, very impressed by it. Seriously solid database.

SQLite is a file and thus uses file-level locking and does not have row-level locking. Data can not be read the same time anything is being written, so it is usually not great for multi-user applications or web applications.

Development, or 1-3 user web app it should perform OK. It doesn't scale and you will get db locks and retries if its under too much load.

To speed this up (if you must use SQLite), use a solid state drive as I/O performance is the biggest bottleneck.

Thanks for clarifying what I suspected. If you don't need high concurrency, it's a champ.

http://linux.die.net/man/2/fcntl

actually it could use row-level locking since it uses row-oriented storage and fcntl locks can be applied to a set of bytes within a file. it could also do file-level locking on tempfiles with names derived from the table name and primary key of the row.

anyone with actual knowledge of it care to chime in with what it does do?

I have actual knowledge of it. The product I work on uses it as the production database because we distribute the software as a desktop application (and MySQL has licensing issues).

It COULD do row level locking, but it doesn't. SQLitening is the closest thing to performant SQLite you will get. And it only runs on Windows.

Never write form confirmations ("Thanks for your email") in red color. Looks like a warning. If a form is processed correctly, it should be black or green.

I hope that usability of djangy service will be better. Can't wait to try it :)

Btw, isn't Heroku itself planning anything for Django?

> Btw, isn't Heroku itself planning anything for Django?

Has that ever been mentioned or rumored before? Is that even feasible given Heroku's technology stack?

Pure speculation, but they've taken funding and seem eager to branch out. I think we'll see more than ruby hosting from them in the near future.

I was throwing around startup ideas and I emailed them in late june 2010 to ask if they were considering it.

The response was quite clear that they had no plans to enter the Django market.

That's what I'd expect. They're only still in the early market penetration stage of the Ruby market; it doesn't make a whole lot of sense to complicate their product offering. (Particularly when App Engine is available for Python/Java.)

Will it work for Flask? Any plans?

http://flask.pocoo.org/

Hopefully it will work with any WSGI application because even Django applications might use WSGI components and thus have a custom WSGI application object.

Finally it arrived :) I love Heroku and was looking for something similar for Django for ages now. Can't wait to give it a try!

Is it just Django, or will other Python frameworks work?

They really need to support more than Django. The Python world isn't centered around one web framework, the way Ruby is.

Actually, Heroku supports any rack application - that means most of the ruby web frameworks.

Seems like it wouldn't be hard to support any WSGI application...

How do they plan to compete against Google App Engine?

http://www.allbuttonspressed.com/projects/djangoappengine

Maybe by not requiring any modifications of Django to get the basics running? Allowing standard databases? Or even the popular thing lately - by not being Google.

Or maybe they'll enable more stacks? Pylons shouldn't be that hard to plug in once you have Django deployments working. And since they reached 1.0 lately...

App Engine is a pain in the ass, and I wouldn't recommend it to anyone. Heroku is so simple to use that I've many times felt that someone needs to build a Heroku for Python.

The only thing really wrong with App Engine is the datastore:

- It's very different than anything you've come across before - The docs are not sufficient for it's complexity. You end up having to read lots of blogs and watch lots of videos to figure it out. - It is ridiculously difficult to learn (one of the hardest things I ever had to do) - Important parts of your stack won't work, like Admin - You can't just add the fields you like to an object. Instead you need to meticulously plan _how_ your data is going to be accessed. This is exactly the wrong thing for a startup, since you can't get up and running fast. - You can't iterate fast, because any schema change is very risky and very slow. - If you don't get everything exactly right, your app will be very slow. - There isn't much in the ecosystem.

By contrast, none of these are true for Heroku.

There are some things that are pretty good about App Engine:

- the tools are pretty good. Not as good as Heroku, but good. - the billing is spot on. They give you about $1 per day of free stuff, which can maintain a site with hundreds of thousands of users. And it's pretty cheap from there on. - They scale it automatically. On Heroku, it's really hard to tell how many "dynos" (whatever the hell that is) you need, except by trial and error. I would expect that nearly all Heroku apps are overprovisioned. App Engine just nails the scaling - it happens automatically, and you specify a maximum amount to pay per day, with lots of different dials to control it more precicely if you like.

I hope Djangy takes the best of both worlds.

While I do agree that the App Engine datastore does require for some planning above what you're used to from a traditional RDBMS, I think it really just reflects a learning curve that comes with any new technology: as you use it more often, you get better at it. I think calling App Engine a pain in the ass and that you wouldn't recommend it to anyone is a bit over the top.

No, I truly meant it. Everything related to the datastore is difficult and tedious. Goodbye to incremental development, goodbye to knocking out a feature in an hour. Every single change you make has to be planned.

Adding a field to an "object"? Where do you use that object? Is there somewhere that you don't use that field which becomes too slow?

How are you going to migrate that new field? There aren't tools to help you migrate. Generally, you have to change to an Explando, add the field, load every single object from the datastore, then save every object back to the datastore.

These are two examples off the top of my head. Every change you want to make in App Engine has similar questions raised. I really did mean it when I wouldn't recommend it to anyone.

I think it's a much different learning curve than with any other technology. I'm plenty adept at learning new things, but this was just unnecessary complex, with nothing to redeem it.

I'd have to agree that Django on appengine is a big pita for the reasons you mentioned. Django is a large framework, and was simply not designed with interchangeable backends in mind. Getting it to work either requires a lot of black magic monkey-patching or foregoing the majority of built in features.

On the other hand, I have found the datastore api (and most of the other apis) to be well designed. It works very well with lighter frameworks such as tornado.

It sounds like your problem with App Engine is that you're Django. Yeah, Django is not built for the App Engine data store, so why not use something that is? I've used the Tipfy and Flask frameworks (both are a thin layer on top of Werkzeug), and they experience has been great.

I haven't tried djangoappengine, but it seems possible it would abstract away most of the datastore-related stuff and make the admin work.

Not true, unfortunately. For example, creating objects and deleting them uses a different method (put instead of save). There are tiny syntax differences, also.

Not to mention the fact that designing an information schema for a SQL database is quite different from designing an info schema for a hash-table like datastore (which appengine has).

I would love to hear more about this. I'm currently in the middle of a Django app, and this would be a godsend. I like doing sysadmin stuff, but there's no way I will have time in the next year.

If you guys can pull this off, I will be a very happy customer.

Would be nice if there were something like this (like Heroku) for Java. I know that there are several "cloud" solutions for Java, but I haven't found something as simple to manage/extend/deploy like Heroku so far :(.

yeah there are, http://www.stax.net/ for Java EE developers.

another, https://www.playapps.net/ for play apps., those written using the playframework.

> http://www.stax.net/

I used Stax during beta, and it's nice, but it's no near the Heroku ease of use (puls it's little helper Nezumi http://nezumiapp.com/ when on the way). With very small restrictions, one can install anything on Stax, so it's very flexible, but just not that easy to use as Heroku'.

> another, https://www.playapps.net/ for play apps.

Unfortunately Play framework is not based on a servlet container (something that 99% of java webapps are using).

I've been working on my Heroku for Django (or Python more generally) for a month or two now. It's good to see somebody else thinks this is a good idea.

How does it work? How do you handle persistence and parallelism?

The front end dispatches requests to a Varnish, which is in front of a router that queues up requests and sends them to available copies of the desired application. Persistence is Postgresql, and delayed and scheduled jobs are handled by celery over RabbitMQ.

My number one goal is to provide a deployment environment with as few additional restrictions on the developer as possible.

That sounds quite good! So most of the parallelism is on the view processing layer? Is the postgres setup parallelised as well?

I like the idea. But make sure to differ from what AppEngine offers or you have a very strong opponent.

I really would love to see this for clojure or smalltalk.

Sounds like a contestant in the AWS Startup Challenge.

Intersting!

can't wait for trying it!

I wonder why Dave Paola (apparently admin at djangy according to DEBUG view at http://djangy.com/postreceive ) used room606 instead of his regular account endlessvoid94 for submitting this.

Because he doesn't want to be outed, for whatever reason. Outing someone by calling out their real name is a dick move. It used to be that HN recognized the occasional need for stealth pre-launch, instead of this braggery.

(Disclaimer: same thing happened to my company when we pre-launched.)

I have no idea who Dave Paola is. I saw it posted over on coder.io and I thought it might be of interest

I didn't submit it :-)

Reply on news.ycombinator.com