Open Source Is Really About Documentation – Twisted vs. Tornado
littlehart.net
littlehart.net
...and I agree completely with the op.
Twisted is powerful, flexible, well architected, and reliable. It is also damn near impossible to get over the steep learning curve and to actually USE it.
Show me a competent Twisted developer, and I'll show you someone who had a gun jammed roughly in their mouth, and was FORCED to learn it at some point. We are all glad we know it _now_, but that learning process SUCKED.
I go to a lot of pains so that clients of my framework never have to touch or even see twisted though it is under the hood. I think I spent about a week in the beginning really wrestling with Twisted and what was the appropriate way to transcribe my ideas into the "deferred" model. Once it clicked it was relatively smooth sailing (all in all, Twisted is a pretty sweet framework but not without its gotchas that you have to learn the hard way). A week spent learning a new framework isn't much time, but when clients want to get their app up and running ASAP it is an eternity and ultimately would prevent their adoption of my framework.
Tornado does look nicer and I have to admit purely from the documentation. I read through it for about 10 minutes just now and I can for the first time actually see how it would work within the realms of our project.
With that said it might be a useful way to figure out what Twisted is all about too....
What's wrong with that? Seems quite comprehensible to me...
The best investment in terms of mindshare the twisted project could make at this point would be to simply sit down and write documentation.
Grab sphinx (or another tool that makes doc-writing fun) and start out by writing a page about each Class that is being used in these examples. Why does it exist, what role does it have and don't forget the link to the API doc. Then I'd suggest to make a high-level overview, perhaps even with a little diagram, that explains how everything plays together in the various application types. Hyperlinks to the previously mentioned detail-pages should then work wonders for anyone exploring the project.
Writing good documentation is hard, but you only have to do it once. The lack of documentation on the other hand forces every new user to wrap their heads around innocent lines like this:
internet.TCPServer(79, factory).setServiceParent(
service.IServiceCollection(application))
Making documentations (and contributions to it) a first-class contribution with its bugs & discussions & refactorings, and fun to write (the way the Django project handles it) would probably also go quite a way towards that.
Currently the barrier of entry is too high and I guess too few people are deep enough in twisted to explain all the why's of even a simple baseline example properly.
It's kind of a deadlock situation because without that lowlevel insight it's also hard to get confident enough to contribute high-level docs.
Thing is, through checking out Django I realized you don't really need to be "deep enough" to contribute to that stuff. The thousand-cut effects applies, and when you're trying to make heads or tails of the doc, you're bleeding from these thousand cuts so you know and understand the doc's pain points. If doc contribs are first-class, then right as you understand the issues and what the doc tries to say, you'll be steered towards opening a documentation bug (and maybe even contributing a patch) for that part.
Maybe it's just doing to be a bit of wording, maybe it'll be a link or two to other relevant parts of the doc; maybe it'll be things you'll have figured through your sweat and tears, maybe something you'll have learned by asking a question on IRC.
But even as a beginner, you'll have added a little stone and made the doc slightly better through your current understanding. And it's enough.
I barely contributed to Django myself, but my first ever contrib was simply pointing out that the `now` tag's format character `n` produced an RFC 2822 date, not an RFC 822 one as written in the doc at the time. It was almost nothing, but it still very slightly improved the documentation for the next guy reading it, and it actually made me feel involved in the project. Even though I didn't have anywhere near the knowledge of the codebase to contribute to the code itself.
edit: and re the thousand cuts effect, that's the great thing about making beginners contribute: "old timers" don't notice or remember these small issues (which pile up), because they fill in he blanks or missing parts through their knowledge.
Attention to small details is the greater part of friendliness.
Deleted comment
I did just that with the twisted community and I found them a good bunch.
If I needed what twisted had to offer, I'd give it a try. But if I'm just screwing around (an exploratory phase that sometimes leads to much deeper commitment) then I'm going to go with whatever is easiest that seems to have sufficient depth.
Anyway, I have plenty of self-respect, thank you. I am sure there are others like me.
edit:textarea != vim
Besides, Tornado can afford to have (seemingly) more comprehensive documentation because it is far smaller in scope and support. Twisted has a large number of robust libraries for doing all sorts of things asynchronously, from web servers to ORMs to SSH and everything in between. It's like comparing an orange to a bushel of apples and it's silly. Twisted has never supplied a web framework, which is exactly what Tornado supplies.
The Twisted docs are lacking in many ways, I just don't understand how Tornado is objectively better because it doesn't currently have that problem -- they aren't the same thing!
"The Twisted docs are lacking in many ways, I just don't understand how Tornado is objectively better because it doesn't currently have that problem -- they aren't the same thing!"
Of course it's the same thing. If your documentation is lacking, don't complain when people point it out. It's awesome that Jean-Paul (who commented on the blog post) is available to help people...but if there was more documentation then the simple questions disappear and you are left answering the really tough ones. Easier said then done, obiously.
Upon reflection, this is really a comparison of Twisted.web with Tornado. The rest of Twisted got dragged into it, perhaps unfairly.
There seems to be a segment of the programming society that's just content with spewing whinage about not being able to get what they want, instead of using what they find and improving it to fit their needs.
I am no Python programmer, but i have looked at twisted and found it trivial to grok. I saw how they have wrapped the Unix system calls and put a layer of abstraction over it .. I didn't like it, and went back to using Common Lisp's iolib. I must have missed an opportunity to blog there :-(
I consider this analysis and feedback, not whining.
You seem to suggest that the author is whining in lieu of being self-sufficient. I can't speak for the author, so I will just speak for myself. I wholeheartedly believe in the self-sufficiency that programming culture promotes. However, I also believe that the easier you make it for others to achieve self-sufficiency in your open-source library, the more successful it will be.
I'm happy that you found it trivial to drink deeply of twisted's essence. I also think that you are right that you have an opportunity to blog! I for one would love to read about why you think that CL's iolib is a better abstraction layer than twisted. You haven't missed the opportunity -- I'd upvote that over most of the stuff on the front page!
Tornado is a web framework, in many ways closer to Django than Twisted. Beyond that the underlying approaches are different - Tornado does not use deferreds.
There are a number of great projects that use Twisted. Soon many new projects will use Tornado. But the frameworks' respective documentation does not tell the story.
Not really. It's the combination of an asynchronous HTTP server and an optional web framework. And work is being done [to add WSGI support](http://github.com/facebook/tornado/commit/8ca616088cfb26ff19...) so other frameworks can be plugged into the server.
At its core, Tornado is a server not a framework.
Twisted is, at least for now, more general purpose. Can't wait to see where Tornado goes though.
You could argue that if Twisted was better documented the FriedFeed guys wouldn't have started their own project. I don't buy it. There could have been many (technical and non-technical) reasons to do this anyway.
I (mostly) agree with a comment on the blog: This is just pure NIH syndrome. It happened because it’s easier to write code than to read it, and it’s more fun to create than to fix.
The Twisted devs may have some hurt feelings, but I think they can learn a lot from this.
I agree that documentation is really important in Open Source, and so is finding the documentation. Googling for such a generic names makes things hard. Considering all the hype Tornado got recently the homepage is not still listed in the first 5 pages of Google's results for "Tornado".
So I think it's a good advice to think twice (or thrice) before naming your next project.
A lot of libs around here are, technically speaking, well coded, reliable, fast. But the API sucks. An example is the libpng, if you have some experience with it you'll understand. When an alternative with a clean API arise it's very hard to resist.
It is so strange to create reusable components without detailed guides.
The nginx had this very problem. If it could get a user-friendly documentation it will earn more adoptions.