Pylons & repoze.bfg Merge | Hacker News Reader

Pylons & repoze.bfg Merge

groups.google.com

78 pointsericflo32 comments

As someone who's spent thousands of hours on our current Pylons codebase, I'm very interested in knowing how difficult it'll be to move my now-legacy Pylons 1.0 apps to the new Pyramid framework. They don't make mention of that. They do say that Pylons 1.0 apps will run within the Pyramid framework through a fallback handler to an existing Pyons app, so we're not hopelessly screwed, but that does sound a little kludgey.

A larger concern of mine is, given the enormous amounts of time and money we've spent on developing for Pylons, why would I port over to Pyramid? Why wouldn't I just port over to a more stable web framework, less likely to deprecate the technology on which our entire business is based? It definitely sounds like a neat framework, but I don't see as much value in neat technology when it shows a disturbing tendency to pull the rug out from under me.

To answer your first question, it won't be nearly as easy of a transition as it was from Pylons 0.9.7 to 1.0. For some reason, the fact that Pylons 1.0 hasn't had any updates or feature additions in the 6+ months hasn't seemed to be an issue, it was already mostly in a "it works as well as it can work" stage.

Plugging in the old Pylons app using WSGI isn't really any more kludgey than using WSGI for any other app composition use, which is plentiful.

For the second question, the best reason would be that in the world of web development, all web frameworks will deprecate old technology. It's the only way to move forward if a fundamental design decision in the core needs to be re-architected.

I know many former TurboGears users may have a lot of angst about this, because the rug has been pulled out a bit more than for Pylons users, who have rarely had to deal with everything at the bottom being replaced. This direction was taken to help avoid that again in the future, because now all the development for everything... top to bottom... we have under one roof so to speak.

In pyramid we have a core that is architected to deal with the extensibility issue, and can pick up development of features that the Pylons 1.0 code-base couldn't handle. Since its based on over 2 years of development and use in the repoze.bfg community, it means its very solid and stable already.

In short, I think it's on track to be one of the most stable Python web frameworks out there, just like Pylons 1.0 and prior has been quite stable. My advice would be to try new, small experiments and projects with it first. I'll be publishing more articles highlighting the improvements and extra features it brings to the table soon.

For a new startup, would you honestly recommend going with pyramid? And what about Django.. do you think it'll go faster to prototype and develop with pyramid?

Depending on what you're prototyping, it could very well be done faster with Django. Django provides a bunch of 'generic views' which can make tossing up lists of objects from the db very fast to get started with. Then usually you end up fully replacing it all later, but it gives you a good start.

My views haven't changed since I answered this: http://stackoverflow.com/questions/48681/pros-cons-of-django...

If your app doesn't clearly fit into the sweet-spot's for Django, the argument will be less compelling. The pyramid code-base does have a lot of maturity behind it, it's not completely brand-new, so I don't think it'd be a problem to develop against it to be able to utilize its additional extension points.

To help make the move from Pylons, I'd love to see a document introducing Pyramid from a Pylons perspective and explaining what it does differently.

Please say Pyramid has a debug mode like Pylons :)

easy_install weberror, tempita, pygments

add egg:WebError#evalerror into your pipeline. It isn't in the default Pyramid paster template, but, it was mentioned that it will probably be included in in the default Pyramid paster templates. http://goo.gl/EDqNo is a document I wrote regarding the transition of a Pylons 1.0 to Pyramid 1.0a1 application.

What bothers me is that nowhere in that email, or nowhere on the site, I see a single reason for this move. Why? Why do it?

The only explanation I see is that developers wanted a nicer codebase. I've been on Pylons mailing list forever, and I can't recall a single practical issue or a feature request, which was decided to be hard/impossible to do due to architecture limitations. It just never comes up.

This is a classic case of redesign for the sake of redesign.

Sorry you didn't see it, I'll try again: http://docs.pylonshq.com/faq/pylonsproject.html#why-not-just...

:-) I guess I'll have to copy&paste to support my argument. A link isn't good enough:

Due to the use of sub-classing, extensive, sometimes confusing, use of Stacked Object Proxy globals, and issues with configuration organization

As an active Pylons user I never, ever heard of these issues in Pylons circles. Translation: nobody asked for these improvements. Statements like "sometimes confusing" aren't doing it. When? Confusing for whom? I, for one, loved stacked globals. "from pylons import config" was great. Rails wasn't that simple. This is a big shock coming from nowhere. It just wasn't something that came up often on Pylons mailing list.

Let me make this clear: do I believe Pylons could be better? Yes I do. Do I think killing it to make it better was a good idea? No, I don't.

Anyway, I have tremendous respect for Ben and can't blame him for getting bored and wanting to work on something different.

EDIT: I just realized that you may be Ben. In that case thanks for building the greatest web framework for my favorite language! The fact that I'm passionate enough to bitch and moan about its death on the Internet should be the best testament to its greatness! :)

Many people loved "from pylons import config", until it didn't work, which frequently happened anytime someone wanted to use something from a Pylons app in a command line script (which seems to be pretty often). I do have a few minor clean-ups that help slightly with that stuff which I'll likely put out a Pylons 1.0.1 to address.

> Let me make this clear: do I believe Pylons could be better? Yes I do. Do I think killing it to make it better was a good idea? No, I don't.

So, the whole 'killing' thing seems slightly extreme. I've maintained Pylons with few changes besides bug fixes, security fixes, and deprecating old things for the past several years. And now the announcement that I don't plan on doing any more than the same thing for the pylons package code-base is "killing it off"?

If you're happy with pylons, there's no need to stop using it. The docs will always be available for it, the code-base will not be removed. By design, I just couldn't see how to continue it forward in ways that I (and a lot of people I've talked with) need.

Of course, a lot of people may not need anything more than pylons provides right now, and in that case I'd be hard pressed to tell them to switch their code to use pyramid. pylons is stable, mature, and has been maintained as such for quite awhile, there's a book that still pretty much applies to 1.0.

> Anyway, I have tremendous respect for Ben and can't blame him for getting bored and wanting to work on something different.

Thanks, sorry my nick doesn't reflect it, this is Ben. Seems I can't change my profile info on ycombinator, sigh. Or that 'feature' doesn't exist, anyways...

As I mentioned, the Pylons code-base was essentially frozen for any direction forward which I needed to build higher level tools. I know this may not be immediately obvious to everyone depending on your requirements, and pylons suits many peoples needs perfectly well.

I will be blogging about this in the future, at which point some of the key differences, and what they enable will hopefully become more clear. But Pylons is definitely not dead, and I think it won't be too long before people start to see what a larger development team can do. :)

This is the first post in the thread that makes me feel good about what's happened - perhaps this approach should have been the lead story, rather than what appeared to be "If you're using Pylons then say goodbye to all the cool developers - they've found something else new and shiny to work on".

( I've been watching this thread with keen interest - having chosen Pylons (vs Django, and RoR) about two months ago. )

This looks like good news to me: I've noticed that Pylons and Repoze shared a lot of common ground. Ben and Chris are seriously smart dudes, and I'm excited to see what they come up with when they put their heads together.

I am a bit surprised though by the way that this was done. As an outsider, it sounds to me like this decision was made among the lead developers of each framework privately, developed in private, and announced as a fait accompli. Was there really no step that involved consulting users of both frameworks to find out what they wanted?

I ask because we (Django developers) spend a lot of time obsessing over backwards compatibility, often to the point where we have to compromises our grand goals for more pragmatic ones. I'd be terrified of making a move like this, so I'm curious to hear if users feel they've been slighted, or if they're pretty much happy to go with the flow.

Before this, there were several efforts and results of those efforts to combine code in common and share more. First, we had an informal set of talks that resulted in a 'pipefitters' type mail list where people sounded ideas around on how to build our stacks on more common components.

Second, there was an effort on a code-branch called 'marco', that was to be a common set of code for a bunch of the underlying configuration setup. It's code was also out there, and ppl poked at it some: http://bitbucket.org/chrism/marco

It got a little ways, but it kept looking more and more like the existing repoze.bfg... which brought us to a point where pylons-trunk was built on BFG. This entire discussion is on the pylons-dev list here: http://groups.google.com/group/pylons-devel/browse_thread/th...

Then, in an effort to resolve remaining differences in the code, several of us flew out to meet in person to discuss remaining issues in the code and see if a better collaboration could work out. We decided to try out the idea of a merger, since pylons-trunk for Pylons 2 was doing little more than minor extensions of repoze.bfg anyways, which was collapsed into the pyramid code-base (and is a distinction in that respect from repoze.bfg which doesn't have those bits).

It worked out quite well from some attempts at 'trying it out' and the news of this leaked quite a bit earlier than we anticipated making it hard to try and put together what I think could've been a better process.

Was there a discussion where we asked everyone in both communities for their input on merging? Not really, we talked to primary contributors and developers who had worked on each stack to try and solicit input and ensure they were all in favor of it. But yes, I agree entirely that I should've solicited feedback from the pylons community at large.

For pylons the code-base, there really isn't going to be any difference from what there is right now, and has been. A maintained, stable, mature code-base, that has hit a point where there's little more to be done with it. It's been at a feature-standstill for about 2+ years now. It's not going away, and it will still be maintained.

So I think the way the merger was done definitely could've been announced differently, with more emphasis on what it meant for the pylons code-base (nothing really), vs. the continuing development efforts of myself and other pylons developers.

- Ben

Thanks, Ben. I knew there had to be previous discussion that I was missing. Again congrats - this looks like a good thing to me, at least.

This sounds more like "Ben left Pylons to fork BFG"

As mentioned below, there's no "leaving" involved. The pylons code-base has been maintained for several years with little in the way of new features, just bug fixes, some minor clean-ups, and security fixes. That isn't changing... so there's nothing 'different' for the pylons code-base involved.

I'm confused where this notion of leaving pylons comes from, when nothing is changing with it.

Last week I bought a top-of-the-range camera from Nikon. Excellent camera in every way : Great purchase.

This week, Nikon has discontinued the camera, and has announced that there's going to be a new range, even better, coming along soon.

Do I now love my camera as much as I used to?

Rationally, my opinion shouldn't have changed. But I sure wish that I had known that my investment in the previous product was a dead-end.

Not quite the same, because the new version has a bunch of features added. If you'd like to stick with the camera analogy, you should think of the pylons framework more as a good lens for the DSLR. Most of the features are from the body, so you upgrade that and keep using the same lens.

The Pylons framework was almost entirely just glue between your choices of ORM/templates along with using WebHelpers and Beaker. Those projects are still in use with pyramid, still getting updates (and thus a pylons project will still get feature updates via them).

I tried to explain this better here: http://groups.google.com/group/pylons-discuss/browse_thread/...

That's not merging - that's just killing Pylons.

Ughh... The lesson for the future: when it comes to tech, stick with the leader. I didn't go Django route because it felt clunky and bloated compared to more elegant and flexible Pylons, but I guess it was a huge mistake. Pylons decided to "collapse" itself in favor of even more (!) elegance (like someone needed that), and a year worth of work is now based on something "in maintenance mode", while Django is still around.

Tip: looking for a Python web framework? Django. Why? It will be there tomorrow.

I'm sorry you feel that way, as the developer of Pylons, I really don't look at this as killing Pylons anymore than the lack of any new functionality has killed Pylons 1.0 in the past 6 months.

Pylons 1.0 has been in maintenance mode already (effectively), 1.0 was in fact nothing more than a few tweaks on 0.9.7, ie. maintenance release with a 1.0 version number and finally killing a few deprecated things.

The FAQ explains why when attempting to extend Pylons for new functionality, I hit a dead-end: http://docs.pylonshq.com/faq/pylonsproject.html#why-not-just...

Finding a path forward for extensibility that didn't hit this forking problem: http://lincolnloop.com/blog/2008/apr/4/reusable-django-apps-...

was a very major concern of mine. I think this will address that, and when Django makes the changes necessary to remove the forking problem I won't be at all surprised if it requires porting your apps...

Why Pyramid is not called Pylons 2?

We had originally considered calling it p2, unfortunately that would've sort of bound the package name to sound kind of odd when someday we get to Pylons 3. Also, given the amount of new code to merge in, and then slowly deprecate the old code in the pylons package... it seemed more convenient to use a new name.

This also means that no one installing the 'pylons' package will have to worry about someday installing a version that deprecated things their old pylons app needed. Using pyramid means that you can use the new, along with the old one, at the same time, without worrying about version conflicts in the package.

This is not at all an apt comparison. With Django, you're buying into its templating, routing system, its ORM, its view/controller paradigm, its central app configuration, its core request delivery system, whatever it does for caching/sessions, and all the admin stuff. With Pylons, you've bought into some choice of templating (usually Mako), routing system (Routes), some choice of ORM (usually SQLAlchemy), its view/controller paradigm, its central app configuration, its core request delivery system (Paste, PasteScript, etc.), caching/sessions (Beaker), and it doesn't come with any admin stuff.

Pylons' change to Pyramid most certainly does not replace that whole stack. It replaces exactly two of those components, the view/controller paradigm and central app configuration. All the things that don't change at all are: templating, ORM, routing, Paste/Pastescript, caching/sessions still on Beaker.

So this nothing at all like moving to an entirely new framework, Mako/SQLAlchemy/Beaker aren't going anywhere.

edit: the routing changes a bit, but its an improved and higher performing package with mostly complete feature parity. I'm looking forward to using it.

Nit-pick: with Django, you're buying into an HTTP request/response interface and a system for mapping URLs to callables which obey that interface. Everything else is optional.

(and you know this)

yes, but I'm sure you understand the point, that the Django stack is "batteries included" whereas Pylons is not.

Glad I ran across this. I was about to dive into pylons this weekend for a small project/learning experience. Had waited all this time for them to stabilize on 1.0 and get some of their documentation updated. Now I'd have to ask why would I bother?

Pylons is still a great framework, and will be for the foreseeable future (a few years?).

Just because it matured and reached its end-of-enhancements life, doesn't mean it's not useful. It will take a while for Pyramid to mature to this level.

Yeah, I just got my company to go with Pylons 1.0. Wondering if I made a good choice.

If anyone happens to be in the bay area and is interested in a meetup to sit down and learn repoze.bfg and the differences between pylons let me know. I think it would helpful to get a bunch of us Pylons users together and talk about it.

I agree, I'm in the North Bay, and I'd be open to helping people work through pyramid, and discussing things in person. Would a meet-up next Thursday night (the 11th) work for some folks? Maybe around the South Bay or SF if someone can get some space?

wow - no new features in the last 2 years. web2py seems to get a new feature every week even though they promise backwards compatibility.

I think you missed the bit about Pylons being a 'glue' framework between other packages. Those packages have gotten quite a few new features over the years, and they continue to get new features regardless of whether Pylons users use them.

This is the kind of nice thing about using multiple open-source packages that are in heavy use throughout the Python world beyond just web frameworks (unlike web2py).

Reply on news.ycombinator.com