Comment by jacquesm | Hacker News Reader

Comment by jacquesm | Hacker News Reader

Parent Full thread

This sort of thing is one of the reasons why I'm very careful in picking the pieces that make up the tools that I earn my living with.

Left right and center you're bombarded with 'use this', 'use that', but most of these projects end up not having much staying power and if you depend on them for the running of your business then sooner or later you might end up having to support them.

When I choose an open source component these are the things I like to see in the eco system around it before deciding to jump in (obviously not every project selected will have all of these, but more helps):

- multiple maintainers, active response to patches being sent in

- broad support, using open standards

- preferably a drop in replacement available

- lively community of people willing to help each other out

- two years track record with a good roadmap for future support

- people working on it because they like it, not because they get paid to (this one is probably very counter intuitive when you're looking at this problem from a business point of view, but it is my opinion that people working on something because they like it are automatically in for the long haul).

Every time I've been seduced by new, hot and sexy technology I've come to regret it sooner or later. Old, boring and solid seems to win the race every time.

Running a small shop means that the investment you make in a tool is one of the most important decisions you can make, the wrong decision could cost you a lot of time and/or cripple your company.

I hope that everybody using unladen swallow is able to easily transition to a replacement.

There is no need to transition from unladen swallow since almost nobody was using it. It was a fork of cpython, and the speed improvements were so negligeable that I doubt anyone sane would try to use this without mastering the - heavy - technology behind.

I was actually quite shocked when unladen swallow was somehow almost accepted for python 3.2 at some point, given how bad the performance results were: if you add C++ + llvm and all the new infrastructure/complexity it brings, you rather have good numbers, not something like 10 % speed improvements which was the order of magnitude last time I checked.

It did bring useful code to python, though: in terms of benchmark code, but also proving that speeding up python while keeping cpython compatibility (vs just python compat) is actually pretty hard (which was expected, but they confirm the suspicion).

> I was actually quite shocked when unladen swallow was somehow almost accepted for python 3.2 at some point, given how bad the performance results were

Memory is getting cheaper. I was looking forward the GIL removal (at first), but I'll be happy with any multi-threading improvement.

OTOH, when I really needed it, the multiprocessing module did the job nicely. And it also prevents some very hard to catch bugs.

Unladen has never intended to remove the GIL - their primary goal was to keep compatibility with C extensions without even recompiling IIRC (google seems to rely a lot on python extension around their infrastructure), and removing the GIL will break many, many extensions: the only known way to remove the GIL without cause a huge slow down is to remove reference counting, which C extensions rely on, and also most C extensions rely on the fact they don't need to be reentrant.

Removing the GIL was one of the explicit goals as originally conceived, but was dropped when it was realized how difficult it would be (source: http://code.google.com/p/unladen-swallow/wiki/ProjectPlan#Gl...).

I run most of my Python program on fairly low end boxes, so consuming 9MB per process vs 90 would be a pretty big deal. If you have RAM to burn, share :).

The rule is that next year, you'll be able to run twice as many processes as you do today for the same amount of server money.

And, besides that, Unladen Swallow would most probably take 18 or 27 MB per process on your scenario. 10x more memory seems weird.

In my use case, large data processing, I would prefer slower processing in exchange for lower memory requirements. The more ram it eats the sooner my processes have to hit disk.

Yes, I suppose you are right about the memory usage: http://www.python.org/dev/peps/pep-3146/#memory-usage. And of course Moore's law has been on target so far. However, next year I'll want to run more kinds of different processes, putting more projects on my servers and I'd rather conserve RAM as much as possible.

The PEP was/is accepted. The problem is all the resources with the knowledge to merge it have since been reassigned. It's still pending merge into py3k.

That's not entirely accurate, the PEP is about merging it in a new branch called py3k-jit. AFAIK there are no plans of merging it in the main, vanilla, py3k.

Look at the header - 3.3 the intention was to accept the PEP, do the merge, and then ask for final approval for merge into Py3k. The intention on both sides was to always merge it into vanilla Py3k.

Yes you are right, what I meant is that it will still need a "final approval", which I think will imply a more thorough discussion and a round of voting.

The "merge plan" section states that, before that, all the jit-related patches will be confined in the py3k-jit branch

And you're right; final approval won't come because well, it ain't done.

Agreed. I'm not waiting for another Java, either, When carelessly integrating a JIT there would be a big risk of long startup times and other memory-heavy behaviour.

Of course, there are also examples of JIT that seem more lightweight, such as the javascript JITs.

pypy has quicker start up than CPython and also has a JIT. They're working on interesting memory optimisations too.

No one was using Unladen Swallow jacquesm - it was an experimental fork of CPython.

My bad, I thought it was in production use as well. But in a way that does not really detract from the main point, just from this particular application of that point.

Isn't unladen swallow pretty much a JIT drop-in replacement for Python itself? If that is the case, it ticks almost all of the boxes such as."using open standards", "drop in placement available".

     I hope that everybody using unladen swallow is able to 
     easily transition to a replacement.

Like CPython? :-)

Pypy

That's hardly a drop-in replacement. CPython is.

Actually, PyPy is already a drop-in replacement for many uses, and is getting closer to being a complete drop-in replacement all the time. For example, they have a source-compatible implementation of the CPython C API: http://morepypy.blogspot.com/2010/04/using-cpython-extension...

_sooner or later you might end up having to support them_

It seems ironic that this is often cited as a reason not to use an open source project. Isn't the selling point supposed to be that you can support it if you really need to?

That said, the pragmatic part of me would rather work on my problems instead of my tools.

Reply on news.ycombinator.com