Hacker News Reader

Comment by ikken | Hacker News Reader

We see people hyping all the time for NodeJS mainly because it can handle millions of concurrently open connections. Now we get a much cleaner (IMHO) way to achieve the same thing in python (of course python is slower) - i.e. see aiohttp as websockets server. I would not call it a disaster.

View on HN

It's great that it works for you for this one particular type of application, but the whole world isn't io bound. Some of us just want regular, plain old, concurrency. As this shows, multiprocessing is still the only, unsane, solution right now. I just want to be able to make a thread, and do stuff in it in parallel to other threads, like I can with most other languages.

"Go write a C extension" you tell me, "use something besides cPython" he says, "just use multiprocessing" I hear. Sure...but ffs, we've had multicore processors for almost TWO DECADES now.

One of my biggest, and apparently unchanging, problems with Python is the desire to keep things simple in the interpreter, to the disadvantage of the language. Sure "implementation for interpreters may vary" blah blah, but you have to target the bottom end in performance, and most widely installed, which is definitely cPython for both points.

I think these are two distinct use cases. NodeJS actually does not (AFAIK) handle one of them. Basically, you have IO bound tasks and CPU bound tasks (and a mix of both which is really nasty business). Python has had CPU-bound task concurrency via multiprocessing and it's been OK. My preference would be to get rid of GIL and improve how threading is actually done, but technically you can serve CPU-bound tasks today with Python 2 and 3. This is (AFAIK) not something that Node does out of the box.

The IO bound tasks in Python are a problem and I wish there was a clean solution. Python does not have a global event loop, so there is not an easy place to hook in coroutines, callbacks, etc. So for a while we were stuck with one of the following:

1. Use threading or multiprocessing. This sucks for more than concurrency of like 2-8.

2. Use eventlet, gevent, or another event loop. The problem here is that you have to buy into it whole hog. No component of yours can be blocking, and that's hard to tell.

3. Write your own event loop. I've done this and find it to be the most understandable and easy to debug approach. This sucks because of the amount of effort it takes for something so fundamental (because networking is tricky).

Some people would be happy if Python got better at solving IO-only bound tasks. I guess that's where this feature comes in. I haven't played with 3.5 yet because I am mostly stuck on 2.7 for reasons. However, looking at it, I feel like there should have been more of a separation between blocking and non-blocking code here. Something alongs the lines of an async function not being able to call a blocking function.

Re: CPU and IO bound tasks: I know of no great framework for this besides threading (not the kind in Python + GIL, but real threading). I usually just side-step this problem by separating tasks that are both IO and CPU bound into smaller tasks that are only CPU or only IO bound. Thankfully, that's generally pretty easy to do.

asyncio provides an event loop in the standard lib now

Try this: http://dask.pydata.org/

And ironically, nodejs is also single threaded with it's own GIL, which somehow no one ever mentions. It's great for async IO bound stuff but is really worse that Python for mutlicore CPU bound work - at least Python has concurrent.futures which makes throwing together a parallel map operation a few lines of code.

This is like the cripple making fun of the blind man.

When your competition is NodeJS there is nothing really left to say.

Reply on news.ycombinator.com