2. You don't want to fork on every request, this is very vulnerable to fork bombs.
3. In worker threads model, you already have the work threads spawned and ready for crunching, which will reduce the system load because you aren't forking on every request.
(I'm not saying they aren't or that I know the answer.)
Then again, as someone else said, the amount of time it takes to spawn a process to encode a video relative to the amount of time it takes to do the encode is probably trivial, and you would benefit from having the process isolation in case something goes wonky.
You would probably write some sort of "process gate". Though in a distributed architecture I'd do this with some sort of distributed work queue.... I did this in .NET many years ago for a similar service: http://blog.jdconley.com/2007/09/asyncify-your-code.html
There are certainly advantages and disadvantages to both threads and processes, but it's not really a fair comparison to claim that processes are as fast or faster than threads because you can spawn them at a certain rate. The performance cost of separate processes is something you pay gradually, every time you have to take a page fault and copy 4KB.
When Python forks a process using the multiprocess module, that process can execute concurrently with the parent process. On a mutlicore machine, it can be simultaneous.
When Python spawns a thread, the thread and the parent process cannot execute concurrently. They need to grab the Global Interpreter Lock (GIL). Whoever has it can execute. Whoever does not must wait.
So, I suspect that what we are seeing is that even though the new processes/threads have very little work, the processes can exit faster because they don't have to wait for the parent process to give up the GIL. This is a misguided experiment.
I reran this test with Python 2.7 and it no longer appears to be true:
Spawning 100 children with Thread took 0.03s
Spawning 100 children with Process took 0.28s
Regardless, I don't think it's a misguided experiment – it was an objective observation. It shows that things aren't so black and white depending on your toolchain.
Two, threads don't buy you parallelism in Python, unless the majority of the work is being done in C modules.
Finally, this test is really just testing the multiprocess and thread packages provided by Python. I say this is misguided because the way the author talks about it, I don't think he understands that the difference between those abstractions and OS threads and processes. (Which, of course, are an abstraction as well.) I suspect the Python overhead will be more than the difference in cost between forking OS-level threads and processes.