Hacker News Reader

Comment by ricardobeat | Hacker News Reader

At that point what do you gain versus porting all the C stuff to the host language so it can be JIT compiled?

I'm not sure what you mean - you gain not having to port anything, which sounds pretty great to me.

And for the Ruby C extensions we tried not only did we not have to port them from C to Ruby, they actually ran faster than either the compiled C version or the pure Ruby version because we can inline and optimise between the two languages.

I mean the cost of porting vs writing a whole new C interpreter - sounds like a lot of work.

This way it's a lot of work for one small team to write the C interpreter, compared to a still quite a lot of work for every team with a C extension that would need to port it to Python.

There's a reason new languages, compilers are made rather than everyone just porting their code to the Assembler of new CPU.

Same reason applies here.

Writing a C interpreter isn't that bad, especially considering you can co-opt Clang's frontend and use it to do all the syntax and semantic analysis.

In the case of python, we already have Cython that allows the porting of Python to C. Why do work creating the inverse (with potentially much harder semantics to JIT) when we can already go from Python to C.

Agreed! JIT's are interesting but we use Cython in production to build Userify Enterprise (SSH key management) (self-hosted distributable). It works incredibly well and we are thrilled with the result. (We do require a bunch of third party pip installs rather than wrap them into the binary.) This was partly because one thing that we had trouble with was lots of imports, especially for (ironically) compiled C libraries. It's a lot easier and probably safer to just give a user a script of installs (or a requires manifest.) Someday soon we'll probably even make it installable via pip and bring things full circle. (We're exploring a free five server version.)

Userify's architecture is probably a bit unusual. It's distributed as a single binary that uses an external redis server for locking and stores data in either S3 or a local filesystem (NFS, EBS, iSCSI, etc.) When it first launches, it automatically installs any missing python prereqs (if pip is available) and explodes out a web server static file repo in case you want to front-end it with nginx (etc) and then starts up multiple web servers (HTTPS, HTTP, alternative ports, etc). We've found Python to work extremely well for all of this. We've looked at a few other languages that we think fit our core requirements (incl lua, scala, and haskell) but Python works pretty much perfectly for this use case, we can scale it horizontally using Redis as a synchronization mechanism, and the GIL hasn't caused any problems for us because we just add more processes as needed. Cython is really fantastic and 'just works'.

Things aren't perfect.. we're still on Python 2.x, but we're going to be moving forward as soon as all of the third-party libraries we rely on move to 3.

Extensibility from C isn't there for people to accelerate their slow code, it's much rather there to cooperate with existing libraries, and C is the lowest common denominator there.

Reply on news.ycombinator.com