From Python to Go: migrating our entire API
blog.repustate.com
blog.repustate.com
Really liked: the simplicity of Go's build, not having to spawn N processes on the server side, the compiler catching so many issues, the very strict formatting rules, all a major win over Python. However, didn't like how verbose the code turned out, especially for parsing or any data manipulation tasks. For server-side debugging/tools, it's also a bit of a build-your-own adventure.
Elixir I found to be much more pleasant to write and subsequently read, more well-suited for back-end tasks, great for generating and parsing data. Also, more things in the toolset for back-end deployment, in large part due to the OTP stack.
Perhaps Go's ideal use case is local system tools, compilers, or anything else that requires concurrency (let's say, creating tools like "ls," or Apache bench, or gcc, or cURL), and something like Elixir is more ideal for building actual servers.
If simplicity were the primary concern, they could have stuck with Python. You do get a much more distributed element with Elixir/Erlang (which certainly helps with volume), but I'm not sure they'd reach that 10ms response time average with it.
Of course that's a bit like a benchmark (meaning, it means absolutely nothing to you/your code). If there were any more latency in the Erlang solution it would be the same algorithmic issues that could pop up in any software.
Go is more efficient for CPU throughput, but BEAM has been fine tuned for decades to keep latency down and wouldn't be able to top for responsiveness.
My personal opinion is that they made a mistake. Reporting about swapping out a stack like this is weird because it looks like they don't know what they're doing. What's next month's blog? We're Moving to Rust? I'm glad they're satisfied, but you gain improvements anytime you do a rewrite- even in the same language. I wouldn't have migrated my existing stack anywhere. I'd use PyPy which would've done wonders for responsiveness on the existing product and adopt containers for deployment.
That's an easy claim to make, but you don't know whether their stack would actually get much improvement out of PyPy. We've had trouble in the past using PyPy with C extensions. Trying to migrate to pure Python equivalents was a headache. Once we managed to get a PoC of a service running under PyPy, we saw very little improvement.
But I'd be hesitant to leave a massive, thriving ecosystem like Python (especially if on 2.7 which has everything). Win some things, lose some others when you can just adopt containers and PyPy and gain the benefits that Go has without losing Python's advantages.
i wouldn't consider djangob for anything other than a heavily cached content site.
seems like it's a company that just plow through problems. i bet whatever the problem is in go they will find only next week.
python 2.7 and 3 have a marvelous, but not very well documented, multiprocessor module. it gives you more performance for that the cython and other shenanigans.
but i guess using the existing tool right doesn't give promotions.
... yeah I'm making a lot of assumptions about then but that's just because I'm salty today. they should still learn to use python multiprocessing module. :)
I've had a huge amount of success using pyinstaller for this problem, shipped many products into hugely diverse environments.
"We migrated our entire API stack from Python (First Django then Falcon) to Go, reducing the mean response time of an API call from 100ms to 10ms We reduced the number of EC2 instances required by 85%"
For pretty much everything else that people use multithreaded applications for in the past (encoding, image editing)- multinode processing is vastly superior to multithreaded. Erlang calling out to C (if even necessary) is going to win there over most. Or you could use Python or whatever you wanted.
The Java/Go way of local multithreaded applications seems archaic to me.
Perhaps this is a bigger thing among corporate programmers? I've only worked at smaller companies where they are more open to experimenting if you talk them into it.
[1] See this essay from Rob Pike where he says Go is attracting programmers mostly from dynamic languages (though he raises C++ programmers as the chief contrast, but still mentions Java too) : http://commandcenter.blogspot.fi/2012/06/less-is-exponential...
I follow some PL lists and there seems to be a sizeable group of people who prefer a position in their favorite language. Of course, I don't know what portion of the overal market they represent.
Some times, it's even better not to look.
Python needs a .war format and I'd like to see more people get behind https://github.com/pantsbuild/pex