Ask HN: Python or Go? | Hacker News Reader

Ask HN: Python or Go?

12 pointscweagans19 comments

I have a strong background (8 years) in PHP development with Drupal and other misc frameworks/components. I'm a comaintainer for a system called Aegir, which is a free software control panel for managing large fleets of Drupal sites. Aegir is currently written in all PHP, including a daemon for running background tasks (which is not a good thing).

We're in the process of re-architecting Aegir to be a frontend + simplified API for Kubernetes or other prerolled container management systems (Flynn, Deis, etc). Our goal is to create a superset of what Aegir supports.

Anyway, we determined very quickly that PHP is not a very good language for the majority of what we're building. We've narrowed down the alternatives to Python or Go, and we're wondering if there's any strong, technical reason to go with one or the other for systems/container orchestration type work.

View on HN

First of all php, go or Python are all valid choices. You can shoot in the foot with all of them. Personally I prefer Python, but worked a bit on go. I personally will recommend Python given its rich library set helping you secure your api's end points. Doing same in go is pretty convoluted with more lines of code, although you get performance, but that's good for consumer facing part. For software like control panel Python is a very good choice. In general Python is better where you need a decent 3rd party library support with concise, clean and readable code. Go is a good choice when you need performance and scale although Python can do it as shown by Instagram and Google itself but it requires clever programming and a lot of cruft. For systems/containers they build there code in go but still needs to integrate with c code of kernel. So the code is complex and since your tool is focusing only on control panel part I feel Python is a way to go.

"Python given its rich library set helping you secure your api's end points. Doing same in go is pretty convoluted with more lines of code"

Is there any way you can elaborate on your experience that brought you to this conclusion?

I am rather curious, as Go has the same middleware concept that django has to add in security, session, and token information to an http call.

Try building a production rest api using flask, with encryption, authentication and authorization backed in, taking care of internationalization and localization. See for yourself how many lines of code you will need and obviously see which one is more readable and cleaner. Do the same with go and you will know what I'm talking about. Go is good for its own use case for solving Google scale problems which requires additional efforts and time. Python is very approachable from the beginning. Moreover the wealth of documentation and help makes it so much easier when you hit a roadblock. You will need twice the amount of time to do the same in go.

Currently doing a fair bit of both Python and Go. Python: A decent packaging system. Lots of packages. A debugger!

   Go:
     Go-routines and channels are quick, easy to use concurrency.
     Compiles down to single distributable binary.
     Lousy packaging.
     Debugger somewhere between primitive and none.

In Go we're consistently writing more lines of code than in Python, fighting to get third-party package versioning correct, and reverting to Printf as primary debugging technique.

On the other hand, Go's general binary distribution is much simpler, running the compiler is like running lint on your code, concurrency is relatively easy.

At the moment, we're moving from a primarily Python environment to an environment to where we use Python for all user-facing web applications and Go for web service APIs. Your milage may vary.

Its not clear to me why you think "PHP is not a very good language for the majority of what we're building". In my opinion, you should have a strong reason for rewriting an application.

A good answer strongly depends on that. Here are some thoughts:

1) If it is for style and maintenance reasons, then Python with one of the frameworks (Flask is very easy to get started) would be a good start.

2) If performance is a reason, then see if you can separate out a portion of your PHP code to call the Go service. You can use message queues as a bridge between the services for better scalability.

For most applications pure Python will not hurt. Python is very productive to develop. If you need to scale in the future, you can identify bottlenecks and rewrite them.

What issue do you think that your going to avoid with python that you have with PHP?

On Golang:

1. There are a bunch of good frameworks out there for you to bootstrap with -- if you pick one realize its going to have shortcomings -- if you don't pick one and you think your going to use the standard library + tools like gorilla and alice, your going to end up doing a lot more work than you expect. -- a lot of the info on this front is stale, look for the context package on the google blog to see how the thinking is evolving (and its pretty rational)

2. There are parts of Go that look great on the surface, and work for "small programs" but if your building in the large they are going to be lacking. Logging is the example that comes to mind, and a lot of the "solutions" are just coming up short at the moment. Profiling on OSx being broken is another (and this isn't the fault of the GO, but rather apple.

3. Your going to have a very different workflow with go than you do with a python or PHP. This same would hold true if you moved from PHP to C or java. The distinction is that C and Java have a LARGE toolchain to support your transition, here again Go is lacking.

All that having been said, I would choose Golang over Python or PHP for what your trying to do. Your going to pay a large tax upfront (in both ramp up and real time development costs) for something that (potentially if you don't screw it up) is much more maintainable in the long run.

PHP is horrible at long running processes, among other things. We regularly run out of file descriptors and memory usage is a big issue too. The "solution" is for the process to kill itself every couple minutes and restart with a clean slate. It's a gigantic hack. A long running process on Python, OTOH, is less of a problem.

More generally, it becomes pretty clear when you're using a tool for something it's not really meant to do. PHP is an okay programming language/runtime, but there are other tools that would do some of these jobs better, hence our interest in Go and Python.

Python is "less of a problem" but you still may hit that wall. You can do "stupid stuff" in any language and have that same outcome, php just gives more rope than the rest!

Recently, on a short term contract, I had to deal with a lot of PHP code that was running in a similar manner. I ended up using golang, to monitor the queue and spin up Goroutines that executed the PHP code ONCE. Between the concurrency, and php back in "one and done" mode the problem was "solved" and the internal team happily took over finishing the porting to pure Golang. Its a great example of Go "in the small" and it really does shine.

Go "in the large" is a whole other matter. You should read the following (in this order)

http://thenewstack.io/a-survey-of-5-go-web-frameworks/ <<< Im a user of gorilla/mux gorilla/context and alice for middleware chains. Gorilla/context and alice are not mentioned by this article, but the lament a the end about the concept of context being lacking in golang is spot on.

https://blog.golang.org/context <<< but this would be better for context however its does not mix with the current HTTP server (no ones solution does)... its from the go team but there isn't a lot of indication on how to use it and be future proof

https://joeshaw.org/net-context-and-http-handler/ <<< explains the context package well

https://github.com/remind101/pkg <<< has wrappers for getting net/context into gorilla and alice..

The end result is non of the GO web support is what I would call "great" yet. However it isn't stopping me from building with them. Look at the tradeoffs your going to have to make, be smart about how you couple your business logic to your server/framework, as your framework will quickly become a hard dependency. If your careful in that implementation, this http://blog.golang.org/introducing-gofix might be a way to back out one of those dependency if your unhappy.

The tooling to help you root out an issues exists and you should know its there and how to use it: http://blog.golang.org/profiling-go-programs http://saml.rilspace.org/profiling-and-creating-call-graphs-... https://deferpanic.com/blog/compile-time-code-weaving-in-go/

Yeah, we definitely won't be writing a web app in Go. We can just build it with Drupal and it'll be fine. Most of what we'd use Python or Go for are small services that handle one thing that run in their own container. https://github.com/progrium/gitreceive is a good example of the kind of services that we'll be building (we didn't build gitreceive, obviously, and we probably won't re-implement it because it's already pretty good. Just wanted to give an example)

If your server needed to scale high (say with scores of provisioning events per minute, or especially low latency), Go's lighter weight would be a selling point. But orchestration is about coordination, often of dissimilar things; that means dealing with different APIs, end-points, and subsystems. I've found Python to be quite good at such tasks. Its higher level nature and well-developed Web frameworks (e.g. Flask, Tornado) seem a good fit.

I've done a lot of work in both Go and Python recently - using httprouter and flask, respectively. Python libraries are a lot more mature. But in bigger projects, debugging in Python can be incredibly painful. I think it comes down to the fact that Python is a really complex language. On the other hand, Go is so simple. (And you may hate it for this, at some points.) Sometimes, I find it more helpful to think of Go as a higher-level C rather than a Python competitor. The syntax is easy and familiar when you get started, but I think you will miss a lot of the Go beauty if you are looking for a "faster Python".

tldr; Go is not Python, but it is still beautiful.

Depending on your target date and algorithms needed, I would say stick with PHP as your front end and move your backend to either a Go based service or a Python one.

The single binary feature of Go makes deployment really easy. And the standard library has everything you need to do an API. I chose the gin framework in my case with the standard html/template. The front end uses knockoutjs.

If you need some specialized library like machine learning or natural language processing, I would choose Python as there are many great libraries out there.

Go is a compiled language and is easily used in a cross-compiled situation. The executables are single image with no dependencies on external libraries.

Python is interpreted and deployment requires a large-ish runtime environment as well as making decisions about v2 vs v3. You can use virtualenv to address these and related issues.

Personally, I like Go, but keep ending up using Python and JavaScript far more often.

Go is a clear winner IF you don't want to have difficulty dealing with asynchronous functions and synchronous functions. In Python (and JavaScript and other single threaded languages) asynchronous functions can use synchronous functions, but not the other way around. Synchronous functions that expect output from asynch functions will end up with weird bugs, or might not even be able to get output, since asynch functions often hand their outcomes to callbacks within the function. So everything that requires an asynchronous operation ends up getting infected with the coding style required to handle asychronicity; you're forced to code everything that needs an asynch op using promises, callbacks, etc.

Go, being multi-threaded, and being designed to be usable by teams with a broad range of skill, eliminates the need to worry about asynch vs. synch code.

See this article explaining the problem with Async functions. "What Color is your function?" http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y...

Quote from the article:—————————————————————————

Go is the language that does this most beautifully in my opinion. As soon as you do any IO operation, it just parks that goroutine and resumes any other ones that aren’t blocked on IO.

If you look at the IO operations in the standard library, they seem synchronous. In other words, they just do work and then return a result when they are done. But it’s not that they’re synchronous in the sense that it would mean in JavaScript. Other Go code can run while one of these operations is pending. It’s that Go has eliminated the distinction between synchronous and asynchronous code.

Concurrency in Go is a facet of how you choose to model your program, and not a color seared into each function in the standard library. This means all of the pain of the five rules I mentioned above is completely and totally eliminated.

So, the next time you start telling me about some new hot language and how awesome its concurrency story is because it has asynchronous APIs, now you’ll know why I start grinding my teeth. Because it means you’re right back to red functions and blue ones.

—————————————————————————

If you do not have to worry about asynchronicity, then this advantage may not mean as much to you, and Python's larger user base and libraries may be a compelling advantage. But if you do, not having your world of functions divided into two "colors" is a huge advantage.

I would say go has far easier primitives to be a tool for orchestration. I myself have worked with both languages.

go. The concurrency primatives are great. Start doing multi-threaded things in python and watch your computer run out of memory very quickly.

I actually had to make a decision on the same thing a couple of weeks ago.

Golang is probably the better programming language. The main reason for this is its statically typed, and the type system is dead simple.

If I want rock solid code in Python, I have to write tests and isinstance checks everywhere. You begin to feel like making your reinventing the wheel just to get type safety.

Golang on the other hand won't let you compile of the code is wrong. In various ways the linter is more strict than pep8.

I also love how they kept struct. It leaves the impression your working with bare metal. The idioms for goosing, like Python, become intuitive and set in.

The issue that made be give up golang is the libraries are a joke. I'm talking dead simple things like orm and logging alone rule it out for anything serious.

You also begin to miss things like REPL, Python prompt toolkit, nummy, Ipython, etc. Django and sqlalchemy are things you'd kiss.

Golang is a better programming language, but it's way too green. You'd be reinventing the wheel constantly,

I will 2nd that native logging in GO is awful (and most of the packages for logging are pretty bad too). For me logging was missing basic levels, and any sort of contextual capability.

I would like to know what issues you had with it, as I'm in the middle of building out my own replacement logging package

> I would like to know what issues you had with it,

Specifically, I wanted "logging" from python; leveled logging handlers, with colors (like colorama) and per-module level permissions.

> as I'm in the middle of building out my own replacement logging package

None of the alternatives did exactly what was needed. And this extended other modules as well. It turned out we would be maintaining forks of all these projects.

If you haven't already, https://github.com/avelino/awesome-go#logging is a good resource.

If the original poster / you do decide to go with golang I wish you the best. It's a great programming language - but you're time spent reinventing the wheel will outweigh golangs benefits over python.

And it's kind of a shame. The benefits of the static type checking, concurrency, compile times, testing, etc.

I can tell you, in every category - at the present moment - golang's available libraries fall short to what you get in python. Not golangs fault, it's just want maturity gives you.

Even things like asserts in python, (assert is even a builtin in python, no need to import unittest!) is non-existant in golang. It's not a "bad" thing.

My outlook for golang it strong - my concern is that you may drain away your runway doing what you should otherwise be doing on freetime.

Reply on news.ycombinator.com