Hacker News Reader

Comment by axaxs | Hacker News Reader

> is it's much more verbose/harder to handle edge cases - it feels like I'm less productive writing the program, but end up building something much more stable than the Python equivalent

This is exactly how I felt moving from a Python codebase to Go. But you realize on the 100th or so Python stacktrace that it's worth the investment up front, if not for type safety alone.

> I really need the performance for my workload

Careful with that assumption. A pure Go application will in many cases be more performant than a pure Python one, but Python has a better C interop story. In the particular cases of parsing XML and JSON, I'd wager Python is much faster, actually.

View on HN

> on the 100th or so Python stacktrace

Go's not perfect in this respect, either. I had your typical map[string][]interface{}, because the data in the map was an array of records, but different records in different cases, and made an error in an append(), so ended up with nested arrays. That's correct according to the type definitions, but unwanted. Generic could have helped here. So I think the language still needs to grow a bit.

I tend to write custom parsers for jsoniter in this case so that I catch any unexpected data types as early as possible.

> This is exactly how I felt moving from a Python codebase to Go. But you realize on the 100th or so Python stacktrace that it's worth the investment up front, if not for type safety alone.

Yep. Maybe I just suck at programming but I couldn't write enough tests to avoid not being fed up debugging stack traces in production. I started using Python's type hints, since this was the source of most of the bugs which helped but I'd rather just use a strongly typed language next time.

> Careful with that assumption. A pure Go application will in many cases be more performant than a pure Python one, but Python has a better C interop story. In the particular cases of parsing XML and JSON, I'd wager Python is much faster, actually.

I've got a few million records pulled from an API so I'm extremely I/O bound and goroutines are easier to work with than Python concurrency. I've also got reasonably complex data processing beyond just XML/JSON parsing.

I think on the whole Go is still going to be faster, but thanks for the heads up.

Have you tried async? I find it much easier personally to reason about and debug than channels, and is perfect for IO-bound problems. If you really like channels, note that Python 3.9 will have sub-interpreters and channels as well: https://hackernoon.com/has-the-python-gil-been-slain-9440d28...

Unfortunately I deploy on App Engine which is still at Python 3.7.

Goroutines are quite a lot nicer than async for a few reasons. Firstly, there are no “forgot to await” errors, secondly there is no need to worry about some library making a sync call way down the call stack and therefore blocking your event loop, and thirdly goroutines can use all cores on a CPU to parallelize CPU intensive tasks, which means you don’t block the event loop unless you really are out of CPU. Blocking the event loop is a really big deal since it can cause health checks to fail which can cascale onto other instances and bring your app down. We’ve experienced this with both CPU- and IO-bound event loop blockages, and they’re really hard to debug. These just aren’t issues that happen in Go.

A lot of what you describe is for CPU bound problems, most of the problems I face personally and the the problem, in particular, the parent mentioned are IO-bound. Async in Python is a near-perfect fit for this kind of problem space and is really easy to reason about and debug as you can step through and debug one line of execution at a time.

Only one of those problems is CPU. The others are type system and IO issues, and they happen in IO bound services too.

Regexps is another area where GoLang used to lag behind Python (for certain types of expressions?). Not sure if things have changed, but this issue is still open 4 years later: https://github.com/golang/go/issues/11646

Not sure I fully understand the NFA vs DFA/re2 choices -- seem like the issue above conflicts with the comment below. Can someone clarify?

From [1]: "Go has chosen to follow the re2 path (not surprising, since Russ Cox is a major author of both Go and re2). re2 has much better performance characteristics than some other regexp engines, in that it never has an exponential slowdown, but that comes at a cost for other regexps (https://swtch.com/~rsc/regexp/)."

[1] https://www.reddit.com/r/golang/comments/84o986/why_is_gos_r...

> This is exactly how I felt moving from a Python codebase to Go. But you realize on the 100th or so Python stacktrace that it's worth the investment up front, if not for type safety alone.

Luckily, you don't have to choose between those two outcomes:

https://github.com/python/mypy

Python's optional type hints even with mypy are nowhere near strict/expressive than a statically typed language like Go. Type hints in Python are an afterthought and it shows in a lot of cases.

Do you have any concrete examples?

Type hints in Python are IMO great and quite expressive!

You can create sum types (a bit verbose but they are there).

You can introspect them at run time and create things like pydantic[0] and fastapi[1].

Strict null checks with Optional[T] are great!

Now I'm not too familiar with Go's type system, but with protocols, Unions, Optional[T] and other generics offered by MyPy[2] I'd wager the type system is __more__ expressive than Go.

[0]: https://pydantic-docs.helpmanual.io [1]: https://fastapi.tiangolo.com [2]: https://mypy.readthedocs.io/en/stable/protocols.html

The type system is more expressive, but much clunkier and also it needs to be expressive to accommodate all of the weird things people do in Python (and you still can’t model SQLAlchemy types).

Using encoding/json maybe but also using 3rd party libraries? jsoniter is pretty fast, I just ran some tests and for my use cases it outperformed Python.

> In the particular cases of parsing XML and JSON, I'd wager Python is much faster, actually.

Up until you need to do anything interesting with the deserialized data. :)

Reply on news.ycombinator.com