Hacker News Reader

Comment by plg | Hacker News Reader

I really want to get into Go, I like that it's opinionated, I like that it's compiled, I like that it's garbage-collected, I like that coroutines are built-in.

My primary use case is scientific computing, both data processing and interactive visualization.

I know Julia is an option as well.

For reasons I don't want to bother getting into I dislike Python.

Thoughts?

View on HN

It's not well-suited to scientific computing at all.

I use and like Go for writing network servers and ETL processes. In a scientific context though, the type system is awkward in the extreme, there is essentially no library of modules, and the interactive visualization story is nonexistent.

Python, R, Julia, even C++ would be better options IMO.

(I'll clarify again: I like Go! I just think it's not well suited for this context)

If one wants to be a pioneer and be one of the people who starts the movement, now is the time to jump in and make a name for yourself.

This is absolutely true. It's also a lot of work :)

I'd suggest "now" is actually a terrible time to try to make Go do scientific computing. With generics all-but-guaranteed to be incoming, but also not here yet, a lot of what you would do is going to be superceded in the forseeable future, on the same sort of time scale as your library's expected completion date.

I'd say that even once Go has generics, that will simply take it from being an awful scientific programming language to a mediocre one. I don't really understand why there's a few people who seem to think it's a good idea to try to do their scientific computing in Go when there are so many better options for that that already exist.

Post-generics, though, if you really insist, it will probably be the case that Go can be upgraded from "mediocre" to "tolerable" with a lot of library work. (Part of the "mediocre" is lacking libraries. That aspect can be fixed.) But it'll be hard to start that work without running generic code. Even if you assume the current documentation is the final specs, you still won't be able to guess the performance implications of anything you'd be blindly writing, and performance is very important for this sort of code.

With generics all-but-guaranteed to be incoming, but also not here yet, a lot of what you would do is going to be superceded in the forseeable future

I didn't mean "now" as in right this second. Generics will indeed be key.

I was looking at this very recently. It's probably impossible - not just difficult - to write a Numpy equivalent for Go as a library. I didn't feel I had what it took to extend the compiler to make that work, but some day someone will.

https://github.com/gonum/gonum is that.

It's frustrating how much résumé-driven development I'm seeing on problems the industry already solved. I'd rather work on harder problems using more powerful tools.

I'm not certain that Go is a great fit for this particular area. Others have mentioned generics, but I'd also worry that the lack of operator overloading is a weakness. There is a lot of Fortran out there, so operator overloading is not required, but I'd worry about the ergonomics of Go, especially given the strength of the other options.

The best course of action for Go on this front, would be to expose operator overloading of numeric operators, combined with a community expectation that this isn't to be used for anything but numeric operations.

What would be the disadvantage of using a syntactic preprocessor to accomplish the same thing? We have an API for parsing golang. What if there was a way of marking certain files to be parsed and re-written with function calls? It could be done with a file suffix, and this could be made fairly convenient.

> The best course of action for Go on this front, would be to expose operator overloading of numeric operators, combined with a community expectation that this isn't to be used for anything but numeric operations.

I don't think this solves the issues. For example, I have yet to think of a way to overload `+` in a consistent and performant manner. E.g. think of `a += b` vs. `a = a + b`. Either you make them separate operators (in which case there's no guarantee they do the same thing - see for example Python, where they commonly differ) or you are overallocating in the common case.

Either you make them separate operators (in which case there's no guarantee they do the same thing - see for example Python, where they commonly differ)

Geez. That's obnoxious. Why do we programmers do this to ourselves?

or you are overallocating in the common case.

I see. I was thinking along the lines of being able to use complex numbers with arithmetic operators, not along the lines of doing high performance crunching of floats.

Fortran supports operator overloading since Fortran 90, so 4 language revisions ago.

Agreed. For data stuff if you want type saftey you really need a language with support for recursive types. More languages lack this feature than have it.

If you have a very well-defined problem and aren't doing much exploratory programming, Go works reasonably well for demanding math stuff. This describes a lot of cryptography work, and Go is a top-tier language for that. But for exploratory scientific and math programming, Go is pointlessly painful. Julia is a better option, and I would do Python Sage before I did Go, despite not loving Python.

I used go in machine learning contexts extensively while writing graphpipe[1]. Go is a fantastic language for servers, and distributed communication. Unfortunately, the lack of generics and dependence on interfaces and reflection makes writing things that deal with multidimensional arrays pretty terrible. See, for example, the janky conversion code in graphpipe-go[2] to convert a multidimensional slice into contiguous row-major arrays. Also, libraries that try to create a numpy eqivalent end up with uncomfortable interfaces due to inability to overload operators. I agree with some of the sibling comments that go 2 will help but probably won't make things particularly pleasurable. Rust would be a more interesting route but definitely doesn't have the adoption of go.

[1] https://oracle.github.io/graphpipe

[2] https://github.com/oracle/graphpipe-go/blob/master/helpers.g...

wrt n-dim arrays: yes, it's a pain point. we (Gonum devs) tried to push for an "n-dim slice" type a couple of years ago. it didn't pan out.

generics may help in that department.

in the meantime, there's Apache Arrow:

- https://blog.gopheracademy.com/advent-2018/go-arrow/

>My primary use case is scientific computing, both data processing and interactive visualization.

If you actually have a primary use case of "scientific computing, both data processing and interactive visualization", ignore Go.

And rethink Python.

Julia can be used, but is still fringe.

R perhaps.

It's worth using. I do science things and use Go. It doesn't require much more mental overhead to write than a scripting language (possibly less once you're into it), it's pretty fast without arcane knowledge, it's easy to maintain, binaries are easy to deploy, and it has a super fast startup time unlike Julia, if you care about that. Overall, I'd highly recommend it.

I use it to push files around and possibly parse and move the data those files contain. But once it gets to strictly numerical things, the purpose built tools are the only real choice. Scientific libraries for Go are nearly non-existent, or just copies of fully formed libraries from somewhere else.

Python has a huge ecosystem for this and levering it is a good idea. In one of the projects I'm working on, I train and test tensorflow models in Python but load it in Go for inference - these types of hybrid approaches can work great for scientific computation.

Go's main use case is for writing concurrent servers (mostly web servers) or daemons and command line applications that need to be really fast. Go is a generic programming language, but Python is "more generic", which I mean that you can go far more in a lot more domains (one of them is scientific computing and data analysis) than with Go. If you want to use go for data analysis I suspect that it would feel "a lot more work" to do the same thing than it would take in Python, mostly because it's a lower-level language and the ecosystem is not that huge than for Python yet.

others have already mentioned Gonum:

- https://gonum.org

Gonum is almost on par with _e.g._ NumPy/SciPy (most notably lacking: ODEs). So still ways to Go but it's getting there.

Go-HEP is my attempt to bring a few High Energy Physics oriented packages to particle physicists:

- https://go-hep.org

I've also written a few words on why I think Go is great for science:

- https://sbinet.github.io/posts/2018-07-31-go-hep-manifesto/

TL;DR: Go is great b/c it brings great s/w engineering practices and a s/w engineering-friendly environment to scientists.

Admittedly, generics will change how packages are written. So some code churn will take place when/if they land, but the Go community learned the lessons from Python2/3 and Perl5/6. Expect a better migration path.

Lastly, I guess the 2 remaining weak points of Go are:

- runtime performances sub-par wrt C++ or Rust

- GUIs (which may or may not fall into "interactive visualization")

That said, the Go community worked on a Go kernel for Jupyter:

- https://github.com/gopherdata/gophernotes

- http://gopherdata.io/

hth, -s

Gonum is neat, but to the previously-made point about Go's type system making stuff more painful than it needs to be in this application: gonum's linear algebra is defined over float64 and int, which is problematic if you need arbitrary precision.

I guess it's scientific domain dependent. I've very rarely needed something else than float/double in my C++ days or REAL*8 in my F77 ones.

sure, when you need it, you need it. but float64 caters for a good 99% of my usual work day.

From a user POV, seamless installation of packages is a great boon. From a grid/cloud operator POV, static binaries are great too.

These are things that are great for product development and devops and not in fact all that valuable in scientific computing, which is a reason why so much of it gets done in Python.

> These are things that are great for product development and devops and not in fact all that valuable in scientific computing

I disagree. Again, this may very well be science-domain dependent, but in High Energy Physics (where, finally, Python is recognized as a mainstream language, BTW) many -- if not all -- of the pain points that slow down undergrads, PhDs, post-docs and researchers at large, are these Go features.

yes, the time from "idea" to "little script that shows a bunch of plots" on a subset of the overall data is definitely shorter in Python (exploratory analysis is really really great in Python). but, at least for LHC analyses, python doesn't cut it when it comes to Tb of data to swift through, distribution over the grid/cloud, ... you die by a thousand cuts. and that's when you are alone on one little script. LHC analyses can see well over 10-20 people assembling a bunch of modules and scripts. You really long for a more robust to (automated) refactoring language than python, pretty quickly.

What's wrong with Julia, a language purpose built for scientific computing?

There's Gorgonia too! :)

In my opinion, the big rub for using Go in scientific computing is the lack of a REPL. The nature of Go essentially requires it to compile. On the plus side, compilation is very fast so iterating small code changes is practical. But it's still not nearly as nice as typing commands into the prompt and seeing what happens.

Beyond that, Go is easy and performant. It's great for paralellizing workflows via concurrency and compiling tools for distribution. So if you know what you want to do and need to scale up, it could be a great choice.

Go works with Jupyter Notebook pretty well. It's a little bit complicated to set up, but it is the same REPL than you use for Python: https://walkman.cloud/s/dtoSfw753YSiMAs

there are actually a couple of real interpreters for Go:

- neugram.io

- https://github.com/cosmos72/gomacro

here is a mybinder that used to use the former, and now uses the latter:

- https://mybinder.org/v2/gh/go-hep/binder/master

Dlang is better suited for that use case. Have you looked at mir?

Reply on news.ycombinator.com