Show HN: Golang statistics package with 100% code coverage | Hacker News Reader

Show HN: Golang statistics package with 100% code coverage

github.com

40 pointsanonfunction22 comments

Hey Hackers, this is my first full-featured Golang package. I took a lot of care to ensure it was well tested and provided a lot of common functions that I felt were missing from the standard library. In addition it has a lot of other statistics functionality like regressions, outliers, correlation and more.

The coverage seems to be only line-based. I don't know if it's relevant for Go, but you might want to see how the coverage looks like for branch-based analysis (if Go has software for that).

Otherwise, nice to see someone take quality serious. :)

The coverage is based on the built in go tooling, which supports line or function based analysis. In addition it can start a webserver where you can browse coverage by file with color output.

    # Generate coverage data
    go test -coverprofile=coverage.out
    
    # View coverage by function
    go tool cover -func="coverage.out"
    
    # View coverage line by line
    go tool cover -html="coverage.out"

This seems like a really cool project to hack on. I noticed you have no issues on the github repo, but where do you want to go with it next? Or what would you like more eyes on?

Thanks for your interest!

I think the biggest thing to do is finalize the public API and release v1.0.0. I started with only functions but have since added types, so I'd like to figure out if I should support both or stick to one or the other. I have a feeling I could use an Interface to make stats work for both []float64 and []int but I'm not sure about that.

Another thing I'd like to do is add more test suites, I know they are out there for more mature statistics packages in other languages that could be ported to Go.

Speaking of mature packages, I think the Gnu GSL Statistics[1] could be a good roadmap for future functionality and API design decisions.

1. http://www.gnu.org/software/gsl/manual/html_node/Statistics....

I mentioned elsewhere in the thread, but if you're interested in help buliding statistical tools for go, Gonum is always looking for more hands. Look at what we have in github.com/gonum, and/or come introduce yourself at gonum-dev

I've been using this package for awhile now and I can attest to the quality: It is great! It has powerful and easy to use statistics tools which IMHO should've been included in the standard library in the first place.

Thank you anonfunction et. al. for your sharing the fruits of your labor with us! :D

Thanks a lot, it means a bunch to me :D

Gonum has many of these capabilities, but with a slightly different mentality (gonum has weights and sticks to pure []float64 while this defines types and methods, i.e. Coordinate, Float64Data)

gonum/floats for max, min, round, sum, etc.

gonum/stat for mean, std, variance, etc.

gonum/stat/sample for sampling without replacement

Stats also works fine with pure []float64 data. The types allow for a very clean API for working with data and statistics. You can compare using functions[1] or methods[2] in /examples.

I started with only []float64 but wanted to allow stuff like this:

    func getData() []float64 {
    	// Do stuff to get the data and 
    	// store it in var data []float64
    	return data
    }

    var d Float64Data = getData()
    fmt.Println(d.Max())

1. https://github.com/montanaflynn/stats/blob/master/examples/m...

2. https://github.com/montanaflynn/stats/blob/master/examples/m...

If you're interested in developing more stats stuff, we'd love to have you as part of the effort. gonum/stat/dist in particular needs a lot more functionality.

Why are these all separate repos? And is there any centralized documentation? Can't find any.

They are separate repos because they have different mentalities. Stat is basic statistics for a (weighted) set of values. Floats is closer to dealing with properties of a vector (norm, distance, etc.). Sample has sophisticated sampling from distributions (metropolis hastings, etc.). gonum/stat/dist has sampling distributions themselves.

Documentation can be found in godoc, i.e. godoc.org/github.com/gonum/stat, godoc.org/github.com/gonum/matrix/mat64, etc.

Regarding your examples, shouldn't Round return 3.17 and shouldn't Median be 3.5?

You are correct, it's been fixed with this commit[1]. My excuse is it was late when I added those output comments.

1. https://github.com/montanaflynn/stats/commit/a3efee7551d14a9...

Nicely done.

Is returning error common in go (as opposed to raising)?

Thanks! Go doesn't have exceptions and it is idiomatic[1] to return the error type. Dave Cheney has a really great blog post[2] which he describes why go does away with exceptions. The Golang wiki on GitHub has a great section[3] on errors as well.

1. https://gobyexample.com/errors

2. http://dave.cheney.net/2012/01/18/why-go-gets-exceptions-rig...

3. https://github.com/golang/go/wiki/Errors

> Go doesn't have exceptions

Go has panics, which are pretty much the same thing as exceptions in other languages (the details of how they are used are slightly different from, say, Java/Python/Ruby exceptions, but much more than, say, Perl 6 exceptions.)

Go has a strong stylistic recommendation not to expose panics to external consumers of code, and it is idiomatic to instead use error returns from publicly-exposed library code. But "Go doesn't have exceptions" is mostly misleading.

Rob Pike, one of the three creators of Golang, disagrees with you. In his blog post less is exponentially more[1] he shares a list of significant simplifications in Go over C and C++ which includes:

> no exceptions

1. http://commandcenter.blogspot.pt/2012/06/less-is-exponential...

And yet "panics" exist, with very similar semantics to exceptions in C++/Java/Python/Ruby.

Sure, there's differences between Go panics and any one of those implementations of exceptions (just as there are between them, or between any of them and Perl 6 exceptions.) But panics are clearly a variation on the exception theme, just as each of those forms of exceptions is.

The strong idiom of not having panics exposed across public interfaces of libraries is a cultural change, which to the extent that it is followed does present a simplification compared to ecosystems where you have to deal with both returns and exceptions in library code. But panics very much are exceptions, and changing the name doesn't change that.

As I was originally answering a question regarding whether it was normal to return errors or raise exceptions, and because I've read Dave Cheney and Rob Pike's blog posts explicitly stating Go does not have exceptions I replied as such.

However, after doing some research I learned that in addition to Panic() there's also Recover()[1], so I'll concede that there is similarity between panic/recover and throw/catch. Prior to my learning about Recover() I was under the assumption that Panic was just used to print a message and close a program with a non-zero status.

1. https://github.com/golang/go/wiki/PanicAndRecover

> "Go doesn't have exceptions" is mostly misleading.

It's worded too strongly perhaps. While `panic` behavior is similar to `java.lang.Exception`, it is not idiomatic to panic on problems like IOException/SQLException. They are detected by return codes to be handled immediately, rather than a broad swath of code wrapped in `try`.

This in Java:

    try
        code
        code
        code
    catch errA
        handle
    catch errB
        handle
    catch errC
        handle

Would be this in Go:

    code
       handle errA
    code
       handle errB
    code
       handle errC

It's fair to say "Go doesn't use exceptions" since Go doesn't use panics for common error handling.

Reply on news.ycombinator.com