Hacker News Reader

Comment by pbnjay | Hacker News Reader

It's a nice wrapper on a powerful technique. Could be very useful to some folks - but requiring numpy and pandas is kind of excessive.

View on HN

Yeah, I'm not sure that this is a fair comment: how would _you_ avoid the necessity of pandas and numpy? Besides which, in most projects where you're interested in confidence intervals you'll probably already have both imported for other functionally anyway.

TBH, I haven't used python for science in a few years, so maybe numpy is the norm now and I'm showing my age. But when I was doing more python, I wrote bootstrapping, monte carlo and CI code without anything but the standard lib. I probably used pypy to get it fast enough, but if everyone has numpy now then that's definitely the way to go and I retract my comment!

I'm not trying to sound arrogant or anything, if numpy is the standard now then there's definitely no point in reinventing the wheel. (But pandas is still overkill...)

pandas is the standard now for most python data munging I've come across. numpy is "low level".

bootstrapping does not require any munging.

I agree with you! Pandas is only used in the power analysis code (which also has matplotlib for plotting). The best thing would be to pair this down. We would gladly take contributions - i think the path forward on this feedback is clear but will take a little code =)

Most of the important stuff is just numpy which i feel is pretty fair for most peeps.

Accumulating percentile data isn't a very difficult technique. I did the same thing on arduino using nesC so it is surely doable without pandas or numpy.

Completely agree it is easy. Doing it quickly (for Python) is what this is optimized for. We would love a contribution if you have a method for resampling + percentiles that beats numpy.

I will surely give it a look if I have a need for this. Until then, thanks for the contribution and the invitation, and I have nothing against numpy.

Thanks for the feedback - happy hacking =)

I would assume that it is likely that anyone doing statistical analysis with python would have numpy and pandas installed prior. Has your experience indicated otherwise?

Pretty much any stats work I do in python needs numpy. I don't use pandas as often, but that's because I developed numpy based habits before pandas was really mature, and this is a failing on my part.

"pands and numpy are imported by default" is a pretty safe assumption in this space, IMO

Thanks for the feedback!

numpy is used to give a speed improvement when generating the bootstrap samples - this would be very slow in a Python for loop.

Pandas is only used in the power analysis code. Ill make that more clear.

Would love more feedback if you have it!

numpy is pretty much a given for any scientific or numeric python code unless you want to start writing things in Cython.

Not a disagreement, but AFAIK NumPy and Cython are not mutually exclusive, as Cython supports NumPy type annotations and can index into its arrays quickly.

Reply on news.ycombinator.com