The homogenization of scientific computing (2013) | Hacker News Reader

The homogenization of scientific computing (2013)

talyarkoni.org

32 pointsehudla14 comments

The astronomy department I worked at briefly during my undergrad underwent a similar process. The ten or so grad students separately used Matlab, Mathematica, C, IDL, and a couple of the newer members used Python. They spent some time on the side working on the AstroPy package and all started transitioning to Python as their needed functions and data processing pipelines were implemented.

I'm in nuclear engineering now, I'm seeing a similar process. It used to be Matlab for everything but once the Anaconda Python distro came out and made installing python and the science libraries a simple process many undergrads are now using Python instead. The Spyder editor with the documentation window makes it really easy for undergrads to lookup functions, similar to the Matlab editor.

My SO is teaching physics in small university. Used to use Excel, Matlab, some other statistics packages. Now it is Anaconda all the way down.

I wonder why nothing as good as RStudio seems to be available for Python yet? I feel like Python is stuck in Jupyter rutt - a local minima (or maxima, depending on your perspective) where Jupyter is capturing all the interest but is fundamentally not as good for data exploration (just my opinion, from using both intensively over years). I often want to use Python but I keep going back to RStudio because I find Jupyter painful (even after enhancing with BeakerX [1] which I love for what it is, but it can't fix the underlying constraints of Jupyter ...).

[1] http://beakerx.com/

RStudio? Really? What do you like about it? I see various advantages R has, but RStudio never appealed.

As for Jupyter (which I love), there's nteract [1], which Netflix seems to be supporting [2].

[1] https://github.com/nteract/nteract

[2] https://news.ycombinator.com/item?id=17795026

No mention of Jupyter notebooks — I think that's a critical part of this change, in that it provides a sharable human-readable analysis format.

Also for the original poster (whose psycholinguistics work I am familiar with), it's easy to move data between R and Python using the rpy packages (which works in notebooks!)

Does scientific computing happen in notebooks? I'd think they mean Supercomputers, not experiments running in the browser.

Yes, definitely. Only the GUI for Jupiter is in the browser — the computation is on a workstation, server, or an iPython parallel cluster, so you can use proper HPC resources. Another common pattern (because socket-based communication can die) is to run the heavy HPC stuff with SLURM, etc. then compile and analyze the results in the notebook.

I remember the great joy it was rewrtot some MATLAB and Bash scripts for neuroimaging to Python, and how fast it let us iterate. Thanks to that rewriting we ended up having a proper pipeline that we were able to parallelize.

Since then Python is my go-to language, but that didn't stopped me from exploring other languages like Julia, for large numerical analysis or Go.

I see that Pythonification he talks about around me, and it's true that is becoming ubiquitous and the defacto langugae on sciences, specially on life sciences.

Discussed in 2014: https://news.ycombinator.com/item?id=7729006 and https://news.ycombinator.com/item?id=7030097

2013: https://news.ycombinator.com/item?id=6756430

I suspect Julia will eventually rise to the occasion.

As someone who finds Python more difficult to work with than Julia, I sincerely hope so.

I haven't used it, but can Julia replace Python as a production language for dual use by software devs and data scientists?

This really needs a [2013] in the title.

Added - thanks.

Reply on news.ycombinator.com