physicsyogi

When I was in grad school, we used MH to compute 400-dimensional integrals. We computed ground state and excited state properties of a particular system, using a wavefunction as the probability distribution. This was easily parallelized.

e.g http://phya.snu.ac.kr/~gsjeon/Primary/Abstract/abs26.html

While I can't say that one guess communicated with others, I can say that whenever I moved one particle, the others knew about it immediately because the wavefunction described a strongly correlated system. Communication between guesses sounds really interesting though. I've been out of the game for a while, so I'll have to look that up.

I don't have any experience with Haskell, for data science or otherwise, but have been using Clojure a bit for such purposes.

The code I write is mainly for machine learning and natural language processing with "big data". Some of the libraries that I've found useful are:

1) clojure.core.reducers 2) core.match 3) incanter (lots of statistics-related stuff. Comparable to R, perhaps.)

Clojure handles XML, JSON, and YAML very nicely as well. Then you have Cascalog to run map reduce jobs without writing mappers and reducers explicitly. There's also Marceline, which is to Storm/Trident as Cascalog is to Hadoop/Cascading.

There are also libraries for serialization like data.fressian, matrix math like core.matrix. Prismatic had also open sourced three really excellent libraries that are useful for data processing.

True. But, it seems many VCs won't consider you if you're not in the valley.

A recent Forbes article talks about the data that the NSA is sharing with the DEA and IRS. The DEA and IRS are told to lie about where they get the data. This seems no different than when evidence in a criminal case is thrown out if police obtained it through an illegal means. The article calls for a special prosecutor.

http://www.forbes.com/sites/jennifergranick/2013/08/14/nsa-d...

There is some truth in this article. But, I know many physics PhDs, myself included, that had no real trouble getting jobs outside of academia or industry research. Of course, most of us are also programmers and now do R&D related to simulations, machine learning, etc, because our research prepared us for that.