johnmyleswhite

I think MCMC.jl is pretty immature compared with Stan. But there is work on a Stan wrapper. Personally, I mostly write MCMC code by hand these days.

Personally, I don't see Julia replacing R for a while. As you note, the R community has much of what it needs already. When I initially switched from R to Julia, it was because R wasn't really usable for the kind of work I do. It's since become clear to me that I work on problems that don't come up for most R users: problems involving large sparse matrices, including large-scale optimization and MCMC. If you don't deal with those kinds of things, R is a good choice.

No, it is not.

DataFrames are in-memory. It sounds like you're describing what we're calling DataStreams, which are still a work in progress, but do already exist. And we're also building SGD descent for doing things like OLS incrementally.

I have a friend who was an intern with Siemens in computer vision while preparing for Ph.D. programs in machine learning. I would guess that other companies that build medical imaging tools have similar divisions.

If your math background is weak enough that you don't feel totally at home using calculus and linear algebra, I'd recommend "Principles of Data Mining", which they use for the intro to ML class at Princeton. The ratio of English to equations is pretty high, which might help you to get into the swing of things.

If you think that you can learn enough math to follow the material, you should watch the Stanford lectures given by Andrew Ng. They're available freely online through iTunes U. Andrew is also one of the most respected experts in the field. As he points out, you will eventually need to understand the math behind the methods. Otherwise, a day will come when you start wasting time coding algorithms that plainly don't apply to the problem at hand.

This is a fun analysis to see. Here's a suggestion for cleaning up the stats, though it won't help you get around people noting that there's a selection bias for the poll results themselves.

If you're interested in showing that the frequency distribution for personality types is different on Hacker News than in the general population, then the right way to show this would be to run a single multinomial test to see whether your empirical multinomial distribution is outside of the range of chance variation from the canonical distribution listed on Wikipedia for the MBTI. See http://en.wikipedia.org/wiki/Multinomial_test for a description of this hypothesis test.

To give you a sense of why your multiple comparisons across binomial tests is problematic, imagine that the INTP group is underrepresented because all of the people who would test as INTP tested as ENTP this time around; in this case, you run two different binomial tests, but you really only have one result, because the size divergence in one group is effectively the cause of the size divergence in the other group.

I hope that makes it clear why you should really treat your data set as containing an N of 1 and doing a simple null hypothesis test of the probability of getting your results given the canonical values for the global multinomial distribution, rather than testing based on a series of binomial marginal distributions.

It sounds like you're interested in statistics from the words "a few interesting stats-heavy compsci papers". The MacKay book, "Information Theory, Inference and Learning Algorithms," already referenced by benm is quite good, but I'd suggest on top of it Larry Wasserman's amazing book, "All of Statistics". If the level of calculus and linear calculus in that book is too much for you, you might need to review that material first before moving on to anything sophisticated in statistics.

In the interim, you could start with David Freedman's "Statistics" (written along with Pisani and Purves) to at least get a feel for the basic ideas of statistics from one of the great statistical writers of the 20th century.

If there are other things you want to learn about, I'd be happy to make suggestions, as I went in the opposite direction as you're trying to go now, having been a math major as an undergrad who picked up CS along the way. For cryptography, for instance, you'd need to learn number theory and then abstract algebra, but those things will be of little use to you in other contexts.