Think Bayes - Bayesian Statistics Made Simple
greenteapress.com
greenteapress.com
The Bayesian book I want should emphasize how Bayes is a recipe for studying complex problems and teach a broad range of model ingredients. Learning Bayesian statistics is about becoming fluent in describing scientific problems in probabilistic language. This requires knowing how to express and compose traditional models and build new ones based on first principles.
An unfortunate reality is that you still need to know computational methods too, but that should change soon enough.
Now, I can only personally vouch for the first 2 chapters, as I haven't read the rest yet.
* edit: Doh, no Kindle version. I don't mind paying $90+ for a good book though, just like it to be electronic: http://www.amazon.com/Probability-Theory-The-Logic-Science/d...
http://bayes.wustl.edu/etj/prob/book.pdf
It's always nice to see good things come out of Wash U. (Alum here.)
EDIT: In case anyone wants to make me feel bad about pirating, Jaynes is dead, and besides that, I bought a hardcopy as backup.
http://www.naturalthinker.net/trl/texts/Science/Jaynes,%20E....
The first couple pages are a bit funny looking, but after that, there are all 500+ pages. It was the fourth result on Google for me.
Personally, I searched out a PDF and based on what I've read so far, I'm itching to pull the trigger on Amazon as I'm simply loving what I'm reading.
As the book comes along, I am finding that many ideas that are hard to explain and understand mathematically can be very easy to express computationally, especially using discrete approximations to continuous distributions.
For example, I just posted a section on ABC
http://www.greenteapress.com/thinkbayes/html/thinkbayes008.h...
that (I think) really demonstrates the strength of this approach.
Of course, my premise only applies for people who are as comfortable with programming as with math, or more so.
Definitely shoot me an email at tristan@senseplatform.com if you're interested in the computational side of this area. At Sense (http://www.senseplatform.com), we're working on making applied Bayesian analysis as amazing as it should be.
The way he explains it I found to be confusing and counter-intuitive. I've taken basic stats in college, and learned some of the associated problems, though not this one, and learned the material though not in this particular way. I have to agree whole-heartedly with the commenter on that post "JeffJo" who stipulates why it's an ineffectual way to present the material. Furthermore, I found the author's dismissal of the valid criticism to be enough to not want me to read further.
I feel pretty comfortable with Bayesian statistics, and I thought the other examples that I saw were pretty clear. But his very first example jumps you out to another webpage, and then he mixes it with "the red-haired problem". It was irritating.
His next example, "The Cookie Problem" is the classic intro-to-Bayes example, IMO.
Reading the Florida problem solution, it made some sense, but was definitely of a higher level of complexity than the rest of the text.
What I found really interesting was that the answers to some of the other questions on the "Girl Named Florida" discussion required knowledge which I would not have considered general math-ish knowledge:
> If the parents have brown hair > and one of their children has red hair, > we know that both parents are heterozygous, > so their chance of having a red-haired girl is 1/8.
Interesting to learn, but this "if you also happen to know this ..." step is something that was mildly frustrating.
(edit: Since the grandparent post linked the Girl Named Florida blog post, I guess I don't need to.)
This kind of feedback is exactly why I like to post drafts early. Expect this example to magically disappear very soon :)
I re-read the chapter and decided that example was doing nothing except confusing half the audience and antagonizing the other half.
I want to addresss on possible source
should be
I want to addresss one possible source
(on -> one)
Peace.
Ambiguous: "In a family with two children, what are the chances, if one of the children is a girl named Florida, that both children are girls?"
More clear, and emphasizing the importance of precise wording when discussing probability: "Among families with two children, with at least one of the children being a girl named Florida, what portion have two girls? (Assume that all names are chosen randomly from the same distribution, independently of all other factors; and sex is determined as by a fair coin toss.)"
Say you are on a game show, and pick Door #1. The host opens door #3 to show that it does not have the prize, and offers to let you switch to door #2. Should you? Most people will initially reason that door #3 is prize-less 2/3 of the time, evenly split between cases where the prize is behind door #1 and door #2. So it would be pointless to switch. But that is wrong. Few educators will explain why by solving the problem rigorously. They will use an analogy like pointing out how the original choice is right only 1/3 of the time, and since the host can always open a prize-less door, that can’t change.
People don’t believe these educators because their 1/2 answer is indeed more rigorous than the analogy. It just makes a mistake. The probabilities to use are not the probabilities that the cases exist, but the probabilities that the observed result would occur. The existence probabilities are the same, but the probability of the observed result when the initial door was correct is half of what it is when the initial choice was incorrect.
Yes, these sorts of problems can be confusing. But the confusion is propagated by educators who refuse to recognize that what they asked is not what they intended to ask, and so they provide inconsistent answers.
Say you are on a game show, and pick Door #1. The host opens door #3 to show that it does not have the prize, and offers to let you switch to door #2. Should you? Most people will initially reason that door #3 is prize-less 2/3 of the time, evenly split between cases where the prize is behind door #1 and door #2. So it would be pointless to switch. But that is wrong. Few educators will explain why by solving the problem rigorously. They will use an analogy like pointing out how the original choice is right only 1/3 of the time, and since the host can always open a prize-less door, that can’t change.
People don’t believe these educators because their 1/2 answer is indeed more rigorous than the analogy. It just makes a mistake. The probabilities to use are not the probabilities that the cases exist, but the probabilities that the observed result would occur. The existence probabilities are the same, but the probability of the observed result when the initial door was correct is half of what it is when the initial choice was incorrect.
1. Think Bayes
2. Think Stats
3. Programming Collective Intelligence by T.Segaran
3: Get (back) into the swing of thinking about mathematics and algorithms.
1: Bayesian statistics is a principled, coherent, consistent, intuitive, complete framework for reasoning about uncertainty. A good foundation.
2. Traditional statistics is more random and ad-hoc, but can be more practical than Bayesian methods. (Bayesian models are well-motivated, but it can be impractical to compute exact answers and you'll have to switch to approximation techniques, some of which are simple/universal/slow, and others get fairly complex.)
So I would recommend
1. Think Python 2. Think Stats 3. Think Bayes
Let's say we have N different kinds of events with unknown probabilities and unknown dependence or independence between them. The naive approach to gathering data on the probability of event n occurring following an occurrence of event m would require O(N²) in space. Let's say N ~ 10⁹~10¹⁰. Storing that much data as a raw matrix isn't practical in most cases, so we have to find a more efficient data structure - in terms of both space and the operations we need to perform. (and taking into account characteristics of the storage medium, i.e. memory or disk or a combination) What happens if the probabilistic properties of the system change over time?
Are there any introductory books or other resources on modeling this kind of problem? Clearly this has been tackled before, but I'm having a hard time making the leap from theory to practice - and I don't mean import the data into R or SPSS or whatever and let that grind out a solution, but coming up with approximations when you have runtime and space constraints that make that approach impractical.
Do you actually have a stream of more than N^2 observations to process? If not, then most of your correlations are in fact 0, and sparse-matrix techniques apply.
You can send the cheque in the mail. ;)
I'm hoping this will be as good, but all the negative comments here leave me skeptical. Perhaps this is the crowd that would enjoy K&R C more than Think Python. The former is more of a reference to me than an introductory tome. Perhaps everyone here is just better at math than I am.
Just kidding (mostly), but your point is correct: there is no book that is right for all audiences. But if you can program, and the mathematical approach to this material doesn't do it for you, this book might.
Bayesian is not cool because estimating the parameters takes bloody ages on a supercomputer, unless you spend ages being really careful to specify your model.
Frequentist statistics is cool because it is a massive big bag of tricks to estimate all sorts of stuff, and pretty much all of the tricks are already in R.
Frequentist statistics is not so cool because calculating all the specific things you want to can be a pain in the ass.
Once either Quantum computers kick in or a better algorithm than MCMC for Bayesian is created, Bayesian will win.
There are some philosophical arguments about the objectivity of the prior in Bayesian statistics, but these wash out in a decision theoretic framework because of the subjectivity of the utility function at the other end of the process.
Also, less than 5% of people reporting p-values really know what a p-value is.
Frequestist statistics gives you an objective answer to a question that has the same words as the one you asked, but arranged differently.
I would actually recommend the opposite - they have ASCII versions of the symbols that e.g. Chrome might not render correctly, and all I checked looked fine. The PDF meanwhile copies this text from the (not linked) link to the "girl named Florida" article:
❤tt♣✿✴✴❛❧❧❡♥❞♦✇♥❡②✳❜❧♦❣s♣♦t✳❝♦♠✴✷✵✶✶✴✶✶✴❣✐r❧✲♥❛♠❡❞✲❢❧♦r✐❞❛✲s♦❧✉t✐♦♥s✳❤t♠❧
I have recommended think stats to many people, and it appears to be somewhat of a success. And its free documentation, which is wonderful.