Hacker News Reader

Comment by ethn | Hacker News Reader

50% is a bit better than you think. If you were to do a coin toss on each as a basis of determination you would only have a 25% chance of getting 2 correct. As n-schemes becomes larger, the coin toss does much worse in respect to the Bayesian probability.

View on HN

Chance of tossing 2/4 in a 4 coin toss is 37.5%, no?

0/4: 6.25%, 1/4: 25%, 2/4: 37.5%, 3/4: 25%, 4/4: 6.25%

It depends on if you mean "what's the probability of a coin landing heads" or if you mean "what's the probability a coin tossed 4 times will land heads 2 times?" The first is 1/2 = 2/4.

Yes I was trying to estimate while running errands. Thank you for working it out.

But they were detecting 2 out of 4, not 2 out of 2. I think you are incorrect.

I think they're both incorrect.

They ran the model on historical data. Of 4 a priori known P&Ds, the model detected 2 of them. To say that they've got accuracy of a 50% or "no better than a coin toss", ignores all of the non-P&D events that it correctly didn't identify as a P&D. If you did a coin toss, you would flip the coin much more than 4 times.

You really would also want to have some sort of loss function, or at the very least a general idea of whether you most need to avoid false positives or false negatives. If it was most important to avoid incorrectly saying it's NOT a P&D, then this is not good performance. If you need it to nearly always avoid saying something which is not a P&D (and thus might be a great investment opportunity), but given that you want to avoid as many as you can, 50% might be good.

True

Reply on news.ycombinator.com