What I'm asking about, I know I'm wrong about - I just want to know how I can derive that for myself.
What I'm asking about, I know I'm wrong about - I just want to know how I can derive that for myself.
That skips over the distinction between "average" and "probability distribution", but those are nuances are probably better left for a proof of the central limit theorem.
The Law of Large Numbers is about the average, so it's not relevant here (an average of +1 would mean you got heads every single time, which is extremely unlikely for large n).
If you are looking at the sum, then the value depends on whether the number of trials (n) is even or odd. If n is odd, you would indeed get two peaks at 1 and -1, and you would never get exactly 0. If n is even, you would get a peak at 0 and you would never get exactly 1 or -1.
The expected value (aka average) is a number, not a distribution. The expected value for the sum is 0 even when n is odd and you can't get exactly 0 -- that's just how the expected value works (in the same way that the "expected value" for the number of children in a family can be 2.5 even though a family can't have half a child). If you look at the probability density function for a single trial, then it does have two peaks at 1 and -1 (and is zero everywhere else).
The curve you refer to might be the normal approximation (https://en.wikipedia.org/wiki/Binomial_distribution#Normal_a...). It's true that the normal approximation for the distribution of the sum in your gambling game has a peak at 0 even when n is odd and the sum can't be exactly 0. That's because the normal approximation is a continuous approximation and it doesn't capture the discrete nature of the underlying distribution.
However to answer your question more directly, the expected value is just another name for the average or mean of a random variable. In this case, the variable is your profit. Assume we’re betting a dollar per toss on coin flips and I win if it’s heads (everyone knows heads always wins, right?). The expected value is probability of heads * 1 - probability of tails * 1. If the coin is fair, the probabilities are the same so the expected value is zero.
Aside: sequences of random variables that are “fair bets” are called martingales and are incredibly useful. It’s a fair bet because, given all prior knowledge of the value of the variable thus far, the expected value of the next value you witness is the current value of the variable. You could imagine looking at a history of stock values. Given all that information, it’s a martingale (and thus a fair bet) if given that information your expected profit from investing is 0.
See this https://stats.stackexchange.com/questions/416204/why-is-a-mi...