Do not confuse a random variable with its distribution
bookdown.org
bookdown.org
Hmm, that sentence at the beginning is already wrong. Random variables can measure anything, not just numbers. Heads or Tails of a coin, or colours of cars etc.
It's fine to restrict yourself to numeric random variables only. But if you are writing a rant telling other people to be more careful in their analysis, you better dot your i's and cross your t's yourself.
If it maps elsewhere, mathematicians like to call it a random element instead.
I can understand why some might make a cut between pure numbers and other objects, but it's not something that troubles many.
I can see why you would teach the more restricted definition eg in high school.
Definition 3. A random variable X is a real function which is measurable with respect to the underlying sigma-algebra. The function F defined by F(T) = P{X < T} us called the distribution function of X.
Though as kazinator demonstrates in this thread, it looks like this approach can lead to some confusion between random variables and expected values.
For a random variable consisting of a unit vector, it may be possible to construct an Expected Value: take all the vectors that can occur, multiply them by their probability and add together. (Or if the vectors are not quantized, use integration over their space).
We can't do that for bead colors that have not been mapped to values.
That looks like a good litmus test: if we can calculate a meaningful Expected Value, it is almost certain a random variable. Otherwise not.
On the other hand, it's possible to construct probability distributions that have a meaningful expected values, but don't have anything to do with numbers.
> It is not like colors of beads in a jar that have no value until we assign one, which we do arbitrarily.
Huh? 'Red' and 'blue' are a perfectly fine values as far as I am concerned.
A random unit vector can perhaps be represented by a bunch of numbers (assuming you fix some finite dimension), but is not by itself a numerical quantity. Eg you can't multiply vectors.
It's also not completely clear that the definition of expected value of unit vectors you gave is necessarily the right one for every context: with your definition the expected value of a distribution over unit vectors is (in general) not a unit vector itself.
Often it's convenient to rig things in such a way that the expected value is a thing of the same type.
If you allow the expected value to be of a different type, you can say that the expected value of drawing from our urn is the expression `30% * blue + 70% * red` with `*` and `+` being purely formal operators from the free ring.
---
In any case, expected values and random variables are separate subjects.
Right; that needs to be better articulated. For the Cauchy distribution, we can meaningfully explore whether it has an expected value. We can write down the expression.
If it is not absurd to explore whether the distribution has an expected value (even though it may turn out that it isn't numerically defined), the distribution is of a random variable.
It is absurd to think about what is the expected value of a random experiment that produces the words "red", "green" and "blue" with various probabilities.
It doesn't exist as a category, not due to a calculation problem. I.e. it's "not even undefined".
> Eg you can't multiply vectors.
You can multiply vectors together in 2D (complex numbers) and 3D (cross product). Also 4D (quaternions, non-commutatively).
You can multiply vectors in any number of dimensions by scalars, which is all we need for averaging a bunch of vectors, with weights.
But yes, vector fields are not fields though, so vectors are not numbers in every sense.
Thanks for talking yourself into agreeing with my point:
Even for an expected value to make sense, you don't need a 'number'; ie you don't need a field, and you don't even need a ring. You can use a less restricted structure.
> It is absurd to think about what is the expected value of a random experiment that produces the words "red", "green" and "blue" with various probabilities.
Why is it absurd? It's perfectly possible to define the result over a suitable 'free' structure. (In fact you can always do that, even for 'numbers' and then later collapse that free structure into something concrete.)
Btw, it's perfectly possible to define some weighted average of colours, if you wanted to. But that's about as relevant as the different not-quite-multiplications you brought up.
> It doesn't exist as a category, not due to a calculation problem. I.e. it's "not even undefined".
Free algebras are perfectly well studied structures in math. They 'exist' just as much as anything else in math does. And, by definition, they have all the right properties we need to define the expected value.
---
> You can multiply vectors together in 2D (complex numbers) and 3D (cross product). Also 4D (quaternions, non-commutatively).
Those operations are often called 'multiplication', just like we often call any random group operation 'multiplication'. But there's no vector multiplication you can define in general (for all number of dimensions) that would give you a field or even just a ring. So they aren't really the kind of multiplication we need.)
I really appreciate comments like this.
When you're asking P(Heads) of a coin, its short hand for "P(event=heads) for all possible events that can happen when flipping a coin". You notice that in this idealized world the event is always two things (heads or tails), and never null, side, cow because we have defined that the P(event=cow) = 0.
Probability functions are at the end of the day models that we can use to analyze real world non-deterministic phenomena with the rigor of mathematics.
[0] - with credit to Prof Gubner https://directory.engr.wisc.edu/ece/Faculty/Gubner_John/
They are functions that map from the space of random events to whatever other space you want. Why restrict yourself to the real number line?
Eg the 'moral character of the next stranger I encounter' is a random variable that maps from some abstract space of random events (we often used the Greek letter Omega for that) onto some space of moral characters.
And if you want to restrict yourself to numbers, why restrict yourself to reals? Why not complex numbers or p-adic numbers or numbers in F_5 or numbers in [0, 1] or 'natural numbers expect 17'?
EDIT: looks like you are right in English! See https://news.ycombinator.com/item?id=40796146 and my reply there.
Edit: I misread. I agree with you. Sorry!
That said, while I appreciate and admire free textbooks published online, I think the exposition would be much improved if the author had a better sense of who he was writing for (the most common writing advice...).
And I take issue with the view that the sample space is where the random phenomenon lives, as it were. In my experience, it's more common to use the random variable itself to model the (observable aspect of) the random phenomenon, and for the sample space to be either a hidden (i.e., more abstract) aspect of the phenomenon or else a purely abstract formalism introduced only for ease of mathematical computation.
It would be helpful also to see some more context, especially historical (who introduced the concept of a sample space, and for what purpose?).
For a random variable we can calculate something called the expected value. We can't do that if the value of the variable is in the domain { red, green, blue }.
Not all probability distributions over real numbers have expected values. See https://en.wikipedia.org/wiki/Cauchy_distribution
Many probability distributions over something other than the real numbers have expected values.
https://en.wikipedia.org/wiki/Girsanov_theorem https://en.wikipedia.org/wiki/Risk-neutral_measure
The fun part comes when you can ask this computation graph: "what parameters for a random variable early on in the chain would be the ones that optimize some function of variables later in the chain?" And, handwaving a ton of nuance here, when those parameters are weights in a neural network, the function is a loss function on the training data, and the optimization is done by automatic differentiation (e.g. https://pytorch.org/tutorials/beginner/introyt/autogradyt_tu...), you have modern AI.
If you're interested in the theoretical underpinnings here, Bishop's PRML is perhaps the classic starting point: https://www.microsoft.com/en-us/research/uploads/prod/2006/0...
In that article, squaring a number in interval arithmetic is different from multiplying two independent numbers with the same interval. Here, squaring a random variable is different from multiplying two independent random variables with the same distribution.
To clarify what I do understand: so you have the variable, like height, and all its possible values along with their probabilities (that's the distribution, if I understand things right). The distribution represents a big part of what the variable is, although I realise the variable maybe has other attributes too (none comes to mind right now though).
X = 1 with probability 0.5, 0 with probability 0.5 Y = 0 when X = 1, 1 when X = 0 (for the \omega for which X(\omega) = 1, Y(\omega) = 0).
They're both bernoulli distributions with p=0.5 (i.e. they follow the same distribution) and P(X=Y) = 0
- You literally identify sets with their indicators: they are the same.
- You identify the “P” operator as expectation (integration) with respect to the underlying measure, and next…
- You note that integration is linear, so you use linear operator notation everywhere you’d use P.
So if “A” is a set, you just write
P A = 0.5
This is equivalent to:
P A = P 1[ω ∈ A] = ∫ 1[ω ∈ A] dP(ω)
in Lebesgue notation.
There’s an example on page 2 of http://www.stat.yale.edu/~pollard/Courses/241.fall2014/notes... although he’s not using measure theory there.
It can be really clean and terse especially when doing bounds for random variables.
It's just like if X~N(0,1), Y~N(0,1) and you want to know the distribution of X-Y. You need to know what the PDF of (X,Y) looks like. Well, you don't know. X and Y could be correlated or they might not be. e.g. if could be that (X,Y)~N( (0,0), [(1,0),(0,1)] ) or maybe (X,Y)~N( (0,0), [(1,1/2),(1/2,1)] ). The distribution of X-Y cares how correlated X and Y are.
Isn’t it 0.5?
A better convention would make clear the distinction between the set of possible outcomes, the act of obtaining a (range of) samples from that set, and the probability that those events match a value range of interest. p(x=X) is not enough to capture all that information. let alone p(x) vs p(X).
I’m so glad I have ChatGPT now, I always ask for applied examples and ask it to explain things intuitively. I would’ve been a 4.0 student if I would’ve had ChatGPT as my personal tutor when I was in school.
This is simply wrong. Random variables are not defined this way.
> The random variable X is a function mapping the outcome to its probability.
you've stated: "The standard definition is the one given in the two bullet points at the top of the article we are discussing" ie:
> The random variable X itself, that is, the function which maps sample space outcomes to numbers.
you've also stated: "Or, in more detail, at (wikipedia link)" which has:
> A random variable is a measurable function from a sample space as a set of possible outcomes to a measurable space
These all appear to be in rough alignment .. all three agree upon a function mapping from outcomes to measure.
You've described the first as "This is simply wrong. Random variables are not defined this way." Perhaps you can expand on why this is so wrong compared to the other two definitions.
> the function which maps sample space outcomes to numbers,
the "numbers" referred to in the codomain are not probabilities. Similarly, in the Wikipedia definition,
> a measurable function from a sample space... to a measurable space,
the "measurable space" codomain has nothing to do with probability of events. Random variables are related to probability distributions/measures, but they are not the same kind of mathematical object.
As a concrete example, consider gambling on a fair coin flip. We can bet 1 unit of money on Heads and define a random variable for the amount of money we win/lose. First we set up the probability space. The sample space is the 2-element set of coin sides, Ω = {H, T} (Heads and Tails). The event set (aka sigma-algebra) F is the power set of Ω: F = {{}, {H}, {T}, {H, T}}. The probability measure assigns probability 0 to the event {}, probability 0.5 to each of the events {H}, {T}, and probability 1 to the event {H, T}.
Now, we can define the random variable of our winnings by the function X mapping Ω to the real numbers by Χ(ω) = 1 if ω = H, -1 if ω = T.
So, within the setting of the probability space (Ω, F, P), the random variable is defined by the function X, which does not map anything to a probability. It maps coin sides to real numbers, including negative ones!
We have not yet constructed any object that gives the probabilities of the possible values of X. The only thing that "maps an outcome to a probability" in our setup so far is the measure P, but that is defined independently of our numeric random variable. If we want to talk about the probabilities that X takes the values -1 and 1, we are talking about the pushforward measure of P by X. This is another mathematical object, distinct from the random variable X itself.
This is one of the points of the article. Suppose we define another random variable Y for the winnings of the person we are gambling against: Y(ω) = -1 if ω = H, 1 if ω = T. Now X and Y are two different random variables, but they have the same distributions! The pushforward measures of P by X and Y both put probability 0.5 on the measurable sets {-1} and {1}.
Is that a massive assumption that such a thing exists that makes the GP comment egregiously wrong, or more of an impatient jump to the end as such a thing always (?) exists?
Do they?
What about? “The random variable X is a function mapping each outcome to the age of one of my cousins”?
Does that seem like a valid definition of the concept of random variable to you because the age of my cousins are numbers - just like probabilities mentioned by glitchc’s comment are numbers?
"probabilities" and "age of one of my cousins" do not sound at all similiar.
One has the feel of a potentially continuous interval of values that can be mapped | scaled to perhaps match up with other forms of expression, the other is a finite discrete set of probably integers between zero and 120 with total set size likely less than a 100.
The point is that the random variable is about mappings to general numbers.
“The random variable X is a function mapping the outcome to its probability”
makes as much sense as
“The random variable X is a function mapping the outcome to the exponential of the square root of its probability”
Those are valid definitions of two different random variables - not a general definition of “random variable”.