Hacker News Reader

Comment by H8crilA | Hacker News Reader

What kind of operation happens when you pointwise multiply two probability density functions? It wouldn't even integrate to 1.0 any more (it can even integrate to 0.0, like in the example 4). Rather than taking the convolution, which corresponds to adding two independent random variables that generate said density functions.

View on HN

Pointwise multiplication of, say, two density functions gives the likelihood function of two independent random variables with these densities. This function is not a density function, as you say, but is still an important object, as it is used for inference based on the likelihood principle.

I'm sorry, but I'm lost, are we talking about this likelihood function? I'm guessing no, since that's an object defined in a statistical space, not in a probabilistic space:

https://en.m.wikipedia.org/wiki/Likelihood_function

Oftentimes, a “likelihood function” is generalized from that definition to refer to any non-normalized function which is still treated like a probability distribution. Any function with a finite integral over the real line can be turned into a probability distribution easily, and there are other functions that can be used in places where you would normally use a probability function, even when they don’t integrate to a finite value over the real line. (For example, improper priors in Bayesian analysis.)

“Function that can in some contexts be treated like a probability distribution” is just too long.

Yes, that's the right likelihood function. To get probabilities distributions out of the products in the post, all you have to do is normalize them. That would be equivalent to applying bayes rule and combining independent evidence about a parameter with an uninformative prior. Normalizing doesn't change the shapes, so I left out the details.

But then the term would be P(x1) P(x2) and the correct summation/integration to use would be over the cartesian product of the sample space. Under such summation/integration its a valid density

Joint probability of independent events, especially the common case of independent, identically distributed data. (Or exchangeable observations, in the Bayesian framework).

For example, flipping a coin n times, observing heights of people in a population, etc.

But we're talking about "continuous variables", random variables with infinite support for which P(X=a)=0, for all a.

It's kind of like just a single point in the convolution for "X_1 - X_2", corresponding to "X_1 - X_2 = 0" ?

I'm not sure I understand what you are trying to convey.

If we have two continuous random variables in a probability space, we have a mapping P(X, Y): R^2 --> R, such that the integral sums to 1.

If X and Y are independent, then P(X, Y) == P(X)P(Y). The examples in the linked article are a bit confusing because some of them have P(X) and P(Y) as different distributions. But the plot only shows the plot of the 2D domain along the 1D line of X==Y, which isn't a terribly common occurrence in actual practice.

P(X,Y), when integrated along all possible diagonals given by x+y=z where z goes through all real numbers, is what a convolution is (of P(X) and P(Y) generated by those variables). This gives you the distribution of the variable Z=X+Y.

So, we're talking about the same thing, namely that multiplying two probability density functions pointwise roughly corresponds to a single point where X==Y (the same as X-Y==0), but otherwise doesn't seem terribly useful.

Wouldn't you multiply like you do for functions as an inner product for hilbert space?

Something like: $ f \cdot g = \int_\infty^\infty f(x)g(x)dx $

No?

The schwartz inequality gives us that the product then has a value between 0 and 1. ...Not sure what it means if the value is less than one. What does orthogonality mean when it comes to probability density functions?

Usually orthogonal means uncorrelated random variables (not necessarily independent). A good example is sine and cosine - they are uncorrelated in the interval 0 to pi. But clearly they are not independent.

Sine and cosine can be negative. Density functions cannot, and thus, orthogonality of the density functions implies that they don't have the same support with probability 1, or, said otherwise, the measure of the intersection of their support is 0.

Scalar product doesnt give you a function/density whatever, just a number (which can be important, but this doesnt adress the question)

You might be interested in the idea of a copula and Sklar's theorem

https://en.wikipedia.org/wiki/Copula_(probability_theory)#Sk...

Reply on news.ycombinator.com