Note: Below, borrowing from D. Knuth's TeX, we use the underscore character '_' to denote the start of a subscript.
Details: For real valued random variable X, probability measure P, and the set of real numbers R, the cumulative distribution of X is the function F_X: R --> R where, for x in R, F_X(x) = P(X <= x).
If F_X is differentiable, then the probability density distribution of X is the real valued function of a real variable f_X: R --> R where, for all x in R, f_X(x) = d/dx F_X(x) where d/dx is the calculus derivative.
For the connections with sigma algebras, that is more advanced than most engineers care about, but here are some of the details:
For real numbers a and b with a < b, there is the open interval
(a,b) = {x|a < x < b}
A topology on R is a collection of subsets regarded as open and that satisfy the axioms for a topology -- the sets in a topology are closed under finite intersections and arbitrary unions and both R and the empty set are open. The usual topology on R is the smallest (a short argument shows that this "smallest" is well defined) topology that has each open interval an element of the topology -- right, the topology regards the open intervals as open.
The usual reason to discuss a topology is to have a means of defining continuous functions, a means more general than from the usual "for each epsilon greater than zero, there exists a delta greater than zero such that ..." or in terms of limits of sequences. Indeed, there are advanced situations where we can use topologies to define continuous functions where epsilon and delta and where converging sequences don't work. If curious, look up Moore-Smith convergence, nets, and filters or just Kelley, General Topology.
Well, a sigma algebra is like a topology, that is, is a collection of subsets: A sigma algebra is closed under countable unions and relative complements. Right, we avoid uncountable unions because otherwise we will get stuck in a big mud hole. It is an early exercise that there are no countably infinite sigma algebras.
The reason for sigma algebras is to permit defining a measurable function, that is, one where we can apply the Lebesgue integration theory. The integral of calculus is due to B. Riemann and is the Riemann integral. W. Rudin, Principles of Mathematical Analysis shows that for a continuous real valued function with domain a compact set (closed and bounded, where closed is the complement of an open set) has a Riemann integral. Well in this case, the Lebesgue integral gives the same numerical answer -- same thing. The advantage of the Lebesgue approach is that the function can be even bizarre and its domain can be much more general. Indeed, in probability theory, expectation is just the Lebesgue integral. In simple terms, Riemann partitioned on the X axis, and Lebesgue partitioned on the Y axis.
Well, given the usual topology on R, we can ask for the smallest sigma algebra on R that has the topology as a subset. That sigma algebra is the Borel sets of R. Uh, Lebesgue was a student of E. Borel. In Rudin will find the Heine-Borel theorem.
So, in probability theory, we have a sample space. Each point in the sample space is a trial, i.e., essentially a real world experimental trial (note: really our attitude is that in all the universe we see only one such trial -- if this seems far out, then blame the Russians, e.g., A. Kolmogorov, E. Dynkin, etc.!). Well, an event is a subset of the sample space, that is, a set of trials. So, flip a coin. Let H be the event, the set of all trials where, that the coin comes up heads.
Well, to apply Lebesgue's theory of integration, we want the set of all events to be a sigma algebra.
Then a probability measure is a measure in the sense of Lebesgue's measure theory, that is, a real value function, in the case of probability taking values in [0,1], and with domain the sigma algebra of events. So, for the event H, we can ask for the probability of H, that is, P(H), which is a number in [0,1]. For a fair coin tossed by an honest member of the FBI we have P(H) = 1/2.
Then a real valued random variable X is just a real valued function with domain the sample space and also measurable: This part about being measurable is that for each Borel set A, a subset of R, the set of all trials w so that X(w) is in A is an event, that is, an element of the sigma algebra on the set of trials. That is, the inverse image under X of the Borel sets are events, elements of the sigma algebra on the sample space.
So, with X measurable in this way, we have a near perfect shot at defining the expectation of X, E[X]. For this we have a little two step dance:
First we look at X^+ ('^' denotes a superscript) where X is >= 0. So, X^+ is the positive part of X. Similarly X^- is the negative part of X. So, X = X^+ + X^-. Uh, I'm working quickly from memory; maybe we want X^- to be -X where X < 0 and 0 otherwise. So both X^+ and X^- are >= 0 and we have X = X*+ - X^-. Either way.
Well, we can use Lebesgue's theory to integrate X^+ and X^-. Biggie stuff: The X need only be measurable, and that admits lots of really wildly bizarre functions. We've got great generality, and that's good to have in various limiting arguments. Uh, we like limiting arguments because that is our main way to approximate which our main way to being healthy, wealthy, and wise!
So the Lebesgue integral of X^+ we write as E[X^+]. Similarly for X^-. Now no way do we want to be subtracting one infinity from another since permitting that would trash the usual laws of arithmetic.
So, for our second step, in the case X^- >= 0, if at least one of E[X^+] and E[X^-] is finite, then we define E[X] = E[X^+] - E[X^-].
Now we've defined expectation ("average") of a real random variable X. Our definition is just the Lebesgue integral. For the Lebesgue integral, we wanted the sigma algebras.
On the real line we can consider the sigma algebra of Lebesgue measurable sets; that's larger than the Borel sets. Then we just ask, assume, assert, believe, ..., that our random variables are measurable with respect to the sigma algebra of Lebesgue sets and the sigma algebra of the events. Uh, right, Lebesgue measure on R assigns Lebesgue measure b - a to interval (a,b) and extends from there. Fine details are in various texts by Rudin, Royden, etc.
That's the beginnings of the role of sigma algebras in advanced approaches to probability, statistics, and stochastic processes. It turns out, the sigma algebra approach is for several parts of what we want in probability, much nicer, e.g., for defining independence and conditional expectation. E.g., if we want to know that some set of uncountably infinitely many random variables are independent, we can. Same for conditioning on uncountably infinitely many random variables, e.g., the past history of a stochastic process.