Analogy between variance in statistics and probability theory

V

vclaire

New member
Joined
Aug 3, 2019
Messages
11
Hello,

I know that the formula of variance in probability theory is Var(X) = E(X2) - E2(X)
I was just wondering why is it like that, and i was wondering if i could use the steps of calculation of variance in statistics in probability theory as well. I checked and it seems to work, but I want to know if the analogy is correct.

So my thread of thoughts was this:

1. step mean
STAT: calculate the mean: x̄
PROB: calculate the expected value of X: E(X)

2. step deviances
STAT: deduct the mean from every value: xi - x̄
PROB: X - E(X)

3. squared deviances
STAT: (xi - x̄)2
PROB: (X - E(X))2

4. calculate the mean of the squared deviances
STAT: Summa ((xi - x̄)2) / N where N is sample size (Bessel correction is not important for me right now)
PROB: E( (X - E(X))2 )

5. sort equalation
STAT: done
PROB E( (X - E(X))2 ) = E( X2 - 2*X*E(X) + E2(X) )=
E( X2 ) - E( 2*X*E(X) ) + E( E2(X) )=
E( X2 ) - 2 E2(X) + E2(X) =
E(X2) - E2(X)
Done

So my question is: is the analogy correct, and i can take steps of statistics in prob. theory, or if a mathematitian saw my calculation/analogy, would they roll their eyes and frown at me?

Thank you very much for answering my question!
 
Excellent question!

You can think of expected value as a weighted mean, where the weights are ALL m probabilities.

[MATH]E(X) = \dfrac{\displaystyle \sum_{i=1}^mp_ix_i}{\displaystyle \sum_{i=1}^mp_i} = \dfrac{\displaystyle \sum_{i=1}^mp_ix_i}{1} =\sum_{i=1}^mp_ix_i [/MATH]
Now think about population statistics, not sample statistics. And think about a frequency table with m columns. We can think of the population mean as a weighted mean, where f_i is the number of instances of a particular value.

[MATH]\bar x = \dfrac{\displaystyle \sum_{i=1}^mf_ix_i}{\displaystyle \sum_{i=1}^mf_i} = \sum_{i=1}^m \left ( \dfrac{f_ix_i}{\displaystyle \sum_{i=1}^mf_i} \right ) = \sum_{i=1}^m \dfrac{f_ix_i}{n}.[/MATH]
Under the frequency interpretation of probability, [MATH]\dfrac{f_j}{\displaystyle \sum_{i=1}^m f_i} = p_j.[/MATH]
So they are basically saying the same thing (with respect to populations.)

Now let's think about the expected value of the variance around the expected value.

[MATH]E(X - E(X))^2 = \dfrac{\displaystyle \sum_{i=1}^mp_i(x_i - E(X))^2}{\displaystyle \sum_{i=1}^mp_i} = \dfrac{\displaystyle \left ( \sum_{i=1}^mp_ix_i^2 \right ) - \left (\sum_{i=1}^m2E(X)p_ix_i \right ) + \left ( \sum_{i=1}^mp_iE^2(X) \right )}{1} =[/MATH]
[MATH]\left (\sum_{i=1}^mp_ix_i^2 \right ) - 2E(X) * \left ( \sum_{j=1}^mp_ix_i \right ) + \left (E^2(X) * \sum_{i=1}^mp_i \right ) =[/MATH]
[MATH]E(X^2) - (2E(X) * E(X)) + (E^2(X) * 1) = E(X^2) - 2E^2(X) + E^2(X) = E(X^2) - E^2(X).[/MATH]
Again using a frequency table approach, let's compute the population variance around the mean. The algebra turns out to be the same.

[MATH]\dfrac{\displaystyle \sum_{i=1}^m f_i(x_i - \bar x)^2}{\displaystyle \sum_{i=1}^m f_i} = \dfrac{\displaystyle \left ( \sum_{i=1}^m f_ix_i^2 \right ) - \left (\sum_{i=1}^m 2 \bar xf_ix_i \right ) + \left ( \sum_{i=1}^m f_i (\bar x)^2 \right)}{n} =[/MATH]
[MATH]\dfrac{\displaystyle \sum_{i=1}^m f_ix_i^2 }{n} - \dfrac{\displaystyle 2 \bar x * \left ( \sum_{i=1}^m f_i x_i\right )}{n} + \dfrac{ \displaystyle (\bar x)^2 * \left ( \sum_{i=1}^m f_i \right )}{n} =[/MATH]
[MATH]\text{MEAN}(X^2) - 2 \bar x * \dfrac{\displaystyle \left ( \sum_{i=1}^m f_i x_i\right )}{n} + (\bar x)^2 *\dfrac{ \displaystyle \left ( \sum_{i=1}^m f_i \right )}{n} =[/MATH]
[MATH]\text{MEAN}(X^2) - 2 \bar x * \bar x + (\bar x)^2 * \dfrac{n}{n} = ]text{MEAN}(X^2) - 2(\bar x)^2 + (\bar x)^2 =[/MATH]
[MATH]\text {MEAN OF THE SQUARES - THE SQUARE OF THE MEAN.}[/MATH]
It is absolutely parallel, and is consistent with the frequency interpretation of probability so long as you are talking about population statistics..
 
I have one more question.

When you started calculating the variance:
"Now let's think about the expected value of the variance around the expected value.
E(X−E(X))2= "...

You could have written E( (X - E(X) )2 ) as well? These two are interchangable, are they not?

Thank you!
 
vclaire said:
I have one more question.

When you started calculating the variance:
"Now let's think about the expected value of the variance around the expected value.
E(X−E(X))2= "...

You could have written E( (X - E(X) )2 ) as well? These two are interchangable, are they not?

Thank you!
Click to expand...
Yes. That probably would have been better. It was getting late, and the coding of the LaTeX was tedious so my mind was elsewhere.

You should probably look to see if pka makes some comments. He is really good with both the mechanics and theory of this stuff.
 
You must log in or register to reply here.
Top