Download Relationship between Conditional and Marginal Means and Variances and more Lecture notes Mathematical Statistics in PDF only on Docsity!
M 384G/374G/CAM384T
CONDITIONAL AND MARGINAL MEANS AND VARIANCES
I. Question : How are conditional means E(Y|X) and marginal means E(Y) related?
Simple example : Population consisting of n 1 men, n 2 women. Y = height X = sex Categorical, two values: Male, Female
So there are two conditional means: E(Y|male) = (Sum of all men’s heights)/n 1 E(Y|female) = (Sum of all women’s heights)/n 2 Then Sum of all men’s heights = n 1 E(Y|male) Sum of all women's heights = n 2 E(Y| female)
The marginal mean is E(Y) = (Sum of all heights)/(n 1 + n 2 ) =
( Sum of men ' s heights ) + ( Sum of women ' s heights ) n 1 + n 2
n 1 E ( Y | male ) + n 2 E ( Y | female ) n 1 + n 2
n 1 n 1 + n 2
E(Y| male) + n 2 n 1 + n 2
E(Y| female)
= (proportion of males)( E(Y| male) + (proportion of females)( E(Y| female)
= (probability of male)( E(Y| male) + (probability of female)( E(Y| female)
Thus: The marginal mean is the weighted average of the conditional means, with weights equal to the probability of being in the subgroup determined by the corresponding value of the conditioning variable.
Similar calculations show: If we have a population made up of m subpopulations pop 1 , pop 2 , …, popm (equivalently, if we are conditioning on a categorical variable with m values -- e.g., the age of a fish), then
E(Y) = Pr( popk ) E ( Y | popk ) k = 1
m
e.g., for our fish, popk = fish of age k, and
E(length) = Pr( Age = k ) E ( Length | Age = k ) k = 1
6
Rephrasing in terms of the categorical variable X defining the subpopulations,
E(Y) = Pr( x ) E ( Y | X = x ) all values x of X
Stated in words:
The analogue for conditioning on a continuous variable X is:
E(Y) = "# fX ( x ) E ( Y | x ) dx
where fX(x) is the probability density function (pdf) of X.
Note :
- There are analogous results for conditioning on more than one variable.
- The analogous result for sample means is
y =
II. A second (related) relationship between marginal and conditional means for populations :
Consider E(Y|X) as a new random variable U as follows: Randomly pick an x from the distribution of X. The new r.v. U has value E(Y|X = x).
Example: Y = height, X = sex Randomly pick a person from the population in question.
U =
μ f = E ( Y | X = female ) if the person is female μ m = E ( Y | X = male ) if the person is male
Question to think about: What might cause P(U = u) to be high? [Hint: There are two ways this might arise]
(Fill in details, and say the final result in words!)
Conditional Variance : Similarly, if we are considering a conditional distribution Y|X, we define the conditional variance
Var(Y|X) = Variance of the conditional distribution Y|X
= E([Y - E(Y|X)]^2 | X)
(Note that both expected values here are conditional expected values.)
What does this say in words (and pictures)?
Exercise: Derive another formula for the conditional variance, analogous to the second formula for the marginal variance. (And say it in words!)
Conditional Variance as a Random Variable : As with E(Y|X), we can consider Var(Y|X) as a random variable. For example, if Y = height and X = sex for persons in a certain population, then Var(height | sex) is the variable which assigns to each person in the population the variance of height for that person's sex.
IV. CONNECTING MEANS AND VARIANCES
Expected Value of the Conditional Variance : Since Var(Y|X) is a random variable, we can talk about its expected value. Using the formula Var(Y|X) = E(Y^2 |X) - [E(Y|X)]^2 , we have E(Var(Y|X)) = E(E(Y^2 |X)) - E([E(Y|X)]^2 )
We have already seen that the expected value of the conditional expectation of a random variable is the expected value of the original random variable, so applying this to Y^2 gives
(*) E(Var(Y|X)) = E(Y^2 ) - E([E(Y|X)]^2 )
Variance of the Conditional Expected Value : For what comes next, we will need to consider the variance of the conditional expected value. Using the second formula for variance, we have
Var(E(Y|X)) = E([E(Y|X)]^2 ) - [E(E(Y|X))]^2
Since E(E(Y|X)) = E(Y), this gives
(**)Var(E(Y|X)) = E([E(Y|X)]^2 ) - [E(Y)]^2.
Putting It Together :
Note that () and (*) both contain the term E([E(Y|X)]^2 ), but with opposite signs. So adding them gives:
E(Var(Y|X)) + Var(E(Y|X)) = E(Y^2 ) - [E(Y)]^2 ,
which is just Var(Y). In other words,
(***) Var(Y) = E(Var(Y|X)) + Var(E(Y|X)).
In words: The marginal variance is the sum of the expected value of the conditional variance and the variance of the conditional means.
Consequences :
1 ) This says that two things contribute to the marginal (overall) variance: the expected value of the conditional variance, and the variance of the conditional means. (See Exercise) Moreover, Var(Y) = E(Var(Y|X)) if and only if Var(E(Y|X)) = 0. What would this say about E(Y|X)?
2 ) Since variances are always non-negative, (***) implies
Var(Y) ≥ E(Var(Y|X)).
3 ) Since Var(Y|X) ≥ 0, E(Var(Y|X)) must also be ≥ 0. (Why?). Thus (***) implies
Var(Y) ≥ Var(E(Y|X)).
Moreover, Var(Y) = Var(E(Y|X)) if and only if E(Var(Y|X)) = 0. What would this imply about Var(Y|X) and about the relationship between Y and X?
4 ) Another perspective on (***) (cf. Textbook, pp. 36 - 37):
Note that: i) E(Var(Y|X) is a weighted average of Var(Y|X) ii) Var(E(Y|X) = E([E(Y|X) - E(E(Y|X))]^2 ) = E([E(Y|X) - (E(Y)]^2 ), which is a weighted average of [E(Y|X) - (E(Y)]^2
Thus, (***) says that Var(Y) is a weighted mean of Var(Y|X) plus a weighted mean of [E(Y|X) - (E(Y)]^2 (and is a weighted mean of Var(Y|X) if and only if all conditional expected values E(Y|X) are equal to the marginal expected value E(Y).)
