Download Axioms of Measure Theory - Lecture Notes | MATH 532 and more Study notes Mathematical Methods for Numerical Analysis and Optimization in PDF only on Docsity!
MATH 532 Axioms of Measure Theory
I. The Space – Throughout the course, we shall let X denote a generic non-empty set. In general, we shall not assume that any algebraic structure exists on X so that X is merely a collection of objects under consideration. However at times we may let X be a metric space with distance function d , or let X be a normed, linear space with norm ⋅ and distance function d (x, y ) = x − y. A
special case of both is when X is the real line with the absolute value being the norm. When we consider probability applications, the set of all possible objects under consideration is called the sample space and is denoted by Ω. In this case, Ω also could be the set of all possible outcomes in a random procedure or sequence of random procedures. A single element from Ω is called an outcome and is denoted by. After determining Ω , we should determine its cardinality, denoted by Ω. The cardinality may be finite, denumerable (countably infinite), or uncountable.
Example 1.1. Describe the sample space Ω in the following scenarios. Then give Ω.
(a) Pick one student at random from a class of 40. (b) Roll two dice. (c) Roll a single six-sided die 10 times in a row. (d) Play the lottery until you win. (e) Pick a real number at random from the interval [–2, 2]. (f) Pick a rational number at random from the interval [0, 1].
Solution. (a) Ω is the set of all students in the class; Ω = 40.
(b) Ω = {{1, 1}, {1, 2},.. ., {6, 6}} = {( x, y) : x, y ∈{1,..., 6}}; Ω = 36.
(c) Ω = {{ 1 , 2 , .. ., 10 }: i ∈{1,..., 6}} ; Ω = 610 = 60,466,176.
(d) Ω = {W, LW, LLW, LLLW, LLLLW, LLLLLW,... }; Ω = ℵ 0 (denumerable).
(e) Ω = [–2, 2]; Ω = c (the uncountable cardinality of ℜ ).
(f) Ω = Q ∩ [0, 1]; Ω = ℵ 0 because the rationals are denumerable.
II. The -algebra – Given a set X , we shall assume the existence of a certain collection of subsets of X , denoted by F. In order to work with these subsets, we must assume that we can perform various set operations, such as unions and complements. The following definition lays out the minimum requirements necessary from which other operations will follow as consequences.
Definition 1.1. Let F be a collection of subsets of a set X. Then F is called a -algebra and its elements are called measurable sets , or events , provided:
(i) F contains at least one set; (ii) F is closed under complements; (iii) F is closed under denumerable unions.
Note : If F is merely closed under finite unions, then F is called an algebra. We shall show below that every -algebra is an algebra.
Proposition 1.1. Let F be a -algebra of a set X_._
(a) X ∈ F and ∅ ∈ F. (b) F is closed under denumerable intersections. (c) F is closed under finite unions. (d) F is closed under finite intersections.
Proof. (a) By Axiom (i), F contains at least one set A 1. Then A 2 = A 1 c^ ∈ F because F is
closed under complements. Now let Ai = A 1 for all i ≥ 3. Because F is closed under
denumerable unions, we have X = A 1 ∪ A 1 c^ = Ai i = 1
∞
U ∈^ F^. Then^ ∅^ =^ X
c (^) ∈ F as well.
(b) Let Ai ∈ F for i ≥ 1. Because F is closed under complements and denumerable
unions, Aic^ ∈ F for all i , and thus ∪
i = 1
∞ Aic^ ∈ F. Thus, by DeMorgan’s Law and closure
under complements, we have ∩
i = 1
∞
Ai = (∪
i= 1
∞
Aic^ )c^ ∈ F.
(c, d) Let Ai ∈ F for 1 ≤ i ≤ n. Then let Ai = ∅ for i ≥ n + 1. Because F is closed under
denumerable unions, we have Ai i = 1
n
U =^ Ai
i = 1
∞
U ∈^ F^.^ Thus,^ F^ is also closed under finite
unions. Now because F is closed under complements and finite unions, we have ∩
i = 1
n Ai
i = 1
n
Aic^ )c^ ∈ F ; thus, F is also closed under finite intersections. QED
Note : (a) F is also closed under set difference which is defined as A − B = A ∩ Bc^.
(b) Given any set X , the power set of X , which is the collection of all subsets of X denoted by P( X) , is clearly a -algebra.
The Generated -algebra
Let X be a set and let A be a non-empty collection of subsets of X. Then we can generate a -algebra from A , denoted by (A) , as follows: (i) P( X) is a -algebra
that contains the smaller collection of subsets A ; (ii) Let {F } be the collection of all
-algebras that contain A ; (iii) the -algebra generated by A is given by
(A) = I F.
We also can weaken the condition of the generating class of sets and still generate B. Specifically, let Aq be the collection of all intervals of the form [a, b) with a, b ∈ Q
and a < b, and let Bq = (Aq ). Then the Borel -algebra B contains Aq. But because
Bq is the smallest -algebra that contains Aq , we must have Bq ⊆ B. As another exercise, we can show that Bq contains all intervals of the form [a, b)
where a, b ∈ ℜ with a ≤ b. But because B is the smallest -algebra that contains these
open intervals, we must have B ⊆ Bq. Hence, B = Bq.
Exercise 1.4. Prove that Bq contains all intervals [a, b) where a, b ∈ ℜ with a ≤ b.
III. The measure – Given a set X with a -algebra F , a measure is an extended real valued function defined on F that satisfies the following properties:
(i) 0 ≤ ( A) ≤ ∞ for all events A ∈ F with (∅) = 0.
(ii) If { Ai}i= 1
∞
is a denumerable sequence of disjoint events, then ( ∪
i = 1
∞
Ai ) = ( Ai
i = 1
∞
i.e., is countably additive.
Collectively, the triple ( X, F , ) is called a measure space. If (A) = 0, then A is said
to have measure 0 and A is said to be a negligible or null set. If (A) < ∞ for all events
A , then is called a finite measure.
A special case of a finite measure is a probability measure P defined on a sample space Ω. But in this case we need the additional condition that P(Ω) = 1.
Some basic properties of a measure and of a probability measure P are given next. Any property of also holds for P , but there are some additional properties that hold only for a probability measure P.
Proposition 1.2. Let ( X, F , ) be a measure space and let (Ω, F , P ) be a probability space.
(a) Let { Ai}i^ n=^1 be a finite collection of disjoint events. Then ( ∪
i = 1
n
Ai ) = ( Ai
i= 1
n
∑ ); i.e.,
is finitely additive. In particular, (A ∪ B) = (A) + ( B) for disjoint events A and B.
(b) For all events A ∈F , (A) + ( Ac^ ) = ( X ) and P(Ac^ ) = 1 − P(A).
(c) If A , B are events with B ⊆ A , then (B) ≤ ( A). If in addition (B) < ∞ , then
(A − B) = ( A) − (B) when B ⊆ A.
(d) For all events A ∈F , 0 ≤ ( A) ≤ ( X) and 0 ≤ P( A) ≤ 1.
(e) Let be a finite measure. For all A , B ∈ F , (A ∪ B) = ( A) + (B) − ( A ∩ B)
and (A ∆ B) = ( A) + (B) − 2 ( A ∩ B).
(f) Let { Ai}i= 1
∞
be a collection of events. Then ( ∪
i = 1
∞
Ai ) ≤ (Ai
i = 1
∞
∑ )^.
(g) Let { Ai}i= 1
∞
be a collection of events that each have measure 0. Then ( ∪
i = 1
∞
Ai ) = 0.
(h) Let be a finite measure and let { Bi}i = 1
∞ be a collection of events such that
(Bi ) = ( X) for all i. Then ( ∩
i = 1
∞
Bi ) = ( X).
Proof. (a) Let Ai = ∅ for i > n. The infinite sequence of sets { Ai}i= 1
∞ is still disjoint, and
because (∅) = 0, we have
i = 1
n
Ai ) = ( ∪
i = 1
∞
Ai ) = ( Ai
i = 1
∞
∑ )^ =^ (^ Ai
i = 1
n
∑ )^ +^ (∅
i = n+ 1
∞
∑ )^ =^ (Ai
i = 1
n
(b) Because events A and Ac^ are disjoint and union to X , we have
(X ) = ( A ∪ Ac^ ) = (A) + ( Ac^ ).
For a probability space (Ω, F , P ), we have in particular that
1 = P(Ω) = P(A ∪ Ac^ ) = P(A) + P(Ac^ ).
(c) If B ⊆ A , then A = B ∪ ( A − B) is a disjoint union. Thus,
(A) = (B ∪ ( A − B)) = ( B) + ( A − B) ≥ (B), because (A − B) ≥ 0.
From (A) = (B) + ( A − B), we obtain (A) − (B) = ( A − B) provided (B) < ∞.
(d) By axiom and Part (c), we have 0 ≤ ( A) ≤ ( X) because any event A is contained
in X. For a probability measure we have 0 ≤ P( A) ≤ P(Ω) = 1.
(e) The event A ∪ B can be written as the disjoint union A ∪ (B − A) , and the event B
can be written as the disjoint union ( A ∩ B) ∪ (B − A). Thus,
(B) = (A ∩ B) + (B − A) so that (B − A) = ( B) − ( A ∩ B).
We then have (A ∪ B) = ( A) + (B − A) = (A) + (B) − ( A ∩ B).
Proposition 1.3. (a) Let { Ai}i= 1
∞ be a nested increasing sequence of events. Then
i = 1
∞
Ai ) = lim
n →∞
( An).
(b) Let { Ai}i= 1
∞ be a nested decreasing sequence of events with (A 1 ) < ∞. Then
i = 1
∞
Ai ) = lim
n →∞
( An).
Proof. (a) We again create a disjoint sequence of events { Bi}i = 1
∞
such that ∪
i = 1
n
Bi = ∪
i = 1
n Ai
for 1 ≤ n ≤ ∞. To do so, let B 1 = A 1 and let Bi = Ai − ∪
j < i
Aj for i ≥ 2. Because { Ai}i= 1
∞ is
nested increasing, then An = ∪
i = 1
n
Ai = ∪
i = 1
n Bi for all n < ∞. Thus we have
lim n →∞
( An) = lim
n→∞
i = 1
n
Bi ) = lim
n→∞
( Bi)
i = 1
n
∑ =^ (^ Bi)
i = 1
∞
∑ =^ (^ ∪
i = 1
∞
Bi ) = ( ∪
i = 1
∞
Ai ).
(b) Because (A 1 ) < ∞ , then all measurable subsets of A 1 have finite measure by
Proposition 1.2 (c). For all n ≥ 1 , we have An ⊆ A 1 ; thus, (A 1 − An ) = (A 1 ) − ( An ).
But { A 1 − Ai}i^ ∞ =^1 is a nested increasing sequence with ( A 1 − Ai )
i = 1
∞
U =^ A 1 −^ Ai
i = 1
∞
I.^ Thus, by
Part (a) and Proposition 1.2 (c), we have
lim n →∞
( An) = ( A 1 ) − lim
n→∞
( A 1 − An) = (A 1 ) − (A 1 − Ai )
i = 1
∞
U
= ( A 1 ) − A 1 − An i = 1
∞
I
=^ (^ A^1 )^ −^ (^ A^1 )^ −^ (^ Ai i = 1
∞
I )
=^ (^ Ai i = 1
∞
I ).
Some Examples of Probability Measures
1. Equi-Probable Outcomes – Let Ω be a finite set with Ω = n and let F be the
power set of Ω , which consists of all 2 n^ possible subsets of Ω. For every ∈ Ω , we can let P({ }) = 1 / n to denote that each individual outcome is equally likely. Then for
any event A ∈ F , we have P(A) = A / n. In particular, P(Ω) = Ω / n = 1.
Because for a finite collection of disjoint finite sets we have Ai i = 1
n
U =^ Ai
i = 1
n
∑ , it
follows that P is finitely additive. (In this case, we only need P to be finitely additive since there is no need to have infinite unions of sets.)
2. Binomial Probability – Let Ω = {( 1 , .. ., (^) n ): (^) i ∈{W, L}}, which represents the set
of outcomes in a sequence of n independent attempts. On the i th attempt, (^) i is either
W for a win or L for a loss. We assume that each win occurs with probability p and
each loss occurs with probability q = 1 − p. Then Ω = 2 n. We again let F be the
power set of Ω , which consists of the 22
n subsets of Ω. With n = 3 , then Ω has 23 = 8 elements, and F has 28 = 256 elements. In this case,
Ω = {(W,W ,W ), (W,W , L), (W, L,W ), (L ,W, W), (W, L, L ), (L, W, L ), (L , L, W), ( L, L, L)}.
Various events in F can be described in words. For example, A 1 = a win on the first
try; A 2 = at least one win; A 3 = never the same outcome back to back. Then,
A 1 = {(W ,W, W), (W, W, L ), (W, L, W ), (W , L, L )}
A 2 = {(W, W,W ), (W, W, L ), (W , L, W), ( L,W ,W ), (W , L, L ), ( L,W, L), ( L, L, W )}
A 3 = {(W , L,W ), ( L,W , L)}.
In all, there are 256 possible events. We define a probability measure P on the set of events F as follows: For a single
outcome = ( 1 , .. ., (^) n ), we let P({( 1 , ..., (^) n )}) = pk^ qn−k^ , where k is the number
of wins in the sequence ( 1 , .. ., (^) n ). Then for A ∈ F , we let P(A) = P({ }) ∈A
∑.^ Then
P is finitely additive as with equi-probable outcomes. Moreover, the individual outcomes are equally likely if and only if p = q = 1/ 2.
Now suppose we let Wk be the event that there are exactly k wins in a sequence for k = 0, 1,. .., n. Then the events W 0 , W 1 ,... , Wn are disjoint and union to Ω. Because
there
n k
(^) distinct ways to have exactly k wins in a sequence, the probability of Wk is
n k
(^) pk^ qn−k^. Then by the binomial expansion theorem, we have
P(Ω) = P(Wk ) k = 0
n
∑ =^
n k
(^) pk^ qn−^ k k = 0
n
∑ =^ (^ p^ +^ q)
n (^) = 1 n (^) = 1.
3. Geometric Probability – Again let p be the probability of a win on any independent
attempt, and let q = 1 − p be the probability of a loss. But now assume that 0 < p < 1.
Let Ω = {W , LW, LLW, LLLW,. ..}, which represents the possible outcomes in a
sequence of attempts where we stop play upon the first win. Then Ω =ℵ 0. Let F be
the power set of Ω which is uncountable with cardinality 2 ℵ^0 = c. For each ∈ Ω ,
define P({ }) = qn−^1 p, where n is the total number of attempts made in sequence.
Then for A ∈ F , let P(A) = P({ }) ∈A
Exercise 1.5. Let { Ai}i= 1
∞ be a sequence of events in a -algebra F. Define the liminf
and limsup of this sequence by
lim inf {Ai} = ∪
n= 1
∞
i= n
∞ Ai
limsup{Ai} = ∩
n= 1
∞
i= n
∞ Ai
(a) Explain why limsup{Ai} and lim inf {Ai} are in F.
(b) Show that (liminf {Ai})c^ = limsup{Aic} and (limsup{Ai})c^ = lim inf {Aic}.
(c) Prove that an element x is in limsup{Ai} if and only if x belongs to an infinite
number of the sets Ai.
(d) Prove that an element x is in lim inf {Ai} if and only if x belongs to all but a finite
number of the sets Ai.
(e) Show that lim inf {Ai} ⊆ limsup{Ai}.
(f) If { Ai}i= 1
∞
is nested increasing, prove that lim inf {Ai} = limsup{Ai} = ∪
n= 1
∞ An.
(g) If { Ai}i= 1
∞
is nested decreasing, prove that lim inf {Ai} = limsup{Ai} = ∩
n= 1
∞ An.
Exercise 1.6. Let A , B be events such that (A ∆ B) = 0. Prove that (A) = (B).
Exercise 1.7. Let A , B be events with B ⊆ A such that (A) < (B) + for all > 0.
Prove that (A) = (B).
Exercise 1.8. Let be a finite measure and let { Ai}i= 1
∞ be nested increasing with
i = 1
∞
Ai = X. Prove that ( ∩
i = 1
∞
Aic^ ) = 0 = lim
n→∞
( Anc^ ).
