Download Uncertainty in Decision Making: Expected Values & Decision Trees in AI Foundations and more Lab Reports Computer Science in PDF only on Docsity!
CS 2710 Foundations of AI
CS 2710 Foundations of AI
Lecture 19
Milos Hauskrecht
milos@cs.pitt.edu
5329 Sennott Square
Decision making in the presence of uncertainty
Decision-making in the presence of uncertainty
- Computing the probability of some event may not be our
ultimate goal
- Instead we are often interested in making decisions about
our future actions so that we satisfy goals
- Example: medicine
- Diagnosis is typically only the first step
- The ultimate goal is to manage the patient in the best
possible way. Typically many options available:
- Surgery, medication, collect the new info (lab test)
- There is an uncertainty in the outcomes of these
procedures: patient can be improve, get worse or even
die as a result of different management choices.
CS 2710 Foundations of AI
Decision-making in the presence of uncertainty
Main issues:
- How to model the decision process with uncertain
outcomes in the computer?
- How to make decisions about actions in the presence of
uncertainty?
The field of decision-making studies ways of making
decisions in the presence of uncertainty.
Decision making example.
Assume we want to invest $100 for 6 months
1. Invest in Stock 1
2. Invest in Stock 2
3. Put money in bank
4. Keep money at home
Stock 1 Stock 2
Bank Stock 1 value can go up or down:
Up: with probability 0.
Down: with probability 0.
Home
(up) (down)
CS 2710 Foundations of AI
Decision making example.
We need to make a choice whether to invest in Stock 1 or 2, put
money into bank or keep them at home. But how?
Stock 1 Stock 2 Bank
Monetary
outcomes
for different
scenarios
Home
? (up) (down) (up) (down)
Decision making example.
Assume the simplified problem with the Bank and Home
choices only.
The result is guaranteed – the outcome is deterministic
What is the rational choice assuming our goal is to make
money?
Bank 1.0 101 Home 1.0 100
CS 2710 Foundations of AI
Decision making. Deterministic outcome.
Assume the simplified problem with the Bank and Home
choices only.
These choices are deterministic.
Our goal is to make money. What is the rational choice?
Answer: Put money into the bank. The choice is always
strictly better in terms of the outcome
But what to do if we have uncertain outcomes?
Bank 1.0 101 Home 1.0 100
Decision making. Stochastic outcome
Stock 1
Stock 2 Bank 1.0 101
- How to quantify the goodness of the stochastic outcome?
We want to compare it to deterministic and other
stochastic outcomes.
(up) (down)
CS 2710 Foundations of AI
Expected value
Stock 1
• Let X be a random variable representing the monetary
outcome with a discrete set of values.
• Expected value of X is:
• Expected value summarizes all stochastic outcomes
into a single quantity
• Example:
Expected value for the outcome of the Stock 1 option is:
E ( X )= x ∑∈Ω XxP ( X = x )
Ω X
0. 6 × 110 + 0. 4 × 90 = 66 + 36 = 102
Expected values
Investing $100 for 6 months
Stock 1 Stock 2 Bank
Home
× + × =
(up) (down) (up) (down)
CS 2710 Foundations of AI
Expected values
Investing $100 for 6 months
Stock 1 Stock 2 Bank
Home
× + × =
× + × =
(up) (down) (up) (down)
Expected values
Investing $100 for 6 months
Stock 1 Stock 2 Bank
Home
× + × =
× + × =
(up) (down) (up) (down)
CS 2710 Foundations of AI
Expected values
Investing $100 for 6 months
Stock 1 Stock 2 Bank
Home
× + × =
× + × =
1. 0 × 101 = 101
1. 0 × 100 = 100
(up) (down) (up) (down)
Selection based on expected values
The optimal action is the option that maximizes the
expected outcome:
Stock 1 Stock 2 Bank
Home
(up) (down) (up) (down)
CS 2710 Foundations of AI
Relation to the game search
- Game search: minimax algorithm
- considers the rational opponent and its best move
- Decision making: maximizes the expectation
- play against the nature - stochastic non-malicious
“opponent”
Stock 1 Stock 2 Bank
Home
(up) (down) (up) (down)
(Stochastic) Decision tree
- Decision tree:
- decision node
- chance node
- outcome (value) node
Stock 1 Stock 2 Bank
Home
(up) (down) (up) (down)
CS 2710 Foundations of AI
Assume:
- Two investment periods
- Two actions: stock and bank
Multi-step problem example
Stock
Bank
0.5 (up) (down)
(up) (down)
(up) (down)
(up)
(down)
Stock Bank Stock Bank
Stock Bank
Assume:
- Two investment periods
- Two actions: stock and bank
Multi-step problem example
Stock
Bank
0.5 (up) (down)
(up) (down)
(up) (down)
(up)
(down)
Stock Bank Stock Bank
Stock Bank
CS 2710 Foundations of AI
Assume:
- Two investment periods
- Two actions: stock and bank
Multi-step problem example
Stock
Bank
0.5 (up) (down)
(up) (down)
(up) (down)
(up)
(down)
Stock Bank Stock Bank
Stock Bank
Assume:
- Two investment periods
- Two actions: stock and bank
Multi-step problem example
Stock
Bank
0.5 (up) (down)
(up) (down)
(up) (down)
(up)
(down)
Stock Bank Stock Bank
Stock Bank
CS 2710 Foundations of AI
- But this may not be the case. In decision trees:
- Later outcomes can be conditioned on the earlier
stochastic outcomes and actions
Example: stock movement probabilities. Assume:
P(1st^ =up)=0.
P(2 nd=up|1 st^ =up)=0.
P(2 nd=up|1 st^ =down)=0.
Conditioning in the decision tree
Stock Bank
(2 nd^ up) (2nd^ down)
(2 nd^ up) (2 nd^ down)
(1st^ up)
(1st^ down)
Stock Bank Stock Bank
Multi-step problems. Conditioning.
Tree Structure: every observed
stochastic outcome = 1 branch
P(1st^ =up)=0.
P(2 nd=up|1 st^ =up)=0.
P(2 nd=up|1 st^ =down)=0.
140
105
(2 nd^ up) (2 nd^ down) (2 nd^ up) (2 nd^ down)
(1 st^ up)
(1 st^ down)
140 80 105
80
Stock Bank Stock Bank
Bank
Stock
(2 nd^ up) (2 nd^ down)
(2 nd^ up) (2 nd^ down)
(1 st^ up)
(1 st^ down)
200
130 60 90
100 125
Stock Bank Stock Bank
CS 2710 Foundations of AI
- Stochastic outcomes can be explicitly represented in the tree
- But collapse of some branches is possible
- Example:
Multi-step problems. Conditioning.
Bank
(up) (down)
(up) (down) (up)
(down)
(up)
(down)
Stock Bank Stock Bank
Stock Bank
Bank
- If conditioning is present – we can always build the tree in
which all stochastic outcomes are explicitly represented
Multi-step problems. Conditioning.
Bank
(up) (down)
(up) (down) (up)
(down)
(up)
(down)
Stock Bank Stock Bank
Stock Bank
Bank
0.4x0.4+0.6x0.5=0.
