Hidden Markov Models: Reasoning over Time with HMMs - Prof. Derek Harter, Study notes of Computer Science

Download Hidden Markov Models: Reasoning over Time with HMMs - Prof. Derek Harter and more Study notes Computer Science in PDF only on Docsity!

CSci 538: Artificial Intelligence

Fall 2007

Ch 15: Hidden Markov Models

Derek Harter – Texas A&M University – Commerce Slides adapted from Srini Narayanan – ICSI and UC Berkeley

Reasoning over Time

 Often, we want to reason about a sequence of

observations

 (^) Speech recognition  (^) Robot localization  (^) User attention  (^) Medical monitoring

 Need to introduce time into our models

 Basic approach: hidden Markov models (HMMs)

 Using Continuous Variables: Kalman Filters

 More general: dynamic Bayes’ nets

States and Observations

 Process of change can be viewed as a

series of snapshots

 each of which describes the state of the world

at a particular time

 Each snapshot or time slice contains a set

of random variables

 Some observable some not

 Observables or evidence E

t

 Unobservable state variables (hidden) X

t

 The observation at time t is E

t

= e

t

Stationary Assumption

 Assume changes in world state are caused by

a stationary process

 A process of change that is governed by laws that

do not themselves change over time

 In practical terms, the conditional probability

can be stated generically for all time t

 P ( U

t

| Parents(U

t

) ) is the same for all t

 Given the stationary assumption, need only

specify variables within a “representative” time

slice.

Markov Assumption

 In addition to restricting the parents of the

state variables, we must restrict parents of

the evidence variables

 P(E

t

| X

0:t

,E

0:t-

) = P(E

t

| X

t

 Called the sensor model (or sometimes the

observation model)

 it describes how the “sensors” - that is the

evidence variables – are affected by the actual

state of the world

Example

 An HMM is

 Initial distribution: P(Rain

0

 Transition Model: P (Rain

t

| Rain

t-

 Sensor Model:^ P (Umbrella

t

| Rain

t

Conditional Independence

 Basic conditional independence:

 (^) Past and future independent of the present  (^) Each time step only depends on the previous  (^) This is called the (first order) Markov property

 Note that the chain is just a (growing BN)

 (^) We can always use generic BN reasoning on it (if we truncate the chain)

X

2

X

1

X

3

X

4

Joint Distribution

P (X

0

, X

1

, ... , X

t

, E

1

, ..., E

t

P(X

0

i=1,t

P(X

i

| X

i-

) P(E

i

| X

i

Mini-Forward Algorithm

 Question: probability of being in state x at time t?

 Slow answer:

 (^) Enumerate all sequences of length t which end in s  (^) Add up their probabilities

Mini-Forward Algorithm

 Better way: cached incremental belief updates

sun rain sun rain sun rain sun rain Forward simulation

Stationary Distributions

 If we simulate the chain long enough:

 (^) What happens?  (^) Uncertainty accumulates  (^) Eventually, we have no idea what the state is!

 Stationary distributions:

 (^) For most chains, the distribution we end up in is independent of the initial distribution  (^) Called the stationary distribution of the chain  (^) Usually, can only predict a short time out

Web Link Analysis

 (^) PageRank over a web graph  (^) Each web page is a state  (^) Initial distribution: uniform over pages  (^) Transitions:  (^) With prob. c, uniform jump to a random page (dotted lines)  (^) With prob. 1-c, follow a random outlink (solid lines)  (^) Stationary distribution  (^) Will spend more time on highly reachable pages  (^) E.g. many ways to get to the Acrobat Reader download page!  (^) Somewhat robust to link spam  (^) Google 1.0 returned the set of pages containing all your keywords in decreasing rank, now all search engines use link analysis along with many other factors

Mini-Viterbi Algorithm

 Better answer: cached incremental updates

 Define:

 Read best sequence off of m and a vectors

sun rain sun rain sun rain sun rain

Mini-Viterbi

sun rain sun rain sun rain sun rain