Advanced Database Management System - Tutorials and Notes: Natural language processing question bank 06

# Assume that we have the Hidden Markov Model (HMM). If each of the states can take on k different values and a total of m different observations are possible (across all states), how many parameters are required to fully define this HMM? Justify your answer.

### Assume that we have the Hidden Markov Model (HMM). If each of the states can take on k different values and a total of m different observations are possible (across all states), how many parameters are required to fully define this HMM? Justify your answer.

There are a total of three probability distributions that define the HMM, the initial probability distribution, the transition probability distribution, and the emission probability distribution.
• There are a total of k states, so k parameters are required to define the initial probability distribution.
• For the transition distribution, we can transition from any one of k states to any of the k states (including staying in the same state), so k2 parameters are required.
• We need a total of km parameters for the emission probability distribution, since each of the k states can emit each of the m observations.
Thus, the total number of parameters required are k + k2 + km. Note that the number of parameters does not depend on the length of the HMMs.

*************************