471 HMM (Hidden Markov Model)

471 HMM (Hidden Markov Model)

A hidden Markov model (HMM) is a probabilistic model that is based on the assumption that "hidden states" that we cannot see directly change over time, while observable data appear accordingly. For example, we may not be able to directly see the weather conditions, but we can observe whether a person is holding an umbrella. In such a situation, HMM is used when we want to estimate the hidden state (weather) from the observed information (presence or absence of an umbrella).

In this model, "observed variables" and "latent variables" play an important role. Observed variables are what we can actually obtain as data, such as audio signals, text, or the presence or absence of umbrellas. In contrast, latent variables are the causes or conditions behind the observation, but which we cannot directly see. In HMM, observed variables are considered to occur in dependence on latent variables, and the relationship is expressed as a "conditional distribution. In other words, it assumes a probability connection such as "under certain conditions, this kind of observation is likely to occur".

The transition of states and the occurrence of observations are all probabilistically determined, and HMM can use this kind of probability mechanism to answer several basic questions. The first is the "evaluation" question, which is addressed when one wants to know how likely a certain sequence of observations is to occur under the model. Then there is the "decoding" question, which estimates the most likely sequence of latent states from the observed data. Finally, there is the "learning" problem, which is the task of optimizing the parameters of the model, such as state transitions and probability of occurrence of observations, based on a large number of observations.

In this learning process, the concept of a "log-likelihood function" is used. This is a logarithmic function that facilitates the calculation of the certainty (likelihood) that the observed data will actually be generated from the model. By varying the parameters of the model and adjusting it so that this log-likelihood is maximized, a plausible model is obtained. Especially in cases where direct maximization is difficult due to hidden states, the Baum-Welch algorithm (EM algorithm) is used to optimize step by step using expected values.

HMMs are widely used in speech recognition, character recognition, natural language processing, and other applications for interpreting data observed over time. Its approach of trying to infer unseen mechanisms from visible data is also akin to human cognition and judgment, making it an attractive model for intelligent inference.