Goal of this project is to build a predictive text model, a language model that helps people type on mobile devices.
- Language model(LM) is to compute a conditional probability of an upcoming word(\( W_{N} \)) given the sequence of previous words(\( W_{1}, W_{2}, ..., W_{N-1} \))
\[ P(W_{N} | W_{1}, W_{2}, ..., W_{N-1}) \]
- Markov Assumption simplifies the calculation of the probability via approximation:
\[ P(W_{N} | W_{1}, W_{2}, ..., W_{N-1})\approx P(W_{N}|W_{N-k},\dots ,W_{N-1}) \;\;\; \]
- N-gram models based on Markov assumption are unigram (k=1), bigram (k=2), trigram(k=3), …
- Words that don't apprear in the training set has zero probability and need to be taken care of by smoothing technique such as backoff.
