LLM Concept Lab

The Markov assumption and the bigram model

Turns raw counts into next-word probabilities using only the previous word.

The idea in plain words

Phone keyboard autocomplete. When you type "good", it suggests "morning" or "luck" by looking only at that last word, not at your whole message. That shortcut is the Markov assumption: the recent past is a good enough stand-in for the whole past.

Step by step

  1. Count how often each word follows each other word in a pile of text.
  2. Divide: P(next | previous) = times "previous next" appeared ÷ times "previous" appeared.
  3. To write a sentence, keep picking a likely next word and moving forward one step.

Play it on paper first

Take any book page. Pick a word such as "the", and tally every word that follows it. The tallies are your bigram table. Now try the same with the two previous words and notice how few matches you find: that is why we cut the history short.

Other labs