LLM Concept Lab: 12 interactive labs on how language models work

Twelve interactive labs on how language models work: n-grams, Word2Vec, tokenization, RNNs and attention. Edit the data and run the real algorithm in-browser.

• The Markov assumption and the bigram model
• Smoothing: giving unseen N-grams a chance
• Evaluating language models: perplexity and friends
• WordNet: meaning written down by hand
• Word2Vec: meaning as a position in space
• CBOW: predict the missing word from its neighbours
• Skip-gram: one word, many training pairs
• Training Word2Vec live: negative sampling, α and subsampling
• WordPiece vs BPE: building a subword vocabulary
• Recurrent neural networks: memory that fades
• Seq2Seq: encoder, decoder, attention and beam search
• Scaled dot-product attention and the √dₖ