WordNet: meaning written down by hand

Groups words into synsets with glosses and is-a links, so "motel" is known to be a kind of "hotel".

The idea in plain words

A hand-built library catalogue for meaning. Words that mean the same thing share one card (a synset); each card points to a more general card ("motel" → "hotel" → "building"). Like folders inside folders, but for ideas.

Step by step

1. A synset is one meaning shared by several words (car, auto, automobile).
2. A hypernym is the "is-a" parent: every dog is a canine. A hyponym is the more specific child.
3. Follow parents upward to see how closely related two words are: nearby in the tree means similar meaning.

Play it on paper first

Pick any noun and keep asking "what is it a kind of?" until you reach "thing". Do it for a second word and see where your two chains meet. The meeting point is how related they are.

Other labs

• The Markov assumption and the bigram model
• Smoothing: giving unseen N-grams a chance
• Evaluating language models: perplexity and friends
• WordNet: meaning written down by hand
• Word2Vec: meaning as a position in space
• CBOW: predict the missing word from its neighbours
• Skip-gram: one word, many training pairs
• Training Word2Vec live: negative sampling, α and subsampling
• WordPiece vs BPE: building a subword vocabulary
• Recurrent neural networks: memory that fades
• Seq2Seq: encoder, decoder, attention and beam search
• Scaled dot-product attention and the √dₖ