transformer
How each token looks at the others. Softmax over keys, one query at a time.
Tokens
Query × key
Causal mask
Each query sees only itself and the past.
Attention(Q, K, V) = softmax(QKᵀ / √d) V · Heads are a local window, lexical overlap, light syntax, and a first-token bias — a sample, not a trained model.