🙈 2D Masked-Word Prediction: Bidirectional versus Causal Attention Masks against the Exact Bayes Limit on a Hidden-Theme Cloze Task
Sentence tiles with attention arcs for a bidirectional and a causal mask, 16 candidate-word bars against the exact Bayes predictor, held-out loss curves per hidden slot and six independent checks (finite-difference gradients, mask leak tests, calibration, ln 16 baseline, Bayes bound).
⚙ Under the hood
Sentence tiles with attention arcs for a bidirectional and a causal mask, 16 candidate-word bars against the exact Bayes predictor, held-out loss curves per hidden slot and six independent checks (finite-difference gradients, mask leak tests, calibration, ln 16 baseline, Bayes bound).
Masked Language ModelSelf-AttentionCausal MaskBidirectional AttentionBERTGPTBayes OptimalNatural Language Processing
2D · HTML5 Canvas 2D · 60 FPS target · runs fully client-side, no install