Home▸AI & Machine Learning▸2D Masked-Word Prediction: Bidirectional versus Causal Attention Masks against the Exact Bayes Limit on a Hidden-Theme Cloze Task

🙈 2D Masked-Word Prediction: Bidirectional versus Causal Attention Masks against the Exact Bayes Limit on a Hidden-Theme Cloze Task

Sentence tiles with attention arcs for a bidirectional and a causal mask, 16 candidate-word bars against the exact Bayes predictor, held-out loss curves per hidden slot and six independent checks (finite-difference gradients, mask leak tests, calibration, ln 16 baseline, Bayes bound).

AI & Machine Learning2DModerate60 FPS📱 Mobile-adapted🟠 Blender⇄ 3D version
2D Masked-Word Prediction: Bidirectional versus Causal Attention Masks against the Exact Bayes Limit on a Hidden-Theme Cloze Task ↗ Open standalone
⚙ Under the hood

Sentence tiles with attention arcs for a bidirectional and a causal mask, 16 candidate-word bars against the exact Bayes predictor, held-out loss curves per hidden slot and six independent checks (finite-difference gradients, mask leak tests, calibration, ln 16 baseline, Bayes bound).

Masked Language ModelSelf-AttentionCausal MaskBidirectional AttentionBERTGPTBayes OptimalNatural Language Processing

2D · HTML5 Canvas 2D · 60 FPS target · runs fully client-side, no install

What did you find?

Add reproduction steps (optional)