Home▸AI & Machine Learning▸3D Masked-Word Prediction: Bidirectional versus Causal Attention Masks against the Exact Bayes Limit on a Hidden-Theme Cloze Task

🙈 3D Masked-Word Prediction: Bidirectional versus Causal Attention Masks against the Exact Bayes Limit on a Hidden-Theme Cloze Task

Two Blender-built decks train the same single-head attention model with a bidirectional (BERT-style) and a causal (GPT-style) mask to guess a hidden word in template sentences, with attention pillars, candidate-word bars, and the exact Bayes predictor as the limit that no model can beat.

AI & Machine Learning3DAdvanced60 FPS📱 Mobile-adapted🟠 Blender⇄ 2D version
3D Masked-Word Prediction: Bidirectional versus Causal Attention Masks against the Exact Bayes Limit on a Hidden-Theme Cloze Task ↗ Open standalone
⚙ Under the hood

Two Blender-built decks train the same single-head attention model with a bidirectional (BERT-style) and a causal (GPT-style) mask to guess a hidden word in template sentences, with attention pillars, candidate-word bars, and the exact Bayes predictor as the limit that no model can beat.

Masked Language ModelSelf-AttentionCausal MaskBidirectional AttentionBERTGPTBayes OptimalNatural Language Processing

3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install

What did you find?

Add reproduction steps (optional)