Mixed Reality Occlusion Compositing

Hologram

Depth into room 0.0 m

Depth Sensor

Sensor resolution 100%

Sensor noise 0.000

Live Readouts

Hologram depth

1.0 m

Real surface

—

Occluded surface area (ground truth)

How it works

Augmented reality just draws overlays on top of the camera feed. Mixed reality is defined by the opposite: the headset must know the room's geometry well enough to hide a hologram behind a real couch or table. That requires a per-pixel depth test:

draw virtual pixel only if  z_virtual(x,y) < z_real(x,y) + ε(x,y)

Here z_real is rendered from the room's furniture into a depth texture every frame — a stand-in for a depth-camera scan — and z_virtual is the hologram's own fragment depth. This simulator runs that exact test in a fragment shader: it renders the room to a depth buffer, then draws a voxel hologram with a shader that discards any fragment whose depth is greater than the sampled real-world depth at that screen pixel.

Depth into room — slides the hologram along the room's axis; move it behind the couch or pillar to see it disappear where real geometry is closer to the camera.
Sensor resolution — real depth sensors (structured-light or time-of-flight) sample the room at limited resolution. Lowering it shrinks the depth texture, which the shader still samples at full screen resolution — the occlusion edge turns visibly blocky, matching real headset artifacts around thin objects.
Sensor noise — adds a per-pixel random offset ε to the comparison, mimicking depth-sensor measurement error; higher noise flickers pixels near the true occlusion boundary in and out of view.
Occlusion toggle — turns the depth test off, which is exactly what naive "always on top" AR overlay does: the hologram draws over the furniture regardless of which is actually closer to the camera.
Ground-truth readouts — computed separately by ray-casting a grid of sample rays from the camera through the hologram's silhouette against the real furniture, independent of the shader, so you can compare the true occluded fraction to what the noisy/low-res shader actually shows.

Hologram (voxel avatar) Real-world furniture (depth-scanned)

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.