Home▸Data Science▸MinHash Sketches: Jaccard Similarity Estimation, Variance and Sketch Size

MinHash Sketches: Jaccard Similarity Estimation, Variance and Sketch Size

Two sets are summarised by the minimum value of k random hash functions, and the fraction of matching minima estimates their Jaccard similarity. Change the true similarity, the number of hashes and the set size and see a hundred independent sketches scatter around the truth with variance J(1-J)/k, against the binomial and Gaussian curves and the Hoeffding sketch-size bound. A Blender-built plot table with Monte Carlo and exact set checks built in.

Data Science3DModerate60 FPS📱 Mobile-adapted🟠 Blender⇄ 2D version
MinHash Sketches: Jaccard Similarity Estimation, Variance and Sketch Size ↗ Open standalone

Cloud ML platforms let you trade cost for reliability by choosing how many training instances are cheap-but-revocable spot capacity versus full-price on-demand capacity. This simulator runs a synchronized data-parallel training cluster where spot instances can be reclaimed at random, rolling the whole job back to its last checkpoint. Tune the spot share, interruption rate, checkpoint interval and restart time to see how the mixed fleet's total cost and completion time compare live against an all-on-demand baseline.

⚙ Under the hood

Two sets are summarised by the minimum value of k random hash functions, and the fraction of matching minima estimates their Jaccard similarity. Change the true similarity, the number of hashes and the set size and see a hundred independent sketches scatter around the truth with variance J(1-J)/k, against the binomial and Gaussian curves and the Hoeffding sketch-size bound. A Blender-built plot table with Monte Carlo and exact set checks built in.

cloud mlaws sagemakerdistributed trainingspot instancesmlopscost optimizationMinHashJaccard SimilaritySketchingNear-Duplicate DetectionHashing

3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install

What did you find?

Add reproduction steps (optional)