Git was built to diff and merge small text files. It has no efficient way to
store many full versions of a large binary dataset — every commit that touches
the file adds another complete copy into .git's object history,
and that history is never automatically pruned. Tools like DVC
(Data Version Control) solve this by keeping git tiny: git only ever stores a
small text pointer file (a content hash), while the actual dataset bytes live
in a separate, content-addressable cache or remote storage bucket.
.dvc pointer file to git (a hash + metadata). The real bytes go into the content-addressable store on the right.This is a simplified teaching model — real Git does deduplicate identical blob content by SHA-1 hash too. The real-world problem DVC solves is that large, frequently changing binary files rarely hash identically between versions, so Git's history balloons with near-duplicate blobs it can't diff efficiently, while DVC pushes that bulk storage out of git entirely.
A side-by-side 3D repository timeline: commit dataset snapshots and watch a naive Git repo grow a full-size block every time, while a DVC-style repo stays flat, storing only tiny pointer files and pushing the real bytes into a deduplicated content-addressable store.
Every "Add dataset snapshot" click commits to both repos at once. The naive Git rail stacks a full-size blob per version forever; the DVC rail adds only a ~3.4KB pointer, and its storage rail only grows when the content hash actually changes.
Set the snapshot size and how much of the dataset changed since last time. Drop "Change" to 0% and add a snapshot to trigger a dedup cache hit — the storage block pulses instead of duplicating. Try Auto-commit to watch the gap widen on its own.
DVC's .dvc files are plain text — just a path, a file size and a hash — small enough that git diffs and merges them like any other text file, even though the data they point to can be gigabytes.