NAS Performance Estimation: Weight Sharing vs Full Training (2D)

Performance Estimation

Strategy

Search Strategy

Sampling speed 2.0 archs/s

Mutation / exploration rate 0.30

Live Readouts

Architectures sampled

Compute spent

0 GPU·h

Best TRUE accuracy

—

Proxy↔True correlation

—

Current sampled path accuracy (proxy)

—

How it works

A supernet is one big graph of L stages, each offering K candidate operations that all share one pool of weights. A "child" architecture is just a path — one op chosen per stage. NAS needs to rank thousands of such paths without training each from scratch, so it uses a performance-estimation strategy:

true(path)  = mean(quality_i for op_i in path) + synergy(path)   [ground truth, only revealed by full training]

Full Training:   proxy = true + N(0, 0.02),        cost = 50   → correlation ρ ≈ 0.97
Early Stopping:  proxy = true + N(0, 0.08),        cost = 12   → correlation ρ ≈ 0.80
Weight Sharing:  proxy = true·ρ_ws + noise·(1-ρ_ws), cost = 1  → ρ_ws shrinks as sampling
                                                                  frequency grows skewed
                                                                  (co-adapted shared weights)

Weight sharing (ENAS/one-shot NAS) is nearly free per architecture, but the shared weights get optimized jointly across whatever paths happen to be sampled — so an operation's shared weights end up specialized to its frequent neighbours, and the proxy accuracy of a rarely-co-sampled path becomes a poor predictor of how that path would score standalone. This is the documented weight-sharing ranking-degradation effect (Sciuto et al., 2019, and the ENAS/DARTS literature) reproduced here as a live rank-correlation readout, ρ, between every architecture's shared-weight proxy score and its ground-truth score.

Strategy — switches which estimator feeds the search: notice how Weight Sharing spends ~1/50th the compute of Full Training but its correlation readout drifts lower the longer a biased search runs.
Search strategy — Random samples uniformly; Evolutionary keeps a population of the best-proxy paths and mutates one stage at a time; Bandit Controller draws each stage from a softmax over running per-op quality estimates Q̂, like a simplified RL controller.
Node colour in the 2D lattice is each operation's current Q̂ estimate (blue = low, gold = high); the highlighted teal path is the path just sampled, the persistent white path is the best TRUE architecture found so far. Drag to pan, scroll/pinch to zoom.

Just-sampled path Best true architecture

drag to pan · scroll to zoom