Deciding how many layers a network should have, how wide each layer should be, and which activation function to use has traditionally been a manual process guided by intuition and repeated experimentation. Neural architecture search (NAS) automates that process, treating architecture choices as a search problem in the same way hyperparameter tuning treats learning rate and batch size.
Evolutionary search, step by step
One popular approach borrows directly from evolutionary biology. A population of candidate architectures is generated, each is evaluated for fitness (typically validation accuracy after some amount of training), the best performers are kept, and the next generation is produced by mutating the survivors' encodings: small random changes to depth, width, or activation choice.
Why this actually works
Over enough generations, the population drifts toward architectures that fit the underlying problem well, because good architectures are more likely to survive and pass their traits to the next generation. It does not require understanding why a particular depth or width works, only the ability to measure how well each candidate performs.
The trade-offs
Population size and mutation rate both matter. A larger population explores more of the architecture space per generation but costs proportionally more compute to evaluate. A higher mutation rate explores more aggressively but risks discarding good architectures' traits before they can be exploited.
Try it yourself
The Neural Architecture Search Lab runs an evolutionary search against a synthetic fitness landscape with a hidden sweet-spot architecture, letting you watch generation by generation as the population discovers it.
🧪 Try it yourself: the Neural Architecture Search Lab simulation lets you experiment with everything described above directly in your browser.