Generation 1 - Candidate Architectures
NAS Method
Search Space
Actions
Neural Architecture Search (NAS)
NAS automates the design of neural network architectures. Instead of human experts manually designing networks, algorithms search through architecture space to find optimal designs.
Search Methods
- Evolutionary Algorithms: Mutate and select best architectures
- Reinforcement Learning: Agent learns to generate good architectures
- Gradient-Based (DARTS): Differentiable architecture search
- Bayesian Optimization: Model performance, query intelligently
Famous NAS Results
- NASNet: Discovered by RL, beat human designs
- EfficientNet: Compound scaling via NAS
- AmoebaNet: Evolutionary search
- DARTS: Fast differentiable NAS
Challenges
- Extremely expensive computationally (1000s of GPU hours)
- Search space design critical
- Evaluation accuracy vs efficiency trade-off
- Transfer to new tasks uncertain
Practical Usage
- Most practitioners use discovered architectures, not run NAS
- EfficientNet, NASNet available pre-trained
- AutoML platforms (Google Cloud AutoML) run NAS for you
NAS represents the future: AI systems that improve themselves!