Multi-Objective Optimization Mathematics

Explore multi-objective optimization mathematics. Learn about Pareto optimality, Pareto fronts, and multi-objective optimization methods.

▶ Open the simulation

Introduction

Multi-objective optimization considers multiple objectives simultaneously, leading to Pareto-optimal solutions rather than a single optimum. This is common in hyperparameter optimization when balancing accuracy, model size, and training time.

Multi-Objective Problem

Formulation

Optimize multiple objectives:

min_λ [L₁(λ), L₂(λ), ..., Lₘ(λ)]

Where m is number of objectives.

Pareto Optimality

Dominance

Solution λ₁ dominates λ₂ if:

L_i(λ₁) ≤ L_i(λ₂) for all i
L_j(λ₁) < L_j(λ₂) for at least one j

Pareto Optimal

Solution is Pareto optimal if no other solution dominates it:

∄ λ' : L_i(λ') ≤ L_i(λ*) ∀i and L_j(λ') < L_j(λ*) for some j

Pareto Front

Definition

Set of all Pareto-optimal solutions:

PF = {λ* : λ* is Pareto optimal}

Properties

Trade-off surface
Continuous or discrete
Convex or non-convex
High-dimensional

Scalarization Methods

Weighted Sum

min_λ Σ w_i L_i(λ)

Where Σ w_i = 1, w_i ≥ 0.

Weighted Chebyshev

min_λ max_i w_i |L_i(λ) - L_i*|

Evolutionary Multi-Objective

NSGA-II

Non-dominated Sorting Genetic Algorithm:

Non-dominated sorting
Crowding distance
Maintains diversity

MOEA/D

Decomposition-based approach:

Decomposes into subproblems
Collaborative optimization
Efficient for many objectives

Key Insight

Multi-objective optimization finds trade-off solutions rather than single optimum. Pareto front represents all optimal trade-offs, enabling decision-makers to choose based on preferences.

Hypervolume Indicator

Definition

Volume of space dominated by Pareto front:

HV = Volume({y : y dominated by PF})

Properties

Larger is better
Captures both convergence and diversity
Computationally expensive

Pareto Efficiency

Efficiency Frontiers

Boundary of achievable objective combinations:

Optimal trade-offs
No free lunch
Trade-off analysis

Frequently Asked Questions

What is Pareto optimality?

Pareto optimality means no other solution dominates it: no solution is better in all objectives and strictly better in at least one. Pareto-optimal solutions represent optimal trade-offs.

What is a Pareto front?

Pareto front is the set of all Pareto-optimal solutions. It represents the trade-off surface between objectives, showing all optimal combinations and trade-offs.

How do I solve multi-objective problems?

Use scalarization (weighted sum), Pareto-based methods (NSGA-II), or decomposition (MOEA/D). Choice depends on number of objectives, problem structure, and desired solutions.

What is dominance in multi-objective optimization?

Solution λ₁ dominates λ₂ if it's better or equal in all objectives and strictly better in at least one. Dominated solutions are suboptimal.

Can I combine multiple objectives into one?

Yes, using weighted sum scalarization: min Σ w_i L_i(λ). However, this loses information about trade-offs. Pareto methods preserve full trade-off information.