Multi-Armed Bandit — Marketing Budget Allocator

Allocation Algorithm

Exploration rate ε 10%

Allocation speed 4 / s

Campaign Results

Budget units spent

Conversions won

Cumulative regret

0.00

Best channel found

—

Channels

Channel	Est. CVR	Spend

How it works

Each marketing channel is a "bandit arm" with a hidden true conversion rate. Every budget unit spent is a pull: the algorithm picks a channel, the environment returns a conversion (1) or not (0) with the arm's true probability, and the algorithm updates its running estimate of that arm's reward.

Q(a) ← Q(a) + (r − Q(a)) / n(a)      running mean update
regret += p*  − p(chosen arm)         per-pull opportunity cost

Epsilon-greedy: with probability ε pick a uniformly random channel (explore); otherwise pick the channel with the highest current estimate Q(a) (exploit).

UCB1: pick the channel maximizing an optimism bonus over the estimate —

UCB(a) = Q(a) + √(2·ln t / n(a))

the bonus shrinks as an arm is pulled more (n(a) grows), so UCB1 automatically explores under-tried channels early and converges onto the best one without a tunable rate.

Real-world relevance: this exact loop is how ad platforms (Google Ads, Meta) and A/B-testing tools reallocate budget across creatives or landing pages in production — cumulative regret is the standard metric for how much conversion revenue was lost to exploration versus a hindsight-optimal allocation.

Estimated conversion rate (bar height) Conversion won No conversion

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.