Edge AI Model Partitioning: Split-Point Simulator

Model

Split Point

Layers kept on-device 3 / 8

Hardware & Link

Device compute 1.0 GFLOP/s

Edge-server compute 12.0 GFLOP/s

Link bandwidth 50 Mbps

Live Result

Total latency: — ms

Device compute time

— ms

Server compute time

— ms

Transfer time

— ms

Optimal split

—

How it works

An on-device AI model can be partitioned: the first layers run locally, the rest run on a nearby edge server, with one activation tensor crossing the network at the cut. Unlike a whole-task offload decision, this is about where inside the model to cut:

Device compute time:  T_dev  = Σ FLOPs(layers before split) / P_device
Server compute time:  T_srv  = Σ FLOPs(layers after split)  / P_server
Transfer time:        T_xfer = ActivationSize(split) × 8 / Bandwidth
Total latency:         T = T_dev + T_xfer + T_srv

Each layer of the network has both a compute cost (FLOPs) and an output activation size (KB) that must be shipped across the network if that is where the cut falls. Early convolutional/embedding layers tend to have large activations but cheap compute — cutting there is expensive to transfer. Later layers compress the representation down to a small vector — cutting near the end is cheap to transfer but leaves little work for the (usually much faster) edge server.

Split point slider — 0 means every layer runs on the edge server (raw input is transferred); the maximum means the whole model runs on-device (nothing is transferred).
Auto-optimize — sweeps every possible split point and jumps to the one with the lowest total latency for the current hardware and link.
Raising device compute favors splitting later (keep more layers local); raising bandwidth favors splitting near a large-activation layer since the transfer penalty shrinks; raising server compute favors splitting earlier.

This is the real trade-off behind split computing / collaborative inference in mobile and IoT AI (e.g. Neurosurgeon-style layer partitioning): the same network, sliced at a different layer, can have a very different latency profile.

Runs on device Runs on edge server Activation transfer

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.