Function-calling LLMs insert an API call directly into their own generated text, then continue generating conditioned on the tool's returned result.
P(bind) ~ affinity * usefulness(token)
result_ready after 1/fillRate seconds
- Token candidates — generated token spans that could plausibly be replaced by a tool's output.
- API call sites — the fixed set of tools the model was fine-tuned to call (calculator, Wikipedia search, calendar...).
- Binding affinity — how strongly a candidate token is judged useful enough to trigger the matching API.
- Result-fill rate — how quickly the external API returns and the placeholder gets replaced by the real result.
Toolformer trains itself by sampling API calls, keeping only the ones that reduce perplexity on the following tokens — an entirely self-supervised way to learn when to call a tool.