How Do Agentic LLMs Decide to Call Tools? A Scaffold Default Controlled by Suppression
Xijie Gong ⋅ Tingxu Han ⋅ Jiahao Zhang ⋅ Yuxin Cao ⋅ Ziqi Ding ⋅ Hanqi Yan ⋅ Youcheng Sun ⋅ Lijie Hu
Abstract
Tool calling, the ability to invoke external tools on demand, is central to agentic LLMs. Within an agent, the LLM decides whether and when to use these tools, greatly extending the agent’s capability boundaries. However, the mechanism that determines whether a model calls a tool or responds directly remains poorly understood. Agentic prompts are often long and heavily scaffolded, combining role instructions, tool schemas, format templates, and user requests across thousands of tokens. This creates a noisy and highly entangled context that makes it difficult to identify a single controllable variable for mechanistic analysis.To obtain such a variable, we propose a method to convert complex agentic prompts into minimal contrastive pairs, in which a single request verb determines the tool-call decision. Replacing an execution verb, such as \textit{write}, with an analysis verb, such as \textit{discuss}, reliably flips whether the model chooses to call a tool. This suggests that the choice is mediated by a compact internal state. 1,500 paired prompts across Python, Java, and C++, where 1,200 are for mechanistic analysis, and 300 are held out for evaluation, are constructed.We trace the tool-calling decision to a single vector, $\mu_\Delta$, and show that it is both causally necessary and sufficient.Transcoder analysis reveals how this vector forms: the scaffold makes tool calling the default, while analysis verbs suppress this default by activating features that signal no tool is needed. Execution verbs activate no analogous tool-use features, so $\mu_\Delta$ reflects the scaffold-induced default rather than a positive execution-verb signal. Downstream attention and MLP layers then read out this vector into the first output token. The mechanism replicates across diverse model families. Our code is available at \url{https://anonymous.4open.science/r/MI4ToolCalling}.
Chat is not available.
Successful Page Load