HOW TOOL CALLING WORKS IN LLMS

Summary from the TLDR InfoSec newsletter

LLMs use tools, external functions, or APIs when a user makes a request that the model cannot fulfill given its internal knowledge base. During inference, an LLM is provided with a series of tool definitions as part of its input prompt or context window, much like providing a human with a series of external functions and their documentation. When the LLM detects that a tool must be called, it ranks potential tools based on their relevance to the task and then calls the appropriate tool to address the user's query.