Tool Calling Overview

​

What is Tool Calling

A lot of time, you would want your voice agent to take actions (for example: call an API to book appointment, or to transfer the call, get external knowledge). Right now you can achieve this with certain LLMs by describe functions and have the model intelligently choose to output a JSON object containing arguments to call one or many functions. This is also known as function calling, and these two terms are interchangeable.

We recommend reading this OpenAI documentation or the tutorial to understand what function calling is.

​

When to Use Tool Calling

You should implement tool calling to enhance your agent with capabilities such as booking appointments, transferring calls, or invoking your server’s functions, among others.

​

Where’s Tool Calls in API

In the API, you have two places to define tool calls:

• general tools: A list of tool calls that the model will have access to throughout the whole call.
• state tools: A list of tool calls that the model will have access to in a specific state (we will cover state in the next doc).

Apart from that, if you defined state edges (transition from one state to another), that’s achieved with function call as well.

At any given time, the model would have access to the following tools:

• Tools (with LLM state) = general tools + state tools + state transitions
• Tools (without state) = general tools

You have to make sure that the tools names are unique within all tools available to model at any given time.

For all tools, you can define a description that tells LLM what this function does and when it should be called.

​

Types of Tool Calls

There’re currently two types of tool calls:

• Pre-defined tools: These are tools that are defined by us and are available to all users. You can configure these so that it triggers at desired time, and with desired parameters.
  • End Call: end the conversation.
  • Transfer Call: transfer the call to another number.
  • Check event availability (cal.com): check the availability of an event on cal.com.
  • Book appointment (cal.com): book an appointment with specified event on cal.com.
• Custom tools: These are tools that you define yourself. You can define the function signature, and the model will call the function with the arguments you provide. You get to choose whether you want the agent to speak during function execution, and whether you want the function to be called in the background.

​

Custom Tool & Conversation Flow

When you define a custom tool, you can choose whether you want the agent to speak during the function execution, and whether you want the agent to speak after the function result is received.

​

When is Tool Called

The tool will get called when it’s agent’s turn to speak, and the model decides to call the tool. This does not gurantee the tool would only get called once across the conversation, so it’s important to make sure you can handle duplicate tool calling in your server.

When there’re tools being called, the model would not be able to make any new tool calls until the current tool call is completed. This can ensure the agent can handle interruptions and not get into a state where it’s calling the same tool over and over again when previous tool call is not completed.

​

Tool Calling Best Practices

• Make sure to write in your prompt when the tool should get called. Use the name of tool to refer to it. This helps the model understand when to call the tool, and have improved overall performance. For example:
  • "If user mentioned to check weather for any city, call function get_weather."
  • "Step 6: After all questions answered, call function end_call to hang up."
• Write a clear tool description, so that the model understands what the tool does.
• Be very specific about the parameter format, you can specify that in the description of parameters.
• Make sure your server can handle multiple tool calls at the same time, and can handle duplicate tool calls.
• Make sure your server responds quickly to tools. If the actual execution takes a long time, it’s okay to return a quick acknowledgement and run the tool in the background.