​
What is Function Calling

A lot of time, you would want your voice agent to take actions (for example: call an API to book appointment, or to transfer / end the call, get external knowledge) besides talking. Right now you can achieve this with certain LLMs by describe functions and have the model intelligently choose to output a JSON object containing arguments to call one or many functions. This is also known as tool use, and these two terms are interchangeable.

We recommend reading this OpenAI documentation to understand what function calling is. This guide we will use OpenAI function calling as an example, but feel free to take the idea and use with other models like Claude.

We will first dive into an easy use case of function calling (end the call), and then cover a more advanced appointment booking use case.

​
Case Study: End the Call Intelligently

Youtube tutorial to follow along:

The following steps take codes from Node.js Demo Repo / Python Demo Repo, it’s modified based on the LLM client class you created in the last guide.

Step 1

Define the Function

Note that for OpenAI, it would either give a tool call, or it would give a text response, but not both. Here we defined a message parameter in the tool call, so that when LLM decides to call this function, we can also have something to say to the user.

export interface FunctionCall {
  id: string;
  funcName: string;
  rguments: Record<string, any>;
  result?: string;
}

// some class boilerplate codes

private PrepareFunctions(): ChatCompletionsFunctionToolDefinition[] {
  let functions: ChatCompletionsFunctionToolDefinition[] = [{
    type: "function",
    function: {
      name: "end_call",
      description: "End the call only when user explicitly requests it.",
      parameters: {
        type: "object",
        properties: {
          message: {
            type: "string",
            description: "The message you will say before ending the call with the customer.",
          },
        },
        required: ["message"],
      },
    },
  }];
  return functions;
}

Step 2

Add your function calling into chat request and call it

const option: GetChatCompletionsOptions = {
  temperature: 0,
  maxTokens: 200,
  frequencyPenalty: 1,
  tools: this.PrepareFunctions(),
};

let events = await this.client.streamChatCompletions(
  process.env.AZURE_OPENAI_DEPLOYMENT_NAME,
  requestMessages,
  option,
);

Step 3

Extract the function calling arguments (if any) from the streaming response.

let funcCall: FunctionCall;
let funcArguments = "";

for await (const event of events) {
  if (event.choices.length >= 1) {
    let delta = event.choices[0].delta;
    if (!delta) continue;

    // If toolCalls is not empty, we know ChatGPT wants us to call the function
    if (delta.toolCalls.length >= 1) {
      const toolCall = delta.toolCalls[0];
      if (toolCall.id) {
        if (funcCall) {
          // As explained in the youtube video, another function received, old function complete, can break here.
          break;
        } else {
          funcCall = {
            id: toolCall.id,
            funcName: toolCall.function.name || "",
            arguments: {},
          };
        }
      } else {
        // append argument
        funcArguments += toolCall.function?.arguments || "";
      }
    } else if (delta.content) {
      const res: RetellResponse = {
        response_id: request.response_id,
        content: delta.content,
        content_complete: false,
        end_call: false,
      };
      ws.send(JSON.stringify(res));
    }
  }
}

Step 4

End the call when LLM suggested this function call.

if (funcCall != null) {
  if (funcCall.funcName === "end_call") {
    funcCall.arguments = JSON.parse(funcArguments);
    const res: RetellResponse = {
      response_id: request.response_id,
      content: funcCall.arguments.message,
      content_complete: true,
      end_call: true,
    };
    ws.send(JSON.stringify(res));
  }
}

​
Case Study: Make an Appointment

End call is a simple case to use function calling. In most case, you would like to your agent to say something while calling the function and say something after calling the function.

Check out Node Demo Code and below Youtube toturial for a simplified example of booking appointments.

Please note that this is not production ready yet, as in production, you need to make sure you don’t make duplicate function calls, you can still interat and handle interruptions of users, etc. For a more practical setting, a good practice is to have some internal states to decide and track what function to run, and influence how LLM responds.

We are working on adding a more practical example of function calling to our open source demo repos.

​
What is Function Calling

A lot of time, you would want your voice agent to take actions (for example: call an API to book appointment, or to transfer / end the call, get external knowledge) besides talking. Right now you can achieve this with certain LLMs by describe functions and have the model intelligently choose to output a JSON object containing arguments to call one or many functions. This is also known as tool use, and these two terms are interchangeable.

We recommend reading this OpenAI documentation to understand what function calling is. This guide we will use OpenAI function calling as an example, but feel free to take the idea and use with other models like Claude.

We will first dive into an easy use case of function calling (end the call), and then cover a more advanced appointment booking use case.

​
Case Study: End the Call Intelligently

Youtube tutorial to follow along:

The following steps take codes from Node.js Demo Repo / Python Demo Repo, it’s modified based on the LLM client class you created in the last guide.

Step 1

Define the Function

Note that for OpenAI, it would either give a tool call, or it would give a text response, but not both. Here we defined a message parameter in the tool call, so that when LLM decides to call this function, we can also have something to say to the user.

export interface FunctionCall {
  id: string;
  funcName: string;
  rguments: Record<string, any>;
  result?: string;
}

// some class boilerplate codes

private PrepareFunctions(): ChatCompletionsFunctionToolDefinition[] {
  let functions: ChatCompletionsFunctionToolDefinition[] = [{
    type: "function",
    function: {
      name: "end_call",
      description: "End the call only when user explicitly requests it.",
      parameters: {
        type: "object",
        properties: {
          message: {
            type: "string",
            description: "The message you will say before ending the call with the customer.",
          },
        },
        required: ["message"],
      },
    },
  }];
  return functions;
}

Step 2

Add your function calling into chat request and call it

const option: GetChatCompletionsOptions = {
  temperature: 0,
  maxTokens: 200,
  frequencyPenalty: 1,
  tools: this.PrepareFunctions(),
};

let events = await this.client.streamChatCompletions(
  process.env.AZURE_OPENAI_DEPLOYMENT_NAME,
  requestMessages,
  option,
);

Step 3

Extract the function calling arguments (if any) from the streaming response.

let funcCall: FunctionCall;
let funcArguments = "";

for await (const event of events) {
  if (event.choices.length >= 1) {
    let delta = event.choices[0].delta;
    if (!delta) continue;

    // If toolCalls is not empty, we know ChatGPT wants us to call the function
    if (delta.toolCalls.length >= 1) {
      const toolCall = delta.toolCalls[0];
      if (toolCall.id) {
        if (funcCall) {
          // As explained in the youtube video, another function received, old function complete, can break here.
          break;
        } else {
          funcCall = {
            id: toolCall.id,
            funcName: toolCall.function.name || "",
            arguments: {},
          };
        }
      } else {
        // append argument
        funcArguments += toolCall.function?.arguments || "";
      }
    } else if (delta.content) {
      const res: RetellResponse = {
        response_id: request.response_id,
        content: delta.content,
        content_complete: false,
        end_call: false,
      };
      ws.send(JSON.stringify(res));
    }
  }
}

Step 4

End the call when LLM suggested this function call.

if (funcCall != null) {
  if (funcCall.funcName === "end_call") {
    funcCall.arguments = JSON.parse(funcArguments);
    const res: RetellResponse = {
      response_id: request.response_id,
      content: funcCall.arguments.message,
      content_complete: true,
      end_call: true,
    };
    ws.send(JSON.stringify(res));
  }
}

​
Case Study: Make an Appointment

End call is a simple case to use function calling. In most case, you would like to your agent to say something while calling the function and say something after calling the function.

Check out Node Demo Code and below Youtube toturial for a simplified example of booking appointments.

Please note that this is not production ready yet, as in production, you need to make sure you don’t make duplicate function calls, you can still interat and handle interruptions of users, etc. For a more practical setting, a good practice is to have some internal states to decide and track what function to run, and influence how LLM responds.

We are working on adding a more practical example of function calling to our open source demo repos.