> ## Documentation Index
> Fetch the complete documentation index at: https://docs.retellai.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Integrate Function Calling
> Let your voice agent take actions.
### What is Function Calling
A lot of time, you would want your voice agent to take actions (for example: call an API to book appointment,
or to transfer / end the call, get external knowledge) besides talking.
Right now you can achieve this with certain LLMs by
describe functions and have the model intelligently choose to output a JSON object containing arguments to
call one or many functions.
This is also known as tool use, and these two terms are interchangeable.
We recommend reading this [OpenAI documentation](https://platform.openai.com/docs/guides/function-calling) to
understand what function calling is. This guide we will use OpenAI function calling as an example, but feel
free to take the idea and use with other models like Claude.
We will first dive into an easy use case of function calling (end the call), and then cover a more
advanced appointment booking use case.
### Case Study: End the Call Intelligently
YouTube tutorial to follow along:
The following steps take codes from
[Node.js Demo Repo](https://github.com/RetellAI/retell-backend-node-demo/blob/main/src/llm_azure_openai_func_call_end_call.ts)
/ [Python Demo Repo](https://github.com/RetellAI/python-backend-demo/blob/main/llm_with_func_calling.py),
it's modified based on the LLM client class you created in the last guide.
**Step 1**
Define the Function
Note that for OpenAI, it would either give a tool call, or it would give a text response, but not both.
Here we defined a `message` parameter in the tool call, so that when LLM decides to call this function,
we can also have something to say to the user.
```typescript Node.js theme={null}
export interface FunctionCall {
id: string;
funcName: string;
arguments: Record;
result?: string;
}
// some class boilerplate codes
private PrepareFunctions(): ChatCompletionsFunctionToolDefinition[] {
let functions: ChatCompletionsFunctionToolDefinition[] = [{
type: "function",
function: {
name: "end_call",
description: "End the call only when user explicitly requests it.",
parameters: {
type: "object",
properties: {
message: {
type: "string",
description: "The message you will say before ending the call with the customer.",
},
},
required: ["message"],
},
},
}];
return functions;
}
```
```Python Python theme={null}
# some class boilerplate codes
def prepare_functions(self):
functions= [
{
"type": "function",
"function": {
"name": "end_call",
"description": "End the call only when user explicitly requests it.",
"parameters": {
"type": "object",
"properties": {
"message": {
"type": "string",
"description": "The message you will say before ending the call with the customer.",
},
},
"required": ["message"],
},
},
},
]
return functions
```
**Step 2**
Add your `function calling` into chat request and call it
```javascript Node.js theme={null}
const option: GetChatCompletionsOptions = {
temperature: 0,
maxTokens: 200,
frequencyPenalty: 1,
tools: this.PrepareFunctions(),
};
let events = await this.client.streamChatCompletions(
process.env.AZURE_OPENAI_DEPLOYMENT_NAME,
requestMessages,
option,
);
```
```Python Python theme={null}
stream = self.client.chat.completions.create(
model="gpt-3.5-turbo-1106",
messages=prompt,
stream=True,
# Step 2: Add the function into your request
tools=self.prepare_functions()
)
```
**Step 3**
Extract the function calling arguments (if any) from the streaming response.
```javascript Node.js theme={null}
let funcCall: FunctionCall;
let funcArguments = "";
for await (const event of events) {
if (event.choices.length >= 1) {
let delta = event.choices[0].delta;
if (!delta) continue;
// If toolCalls is not empty, we know ChatGPT wants us to call the function
if (delta.toolCalls.length >= 1) {
const toolCall = delta.toolCalls[0];
if (toolCall.id) {
if (funcCall) {
// As explained in the youtube video, another function received, old function complete, can break here.
break;
} else {
funcCall = {
id: toolCall.id,
funcName: toolCall.function.name || "",
arguments: {},
};
}
} else {
// append argument
funcArguments += toolCall.function?.arguments || "";
}
} else if (delta.content) {
const res: RetellResponse = {
response_id: request.response_id,
content: delta.content,
content_complete: false,
end_call: false,
};
ws.send(JSON.stringify(res));
}
}
}
```
```Python Python theme={null}
func_call = {}
func_arguments = ""
for chunk in stream:
if chunk.choices[0].delta.tool_calls:
tool_calls = chunk.choices[0].delta.tool_calls[0]
if tool_calls.id:
if func_call:
# Another function received, old function complete, can break here.
break
func_call = {
"id": tool_calls.id,
"func_name": tool_calls.function.name or "",
"arguments": {},
}
else:
# append argument
func_arguments += tool_calls.function.arguments or ""
```
**Step 4**
End the call when LLM suggested this function call.
```javascript Node.js theme={null}
if (funcCall != null) {
if (funcCall.funcName === "end_call") {
funcCall.arguments = JSON.parse(funcArguments);
const res: RetellResponse = {
response_id: request.response_id,
content: funcCall.arguments.message,
content_complete: true,
end_call: true,
};
ws.send(JSON.stringify(res));
}
}
```
```Python Python theme={null}
if func_call:
if func_call['func_name'] == "end_call":
func_call['arguments'] = json.loads(func_arguments)
yield {
"response_id": request['response_id'],
"content": func_call['arguments']['message'],
"content_complete": True,
"end_call": True,
}
```
### Case Study: Make an Appointment
End call is a simple case to use function calling. In most case, you would like to your agent to say
something while calling the function and say something after calling the function.
Check out
[Node Demo Code](https://github.com/RetellAI/retell-backend-node-demo/blob/main/src/llm_azure_openai_func_call.ts)
and below YouTube tutorial for a simplified example of booking appointments.
Please note that this is not production ready yet, as in production, you need to make sure you don't make
duplicate function calls, you can still interact and handle interruptions of users, etc. For a more practical
setting, a good practice is to have some internal states to decide and track what function to run, and influence
how LLM responds.
We are working on adding a more practical example of function calling to our open source demo repos.