Configure LLM options

Some LLM models can be configured with different options to influence the output response for the voice agent. Not all options are supported by all models. Please refer to the dashboard for more details.

​

LLM Temperature

Temperature is a value that controls the randomness of the output response. A lower temperature will make the output more deterministic and consistent, while a higher temperature will make the output more random and creative. For agents that require precise function calling and consistent instruction following, we recommend setting a lower temperature. For agents that require more creative and flexible responses, we recommend setting a higher temperature.

​

Structured Output

While all the LLM options provided support function calling, sometimes the required arguments are missing due to LLM is not confined to the pre-defined schema. With this option turned on, it will enforce the output to be in the pre-defined schema, yielding a more reliable output for function calling.

The only drawback of this option is that it needs to cache the schema in system, and thus your agent auto save might be slower.

​

High Priority

Sometimes you might observe that LLM latency can vary from time to time, and although Retell has done a lot of optimizations on LLM call routing and fallback, there might be cases you don’t want to deal with the variance. This option here will direct your LLM calls to a more dedicated resource pool, to ensure higher availability and more consistent latency.

Please note that this option comes at a cost, as the LLM will be charged at 2x of its normal price.