Class ChatParams

Optional cancellation token to abort the chat completion request. When this signal is aborted, the provider should cancel the request and return a cancelled result as gracefully as possible.

If the model supports effort levels, this parameter can be used to specify the effort level.

Whether to enable caching for this request. Implementation depends on the specific provider (the below are examples, many other providers exist):

• For Anthropic: Uses Anthropic's ephemeral cache control to cache system prompt and last user message.
• For OpenAI: Uses automatic caching (provider handles it).
• For other providers: May be a no-op if caching isn't supported.

Default

true - Caching is enabled by default for providers that support it.
Copy

Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.

Not all AI providers support this feature. When supported, this parameter indicates if logprobs are requested. Logprobs provide information about the likelihood of each token in the response. This can be useful for debugging or understanding the model's behavior. When models support this and this property is set to true, the model will return logprobs for the tokens in the

See

• ChatResultData object within the array of
• ChatResultChoice objects.

Array of messages, allows full control over the order and content of the conversation.

Minimum probability threshold for token sampling (0-1). Tokens with probability below this threshold are filtered out before sampling. This is a newer parameter not yet widely supported.

Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.

Whether to use streaming for this request. If true and the provider supports streaming, responses will be streamed. If true but the provider doesn't support streaming, the request will fall back to non-streaming.

Callbacks for streaming responses. Only used when streaming is true.

Top-k sampling parameter. Limits the model to only sample from the top K most likely tokens at each step. For example, k=50 means the model will only consider the 50 most likely next tokens. Not supported by all providers (e.g., OpenAI doesn't support this).

Number of top log probabilities to return per token. Only used when includeLogProbs is true. Typically ranges from 2-20, depending on the provider.

Top-p (nucleus) sampling parameter (0-1). An alternative to temperature sampling that considers the smallest set of tokens whose cumulative probability exceeds the probability p. For example, 0.1 means only the tokens comprising the top 10% probability mass are considered. Generally, use either temperature OR top-p, not both.