OpikBaseModel

class opik.evaluation.models.OpikBaseModel(model_name: str)

Bases: ABC

This class serves as an interface to LLMs.

If you want to implement a custom LLM provider in evaluation metrics, you should inherit from this class.

__init__(model_name: str)

Initializes the base model with a given model name.

Parameters:

model_name – The name of the LLM to be used.

abstractmethod generate_string(input: str, response_format: Type[BaseModel] | None = None, **kwargs: Any) → str

Simplified interface to generate a string output from the model.

Parameters:

• input – The input string based on which the model will generate the output.

• kwargs – Additional arguments that may be used by the model for string generation.

Returns:

The generated string output.

Return type:

str

abstractmethod generate_provider_response(messages: List[Dict[str, Any]], **kwargs: Any) → Any

Do not use this method directly. It is intended to be used within get_provider_response() method.

Generate a provider-specific response. Can be used to interface with the underlying model provider (e.g., OpenAI, Anthropic) and get raw output.

Parameters:

• messages – A list of messages to be sent to the model, should be a list of dictionaries with the keys

• kwargs – arguments required by the provider to generate a response.

Returns:

The response from the model provider, which can be of any type depending on the use case and LLM.

Return type:

Any

generate_chat_completion(messages: List[ConversationDict], response_format: Type[BaseModel] | None = None, **kwargs: Any) → ConversationDict

Generate the assistant turn from a list of role-tagged chat messages.

Implementations should forward the messages to the underlying chat-completions API verbatim. Preserving the caller’s system/user split is what allows providers to cache the stable system prefix across calls — which is the whole point of using this method instead of generate_string().

Parameters:

• messages – A list of {"role": ..., "content": ...} dictionaries following the OpenAI chat-completions shape.

• response_format – Optional Pydantic model specifying the expected output format.

• kwargs – Additional arguments forwarded to the underlying provider call.

Returns:

A {"role": "assistant", "content": ...} dict so callers can append it back onto the input messages for follow-up turns.

async agenerate_string(input: str, response_format: Type[BaseModel] | None = None, **kwargs: Any) → str

Simplified interface to generate a string output from the model. Async version.

Parameters:

• input – The input string based on which the model will generate the output.

• kwargs – Additional arguments that may be used by the model for string generation.

Returns:

The generated string output.

Return type:

str

async agenerate_provider_response(messages: List[Dict[str, Any]], **kwargs: Any) → Any

Do not use this method directly. It is intended to be used within aget_provider_response() method.

Generate a provider-specific response. Can be used to interface with the underlying model provider (e.g., OpenAI, Anthropic) and get raw output. Async version.

Parameters:

• messages – A list of messages to be sent to the model, should be a list of dictionaries with the keys “content” and “role”.

• kwargs – arguments required by the provider to generate a response.

Returns:

The response from the model provider, which can be of any type depending on the use case and LLM.

Return type:

Any

async agenerate_chat_completion(messages: List[ConversationDict], response_format: Type[BaseModel] | None = None, **kwargs: Any) → ConversationDict

Async counterpart of generate_chat_completion().