Experiments Client

The Experiments client provides methods for managing experiments in the Opik platform.

class opik.rest_api.experiments.client.ExperimentsClient(*, client_wrapper: SyncClientWrapper)

Bases: object

batch_update_experiments(*, ids: Sequence[str], update: ExperimentUpdate, merge_tags: bool | None = OMIT, request_options: RequestOptions | None = None) → None

Update multiple experiments

Parameters:

  • ids (Sequence[str]) – List of experiment IDs to update (max 1000)

  • update (ExperimentUpdate)

  • merge_tags (Optional[bool]) – If true, merge tags with existing tags instead of replacing them. Default: false

  • request_options (Optional[RequestOptions]) – Request-specific configuration.

Return type:

None

find_experiments(*, page: int | None = None, size: int | None = None, dataset_id: str | None = None, optimization_id: str | None = None, types: str | None = None, name: str | None = None, dataset_deleted: bool | None = None, prompt_id: str | None = None, project_id: str | None = None, project_deleted: bool | None = None, sorting: str | None = None, filters: str | None = None, experiment_ids: str | None = None, force_sorting: bool | None = None, request_options: RequestOptions | None = None) → ExperimentPagePublic

Find experiments

Parameters:

  • page (Optional[int])

  • size (Optional[int])

  • dataset_id (Optional[str])

  • optimization_id (Optional[str])

  • types (Optional[str])

  • name (Optional[str])

  • dataset_deleted (Optional[bool])

  • prompt_id (Optional[str])

  • project_id (Optional[str])

  • project_deleted (Optional[bool])

  • sorting (Optional[str])

  • filters (Optional[str])

  • experiment_ids (Optional[str])

  • force_sorting (Optional[bool])

  • request_options (Optional[RequestOptions]) – Request-specific configuration.

Returns:

Experiments resource

Return type:

ExperimentPagePublic

create_experiment(*, id: str | None = OMIT, dataset_name: str | None = OMIT, project_id: str | None = OMIT, project_name: str | None = OMIT, name: str | None = OMIT, metadata: Dict[str, Any | None] | List[Dict[str, Any | None]] | str | None = OMIT, tags: Sequence[str] | None = OMIT, type: Literal['regular', 'trial', 'mini-batch', 'mutation'] | Any | None = OMIT, evaluation_method: Literal['dataset', 'evaluation_suite'] | Any | None = OMIT, optimization_id: str | None = OMIT, status: Literal['running', 'completed', 'cancelled'] | Any | None = OMIT, experiment_scores: Sequence[ExperimentScoreWrite] | None = OMIT, prompt_version: PromptVersionLinkWrite | None = OMIT, prompt_versions: Sequence[PromptVersionLinkWrite] | None = OMIT, dataset_version_id: str | None = OMIT, request_options: RequestOptions | None = None) → None

Create experiment

Parameters:

  • id (Optional[str])

  • dataset_name (Optional[str])

  • project_id (Optional[str]) – Project ID. Takes precedence over project_name when both are provided.

  • project_name (Optional[str]) – Project name. Creates project if it doesn’t exist. Ignored when project_id is provided.

  • name (Optional[str])

  • metadata (Optional[JsonListStringWrite])

  • tags (Optional[Sequence[str]])

  • type (Optional[ExperimentWriteType])

  • evaluation_method (Optional[ExperimentWriteEvaluationMethod])

  • optimization_id (Optional[str])

  • status (Optional[ExperimentWriteStatus])

  • experiment_scores (Optional[Sequence[ExperimentScoreWrite]])

  • prompt_version (Optional[PromptVersionLinkWrite])

  • prompt_versions (Optional[Sequence[PromptVersionLinkWrite]])

  • dataset_version_id (Optional[str]) – ID of the dataset version this experiment is linked to. If not provided at creation, experiment will be automatically linked to the latest version.

  • request_options (Optional[RequestOptions]) – Request-specific configuration.

Return type:

None

create_experiment_items(*, experiment_items: Sequence[ExperimentItem], request_options: RequestOptions | None = None) → None

Create experiment items

Parameters:

  • experiment_items (Sequence[ExperimentItem])

  • request_options (Optional[RequestOptions]) – Request-specific configuration.

Return type:

None

delete_experiment_items(*, ids: Sequence[str], request_options: RequestOptions | None = None) → None

Delete experiment items

Parameters:

  • ids (Sequence[str])

  • request_options (Optional[RequestOptions]) – Request-specific configuration.

Return type:

None

delete_experiments_by_id(*, ids: Sequence[str], request_options: RequestOptions | None = None) → None

Delete experiments by id

Parameters:

  • ids (Sequence[str])

  • request_options (Optional[RequestOptions]) – Request-specific configuration.

Return type:

None

execute_experiment(*, dataset_name: str, prompts: Sequence[PromptVariant], dataset_id: str, dataset_version_id: str | None = OMIT, project_name: str | None = OMIT, version_hash: str | None = OMIT, prompt_versions: Sequence[PromptVersionLink] | None = OMIT, request_options: RequestOptions | None = None) → ExperimentExecutionResponse

Creates experiments for each prompt variant and asynchronously processes all dataset items

Parameters:

  • dataset_name (str)

  • prompts (Sequence[PromptVariant])

  • dataset_id (str)

  • dataset_version_id (Optional[str])

  • project_name (Optional[str])

  • version_hash (Optional[str])

  • prompt_versions (Optional[Sequence[PromptVersionLink]])

  • request_options (Optional[RequestOptions]) – Request-specific configuration.

Returns:

Experiments created and processing started

Return type:

ExperimentExecutionResponse

experiment_items_bulk(*, experiment_name: str, dataset_name: str, items: Sequence[ExperimentItemBulkRecordExperimentItemBulkWriteView], experiment_id: str | None = OMIT, project_name: str | None = OMIT, request_options: RequestOptions | None = None) → None

Record experiment items in bulk with traces, spans, and feedback scores. Maximum request size is 4MB.

Parameters:

  • experiment_name (str)

  • dataset_name (str)

  • items (Sequence[ExperimentItemBulkRecordExperimentItemBulkWriteView])

  • experiment_id (Optional[str]) – Optional experiment ID. If provided, items will be added to the existing experiment and experimentName will be ignored. If not provided or experiment with that ID doesn’t exist, a new experiment will be created with the given experimentName

  • project_name (Optional[str]) – Project for traces auto-created from items that provide evaluate_task_result (i.e. without an explicit trace). If null, the default project is used; relying on this fallback is deprecated, please provide project_name explicitly.

  • request_options (Optional[RequestOptions]) – Request-specific configuration.

Return type:

None

find_feedback_score_names(*, experiment_ids: str | None = None, project_id: str | None = None, request_options: RequestOptions | None = None) → FeedbackScoreNamesPublic

Find Feedback Score names

Parameters:

  • experiment_ids (Optional[str])

  • project_id (Optional[str])

  • request_options (Optional[RequestOptions]) – Request-specific configuration.

Returns:

Feedback Scores resource

Return type:

FeedbackScoreNamesPublic

find_experiment_groups(*, groups: str | None = None, types: str | None = None, name: str | None = None, project_id: str | None = None, project_deleted: bool | None = None, filters: str | None = None, request_options: RequestOptions | None = None) → ExperimentGroupResponse

Find experiments grouped by specified fields

Parameters:

  • groups (Optional[str])

  • types (Optional[str])

  • name (Optional[str])

  • project_id (Optional[str])

  • project_deleted (Optional[bool])

  • filters (Optional[str])

  • request_options (Optional[RequestOptions]) – Request-specific configuration.

Returns:

Experiment groups

Return type:

ExperimentGroupResponse

find_experiment_groups_aggregations(*, groups: str | None = None, types: str | None = None, name: str | None = None, project_id: str | None = None, project_deleted: bool | None = None, filters: str | None = None, request_options: RequestOptions | None = None) → ExperimentGroupAggregationsResponse

Find experiments grouped by specified fields with aggregation metrics

Parameters:

  • groups (Optional[str])

  • types (Optional[str])

  • name (Optional[str])

  • project_id (Optional[str])

  • project_deleted (Optional[bool])

  • filters (Optional[str])

  • request_options (Optional[RequestOptions]) – Request-specific configuration.

Returns:

Experiment groups with aggregations

Return type:

ExperimentGroupAggregationsResponse

finish_experiments(*, ids: Sequence[str], request_options: RequestOptions | None = None) → None

Finish experiments and trigger alert events

Parameters:

  • ids (Sequence[str])

  • request_options (Optional[RequestOptions]) – Request-specific configuration.

Return type:

None

get_experiment_by_id(id: str, *, request_options: RequestOptions | None = None) → ExperimentPublic

Get experiment by id

Parameters:

  • id (str)

  • request_options (Optional[RequestOptions]) – Request-specific configuration.

Returns:

Experiment resource

Return type:

ExperimentPublic

update_experiment(id: str, *, name: str | None = OMIT, metadata: Dict[str, Any | None] | None = OMIT, tags: Sequence[str] | None = OMIT, tags_to_add: Sequence[str] | None = OMIT, tags_to_remove: Sequence[str] | None = OMIT, type: Literal['regular', 'trial', 'mini-batch', 'mutation'] | Any | None = OMIT, status: Literal['running', 'completed', 'cancelled'] | Any | None = OMIT, experiment_scores: Sequence[ExperimentScore] | None = OMIT, request_options: RequestOptions | None = None) → None

Update experiment by id

Parameters:

  • id (str)

  • name (Optional[str])

  • metadata (Optional[JsonNode])

  • tags (Optional[Sequence[str]]) – Tags

  • tags_to_add (Optional[Sequence[str]]) – Tags to add

  • tags_to_remove (Optional[Sequence[str]]) – Tags to remove

  • type (Optional[ExperimentUpdateType])

  • status (Optional[ExperimentUpdateStatus]) – The status of the experiment

  • experiment_scores (Optional[Sequence[ExperimentScore]])

  • request_options (Optional[RequestOptions]) – Request-specific configuration.

Return type:

None

get_experiment_item_by_id(id: str, *, request_options: RequestOptions | None = None) → ExperimentItemPublic

Get experiment item by id

Parameters:

  • id (str)

  • request_options (Optional[RequestOptions]) – Request-specific configuration.

Returns:

Experiment item resource

Return type:

ExperimentItemPublic

stream_experiment_items(*, experiment_name: str, limit: int | None = OMIT, last_retrieved_id: str | None = OMIT, truncate: bool | None = OMIT, project_name: str | None = OMIT, request_options: RequestOptions | None = None) → Iterator[bytes]

Stream experiment items

Parameters:

  • experiment_name (str)

  • limit (Optional[int])

  • last_retrieved_id (Optional[str])

  • truncate (Optional[bool]) – Truncate image included in either input, output or metadata

  • project_name (Optional[str])

  • request_options (Optional[RequestOptions]) – Request-specific configuration. You can pass in configuration such as chunk_size, and more to customize the request and response.

Returns:

Experiment items stream or error during process

Return type:

Iterator[bytes]

stream_experiments(*, name: str, limit: int | None = OMIT, last_retrieved_id: str | None = OMIT, project_name: str | None = OMIT, request_options: RequestOptions | None = None) → Iterator[bytes]

Stream experiments

Parameters:

  • name (str)

  • limit (Optional[int])

  • last_retrieved_id (Optional[str])

  • project_name (Optional[str])

  • request_options (Optional[RequestOptions]) – Request-specific configuration. You can pass in configuration such as chunk_size, and more to customize the request and response.

Returns:

Experiments stream or error during process

Return type:

Iterator[bytes]

Usage Example

import opik

client = opik.Opik()

# Find experiments
experiments = client.rest_client.experiments.find_experiments(
    page=0,
    size=10
)

# Get an experiment by ID
experiment = client.rest_client.experiments.get_experiment_by_id("experiment-id")

# Create a new experiment
client.rest_client.experiments.create_experiment(
    name="my-experiment",
    dataset_name="my-dataset"
)

# Stream experiment items
items_generator = client.rest_client.experiments.stream_experiment_items(
    experiment_id="experiment-id"
)