Skip to content

comet_ml.Experiment

Experiment is a unit of measurable research that defines a single run with some data/parameters/code/results.

Creating an Experiment object in your code will report a new experiment to your Comet.ml project. Your Experiment will automatically track and collect many things and will also allow you to manually report anything.

You can create multiple objects in one script (such as when looping over multiple hyper parameters).

Attributes

artifact_name_max_length instance-attribute

artifact_name_max_length: Optional[int] = None

asset_name_max_length instance-attribute

asset_name_max_length: Optional[int] = None

image_name_max_length instance-attribute

image_name_max_length: Optional[int] = None

url property

url

Get the url of the experiment.

Example
1
2
3
4
5
import comet_ml

comet_ml.login()
exp = comet_ml.Experiment()
print(exp.url)

Functions

__init__

__init__(
    api_key: Optional[str] = None,
    project_name: Optional[str] = None,
    workspace: Optional[str] = None,
    log_code: Optional[bool] = True,
    log_graph: Optional[bool] = True,
    auto_param_logging: Optional[bool] = True,
    auto_metric_logging: Optional[bool] = True,
    parse_args: Optional[bool] = True,
    auto_output_logging: Optional[str] = "default",
    log_env_details: Optional[bool] = True,
    log_git_metadata: Optional[bool] = True,
    log_git_patch: Optional[bool] = True,
    disabled: Optional[bool] = False,
    log_env_gpu: Optional[bool] = True,
    log_env_host: Optional[bool] = True,
    display_summary: Optional[bool] = None,
    log_env_cpu: Optional[bool] = True,
    log_env_network: Optional[bool] = True,
    log_env_disk: Optional[bool] = True,
    display_summary_level: Optional[int] = None,
    optimizer_data: Optional[Dict[str, Any]] = None,
    auto_weight_logging: Optional[bool] = None,
    auto_log_co2: Optional[bool] = True,
    auto_metric_step_rate: Optional[int] = 10,
    auto_histogram_tensorboard_logging: Optional[bool] = False,
    auto_histogram_epoch_rate: Optional[int] = 1,
    auto_histogram_weight_logging: Optional[bool] = False,
    auto_histogram_gradient_logging: Optional[bool] = False,
    auto_histogram_activation_logging: Optional[bool] = False,
    experiment_key: Optional[str] = None,
    distributed_node_identifier: Optional[str] = None,
) -> None

Creates a new experiment on Comet.

Parameters:

  • api_key (str, default: None ) –

    Your API key obtained from comet.com

  • project_name (str, default: None ) –

    Send your experiment to a specific project. Otherwise will be sent to Uncategorized Experiments. If project name does not already exists Comet will create a new project.

  • workspace (str, default: None ) –

    Attach an experiment to a project that belongs to this workspace

  • log_code (bool, default: True ) –

    Allows you to enable/disable code logging

  • log_graph (bool, default: True ) –

    Allows you to enable/disable automatic computation graph logging.

  • auto_param_logging (bool, default: True ) –

    Allows you to enable/disable hyper parameters logging

  • auto_metric_logging (bool, default: True ) –

    Allows you to enable/disable metrics logging

  • auto_metric_step_rate (int, default: 10 ) –

    Controls how often batch metrics are logged

  • auto_histogram_tensorboard_logging (bool, default: False ) –

    Allows you to enable/disable automatic tensorboard histogram logging

  • auto_histogram_epoch_rate (int, default: 1 ) –

    Controls how often histograms are logged

  • auto_histogram_weight_logging (bool, default: False ) –

    Allows you to enable/disable histogram logging for biases and weights

  • auto_histogram_gradient_logging (bool, default: False ) –

    Allows you to enable/disable automatic histogram logging of gradients

  • auto_histogram_activation_logging (bool, default: False ) –

    Allows you to enable/disable automatic histogram logging of activations

  • auto_output_logging (str, default: 'default' ) –

    Allows you to select which output logging mode to use. You can pass "native" which will log all output even when it originated from a C native library. You can also pass "simple" which will work only for output made by Python code. If you want to disable automatic output logging, you can pass False. The default is "default" which will detect your environment and deactivate the output logging for IPython and Jupyter environment and sets "native" in the other cases.

  • auto_log_co2 (bool, default: True ) –

    Automatically tracks the CO2 emission of this experiment if codecarbon package is installed in the environment

  • parse_args (bool, default: True ) –

    Allows you to enable/disable automatic parsing of CLI arguments

  • log_env_details (bool, default: True ) –

    Log various environment information in order to identify where the script is running

  • log_env_gpu (bool, default: True ) –

    Allow you to enable/disable the automatic collection of gpu details and metrics (utilization, memory usage etc..). log_env_details must also be true.

  • log_env_cpu (bool, default: True ) –

    Allow you to enable/disable the automatic collection of cpu details and metrics (utilization, memory usage etc..). log_env_details must also be true.

  • log_env_network (bool, default: True ) –

    Allow you to enable/disable the automatic collection of network details and metrics (sent, receive rates, etc..). log_env_details must also be true.

  • log_env_disk (bool, default: True ) –

    Allow you to enable/disable the automatic collection of disk utilization metrics (usage, IO rates, etc.). log_env_details must also be true.

  • log_env_host (bool, default: True ) –

    Allow you to enable/disable the automatic collection of host information (ip, hostname, python version, user etc...). log_env_details must also be true.

  • log_git_metadata (bool, default: True ) –

    Allow you to enable/disable the automatic collection of git details

  • log_git_patch (bool, default: True ) –

    Allow you to enable/disable the automatic collection of git patch

  • display_summary_level (int, default: None ) –

    Control the summary detail that is displayed on the console at end of experiment. If 0, the summary notification is still sent. Valid values are 0 to 2.

  • disabled (bool, default: False ) –

    Allows you to disable all network communication with the Comet backend. It is useful when you want to test to make sure everything is working, without actually logging anything.

  • experiment_key (str, default: None ) –

    If provided, will be used as the experiment key. If an experiment with the same key already exists, it will raises an Exception. Could be set through configuration as well. Must be an alphanumeric string whose length is between 32 and 50 characters.

add_tag

add_tag(tag: str) -> None

Add a tag to the experiment. Tags will be shown in the dashboard.

Parameters:

  • tag (str) –

    A tag to add to the experiment.

add_tags

add_tags(tags: List[str]) -> None

Add several tags to the experiment. Tags will be shown in the dashboard.

Parameters:

  • tags (List[str]) –

    Tags list to add to the experiment.

clean

clean()

Clean the experiment loggers, useful in case you want to debug your scripts with IPDB.

context_manager

context_manager(context)

A context manager to mark the beginning and the end of the training phase. This allows you to provide a namespace for metrics/params. For example:

Parameters:

  • context (str) –

    Name of the context manager.

Example
1
2
3
4
5
6
7
8
import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

# Log validation_accuracy using the context manager
with exp.context_manager("validation"):
    exp.log_metric("accuracy", 0.12)

create_confusion_matrix

create_confusion_matrix(
    y_true=None,
    y_predicted=None,
    labels=None,
    matrix=None,
    title="Confusion Matrix",
    row_label="Actual Category",
    column_label="Predicted Category",
    max_examples_per_cell=25,
    max_categories=25,
    winner_function=None,
    index_to_example_function=None,
    cache=True,
    selected=None,
    images=None,
    **kwargs
)

Create a confusion matrix for use over multiple epochs.

Parameters:

  • y_true (list, default: None ) –

    List of vectors representing the targets, or a list of integers representing the correct label. If not provided, then matrix may be provided.

  • y_predicted (list, default: None ) –

    List of vectors representing predicted values, or a list of integers representing the output. If not provided, then matrix may be provided.

  • labels (list, default: None ) –

    A list of strings that name of the columns and rows, in order.

  • matrix (list, default: None ) –

    The confusion matrix (list of lists). Must be square, if given. If not given, then it is possible to provide y_true and y_predicted.

  • title (str, default: 'Confusion Matrix' ) –

    A custom name to be displayed. By default, it is "Confusion Matrix".

  • row_label (str, default: 'Actual Category' ) –

    Label for rows.

  • column_label (str, default: 'Predicted Category' ) –

    Label for columns.

  • max_examples_per_cell (int, default: 25 ) –

    Maximum number of examples per cell.

  • max_categories (int, default: 25 ) –

    Max number of columns and rows to use.

  • winner_function (callable, default: None ) –

    A function that takes in an entire list of rows of patterns, and returns the winning category for each row.

  • index_to_example_function (callable, default: None ) –

    A function that takes an index and returns either a number, a string, a URL, or a {"sample": str, "assetId": str} dictionary. See below for more info. If left blank, the function returns a number representing the index of the example.

  • cache (bool, default: True ) –

    Should the results of index_to_example_function be cached and reused?

  • selected (list, default: None ) –

    A list of selected category indices. These are the rows/columns that will be shown. By default, select is None. If the number of categories is greater than max_categories, and selected is not provided, then selected will be computed automatically by selecting the most confused categories.

  • images (list, default: None ) –

    A list of data that can be passed to Experiment.log_image().

  • kwargs (optional, default: {} ) –

    Any extra keywords and their values will be passed onto the index_to_example_function.

Note

For more details and example uses, please see: Log confusion matrices. Uses winner_function to compute winning categories for y_true and y_predicted, if they are vectors.

Also, for more low-level information, see comet_ml.utils.ConfusionMatrix

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
import comet_ml
from sklearn.metrics import confusion_matrix

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

y_true = [2, 0, 2, 2, 0, 1]
y_pred = [0, 0, 2, 2, 0, 2]

confusion_matrix = exp.create_confusion_matrix(y_true, y_pred)

exp.end()

create_embedding_image

create_embedding_image(
    image_data,
    image_size,
    image_preprocess_function=None,
    image_transparent_color=None,
    image_background_color_function=None,
)

Create an embedding image (a sprite sheet). Returns the image and the url to the image.

Parameters:

  • image_data (Any) –

    List of arrays or Images.

  • image_size (list) –

    The size of each image.

  • image_preprocess_function (callable, default: None ) –

    If image_data is an array, apply this function to each element first.

  • image_transparent_color (list, default: None ) –

    A (red, green, blue) tuple.

  • image_background_color_function (callable, default: None ) –

    A function that takes an index, and returns a (red, green, blue) color tuple.

Returns:

  • tuple

    Returns the image and url

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
import comet_ml
import numpy as np

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

num_images = 10
inputs = np.random.randint(0, 256, (num_images, 28, 28))
label = np.random.randint(0, 10, num_images)

def label_to_color(index):
    label_colors = [
        (255, 0, 0), (0, 255, 0), (0, 0, 255),
        (255, 255, 0), (0, 255, 255), (128, 128, 0),
        (0, 128, 128), (128, 0, 128), (255, 0, 255),
        (255, 255, 255)
    ]
    return label_colors[label[index] % len(label_colors)]

image, image_url = exp.create_embedding_image(
    image_data=inputs,
    image_size=(28, 28),
    image_preprocess_function=lambda matrix: np.round(matrix / 255, 0) * 2,
    image_transparent_color=(0, 0, 0),
    image_background_color_function=label_to_color
)

print("Generated Image URL:", image_url)

exp.end()
create_symlink(project_name: str) -> None

Creates a symlink for this experiment in another project. The experiment will now be displayed in the project provided and the original project.

Parameters:

  • project_name (str) –

    Represents the project name. Project must exist.

disable_mp

disable_mp()

Disabling the auto-collection of metrics and monkey-patching of the Machine Learning frameworks.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

exp.disable_mp()

exp.log_parameter("example_parameter", 123)

exp.end()

display

display(clear=False, wait=True, new=0, autoraise=True, tab=None)

Show the Comet.ml experiment page in an IFrame in a Jupyter notebook or Jupyter lab, OR open a browser window or tab.

Parameters:

  • clear (bool, default: False ) –

    To clear the output area, use clear=True. This is only used in Notebook environments.

  • wait (bool, default: True ) –

    To wait for the next displayed item, use wait=True (cuts down on flashing). This is only used in Notebook environments.

  • new (int, default: 0 ) –

    Open a new browser window if new=1, otherwise re-use existing window/tab. This is only used in non-Notebook environments.

  • autoraise (bool, default: True ) –

    Make the browser tab/window active. This is only used in non-Notebook environments.

  • tab (str, default: None ) –

    Name of the Tab on Experiment View

Note

The Tab name should be one of:

  • "artifacts"
  • "assets"
  • "audio"
  • "charts"
  • "code"
  • "confusion-matrices"
  • "histograms"
  • "images"
  • "installed-packages"
  • "metrics"
  • "notes"
  • "parameters"
  • "system-metrics"
  • "text"

display_project

display_project(view_id=None, clear=False, wait=True, new=0, autoraise=True)

Show the Comet project page in an IFrame in either (1) a Jupyter notebook or Jupyter lab or (2) open a browser window or tab.

Parameters:

  • view_id (str, default: None ) –

    The id of the view to show.

  • clear (bool, default: False ) –

    To clear the output area, use clear=True.

  • wait (bool, default: True ) –

    To wait for the next displayed item, use wait=True (cuts down on flashing).

  • new (int, default: 0 ) –

    Open a new browser window if new=1, otherwise re-use existing window/tab.

  • autoraise (bool, default: True ) –

    Make the browser tab/window active.

Note

For Jupyter environments, you can utilize the clear and wait parameters. For non-Jupyter environments, you can utilize the new and autoraise parameters.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

# Optionally log some metrics or parameters (not required for displaying the project)
exp.log_metric("accuracy", 0.95)
exp.log_parameter("learning_rate", 0.01)

exp.display_project()

exp.end()

end

end()

The end() method finalizes an experiment by ensuring all data (i.e., parameters, metrics, asset, etc.) are uploaded to Comet before it returns. Unless you are running in a Jupyter Notebook, using the end() method is optional as it will be called automatically by Comet when the script ends.

Note

Only one Experiment can be alive in a Python process, if you wish to start a new Experiment you should first end the previous Experiment by calling the end() method and then start the new Experiment.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
import comet_ml
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

# Initialize Comet
comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

# Run an experiment
data = load_iris()
X = data.data
y = data.target

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

model = LogisticRegression()
model.fit(X_train, y_train)

y_pred_train = model.predict(X_train)

acc = accuracy_score(y_train, y_pred_train)

# Log accuracy metric to Comet
exp.log_metric("accuracy", acc)

# End the experiment
exp.end()

flush

flush() -> bool

Flush all pending data to the Comet server. It works similar to end but without ending the run. This is a blocking operation that will wait for all the data logged so far to be delivered to the Comet server.

Returns:

  • bool

    Returns a Boolean indicating whether the flush was successful or not.

get_artifact

get_artifact(
    artifact_name: str,
    workspace: Optional[str] = None,
    version_or_alias: Optional[str] = None,
) -> LoggedArtifact

Returns a logged artifact object that can be used to access the artifact version assets and download them locally.

If no version or alias is provided, the latest version for that artifact is returned.

Parameters:

  • artifact_name (str) –

    Retrieve an artifact with that name. This could either be a fully qualified artifact name like workspace/artifact-name:versionOrAlias or just the name of the artifact like artifact-name.

  • workspace (str, default: None ) –

    Retrieve an artifact belonging to that workspace

  • version_or_alias (str, default: None ) –

    Retrieve the artifact by the given alias or version.

Returns:

Example
logged_artifact = experiment.get_artifact(
    "workspace/artifact-name:version_or_alias"
)

Which is equivalent to:

logged_artifact = experiment.get_artifact(
    artifact_name="artifact-name",
    workspace="workspace",
    version_or_alias="version_or_alias")

get_callback

get_callback(framework, *args, **kwargs)

Get a callback for a particular framework.

Parameters:

  • framework (str) –

    Specifies the machine learning framework for which you want a callback.

Returns:

  • callable

    Callback object that is compatible with the framework specified in the call.

Note

When framework == 'keras' then return an instance of Comet.ml's Keras callback.

When framework == 'tf-keras' then return an instance of Comet.ml's TensorflowKeras callback.

When framework == "tf-estimator-train" then return an instance of Comet.ml's Tensorflow Estimator Train callback.

When framework == "xgboost" then return an instance of Comet.ml's xgboost.callback.TrainingCallback implementation

When framework == "fastai" then return an instance of Comet.ml's fastai.learner.Callback implementation.

The keras callbacks are added to your Keras model.fit() callbacks list automatically to report model training metrics to Comet.ml so you do not need to add them manually.

The lightgbm callback is added to the lightgbm.train() callbacks list automatically to report model training metrics to Comet.ml so you do not need to add it manually.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
import comet_ml
import tensorflow as tf
from tensorflow import keras

# Initialize the Comet experiment with your API key and project information
comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

# Load a dataset (using MNIST as an example)
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0

# Build a simple model
model = keras.models.Sequential([
    keras.layers.Flatten(input_shape=(28, 28)),
    keras.layers.Dense(128, activation='relu'),
    keras.layers.Dropout(0.2),
    keras.layers.Dense(10)
])

# Compile the model
model.compile(optimizer='adam',
            loss=tf.losses.SparseCategoricalCrossentropy(from_logits=True),
            metrics=['accuracy'])

# Get the Comet ML callback for Keras
comet_callback = exp.get_callback("keras")

# Train the model
model.fit(x_train, y_train, epochs=5, validation_data=(x_test, y_test), callbacks=[comet_callback])

# End the experiment
exp.end()

get_keras_callback

get_keras_callback()

This method is deprecated. See Experiment.get_callback("keras"). Get a callback for the Keras framework.

get_key

get_key()

Returns the experiment key, useful for using with the ExistingExperiment class

Returns:

  • str

    Experiment Key (String)

Example
1
2
3
4
5
6
7
8
import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

exp.end()

prev_exp = comet_ml.ExistingExperiment(experiment_key=exp.get_key())

get_metric

get_metric(name: str) -> Any

Get the last value for a metric from those logged.

Parameters:

  • name (str) –

    The name of the metric to get. This method will return the last value logged to this metric.

get_name

get_name() -> str

Get the name of the experiment, if one.

Example
1
2
3
4
5
6
7
8
import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

exp.set_name("My Name")

print(exp.get_name())

get_other

get_other(name: str) -> Any

Get an other from those logged.

Parameters:

  • name (str) –

    The name of the other parameter to get.

get_parameter

get_parameter(name)

Get a parameter that was logged previously in an Experiment.

Parameters:

  • name (str) –

    The name of the parameter to get.

Returns:

  • str

    Parameter value

Note

If this method is called inside a context, like test, train, validate or context_manager, the current context name will be automatically added at the front of parameter name.

Raises a KeyError if parameter with given name not found.

Example
import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

exp.log_parameter("learning_rate", 0.0001)
exp.log_parameter("batch_size", 64)

learning_rate = exp.get_parameter("learning_rate")
print(f"Learning Rate: {learning_rate}")
assert float(learning_rate) == 0.0001

batch_size = exp.get_parameter("batch_size")
print(f"Batch Size: {batch_size}")
assert int(batch_size) == 64

exp.end()

get_tags

get_tags() -> List[str]

Return the tags of this experiment.

Returns:

  • List[str]

    List[str]: The list of tags.

log_artifact

log_artifact(artifact: Artifact) -> LoggedArtifact

Log an Artifact object, synchronously create a new Artifact Version and upload asynchronously all local and remote assets attached to the Artifact object.

Parameters:

  • artifact (Artifact) –

    An Artifact object.

Returns:

log_asset

log_asset(
    file_data,
    file_name=None,
    overwrite=False,
    copy_to_tmp=True,
    step=None,
    metadata=None,
)

Logs the Asset determined by file_data.

Parameters:

  • file_data (Any) –

    Either the file path of the file you want to log, or a file-like asset.

  • file_name (str, default: None ) –

    A custom file name to be displayed. If not provided the filename from the file_data argument will be used.

  • overwrite (bool, default: False ) –

    If True will overwrite all existing assets with the same name.

  • copy_to_tmp (bool, default: True ) –

    (Deprecated) If file_data is a file-like object, then this flag determines if the file is first copied to a temporary file before upload. If copy_to_tmp is False, then it is sent directly to the cloud.

  • step (int, default: None ) –

    Used to associate the asset to a specific step.

  • metadata (Any, default: None ) –

    Some additional data to attach to the the audio asset. Must be a JSON-encodable dict.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
import comet_ml
import pandas as pd

data = {
    'Name': ['Alice', 'Bob', 'Charlie', 'David'],
    'Age': [25, 30, 35, 40],
    'City': ['New York', 'Los Angeles', 'Chicago', 'Houston']
}
df = pd.DataFrame(data)
df.to_csv('sample_data.csv', index=False)

comet_ml.login()
exp = comet_ml.Experiment(comet_project="comet-docs")

exp.log_asset(file_data="sample_data.csv", file_name="sample_data.csv")

exp.end()

log_asset_data

log_asset_data(
    data,
    name=None,
    overwrite=False,
    step=None,
    metadata=None,
    file_name=None,
    epoch=None,
)

Logs the data given (str, binary, or JSON).

Parameters:

  • data

    data to be saved as asset.

  • name

    A custom file name to be displayed. If not provided the filename from the temporary saved file will be used.

  • overwrite

    If True will overwrite all existing assets with the same name.

  • step

    Used to associate the asset to a specific step.

  • epoch

    Used to associate the asset to a specific epoch.

  • metadata

    Some additional data to attach to the the asset data. Must be a JSON-encodable dict.

Note

See also comet_ml.APIExperiment.get_asset_list, comet_ml.APIExperiment.get_asset, and comet_ml.APIExperiment.get_asset_by_name.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name = "comet-docs")

data = {"key": "value", "number": 123}

exp.log_asset_data(data, file_name = "sample_data.json")

exp.end()

log_asset_folder

log_asset_folder(folder, step=None, log_file_name=None, recursive=False)

Logs all the files located in the given folder as assets.

Parameters:

  • folder (str) –

    the path to the folder you want to log.

  • step (int, default: None ) –

    used to associate the asset to a specific step.

  • log_file_name (bool, default: None ) –

    if True, log the file path with each file.

  • recursive (bool, default: False ) –

    if True, recurse folder and save file names.

Note

If log_file_name is set to True, each file in the given folder will be logged with the following name schema: FOLDER_NAME/RELPATH_INSIDE_FOLDER. Where FOLDER_NAME is the basename of the given folder and RELPATH_INSIDE_FOLDER is the file path relative to the folder itself.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
import comet_ml

# Initialize Comet ML Experiment
comet_ml.login()
exp = comet_ml.Experiment(project_name = "comet-docs")

# Path to the folder containing assets you want to log
folder_path = "./"

# Log the entire folder as an asset
exp.log_asset_folder(folder_path)

# End the experiment
exp.end()

log_audio

log_audio(
    audio_data: Any,
    sample_rate: Optional[int] = None,
    normalize_data: str = "PCM_16_BIT",
    file_name: Optional[str] = None,
    metadata: Optional[Dict[str, Any]] = None,
    overwrite: bool = False,
    copy_to_tmp: bool = True,
    step: Optional[int] = None,
) -> Optional[Dict[str, Optional[str]]]

Logs the audio Asset determined by audio data.

Parameters:

  • audio_data (str | array) –

    Either the file path of the file you want to log, or a numpy array given to scipy.io.wavfile.write for wav conversion.

  • sample_rate (int, default: None ) –

    The sampling rate given to scipy.io.wavfile.write for creating the wav file.

  • normalize_data (str, default: 'PCM_16_BIT' ) –

    The parameter that determines whether audio_data should be normalized to fit within the specific numerical bounds (e.g. 16-bit PCM). If you set this parameter to 'OFF', it is your responsibility to ensure that audio_data is compatible with the format supported by scipy.io.wavfile.write. Supported values: 'PCM_16_BIT', 'OFF'.

  • file_name (str, default: None ) –

    A custom file name to be displayed. If not provided, the filename from the audio_data argument will be used.

  • metadata (dict, default: None ) –

    Some additional data to attach to the audio asset. Must be a JSON-encodable dict.

  • overwrite (bool, default: False ) –

    If True will overwrite all existing assets with the same name.

  • copy_to_tmp (bool, default: True ) –

    (Deprecated) If audio_data is a numpy array, then this flag determines if the WAV file is first copied to a temporary file before upload. If copy_to_tmp is False, then it is sent directly to the cloud.

  • step (int, default: None ) –

    Used to associate the audio asset to a specific step.

Note

See also Log audio and Audio tab.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
import comet_ml
import numpy as np

# Initialize Comet ML Experiment
comet_ml.login()
exp = comet_ml.Experiment(project_name = "comet-docs")

# Create an audio sample
fs = 100
rate = 44100
t = np.linspace(0., 1., rate)
amplitude = np.iinfo(np.int16).max
audio_sample = amplitude * np.sin(2. * np.pi * fs * t)

# Log the audio to Comet ML
exp.log_audio(
    audio_data=audio_sample,
    sample_rate=rate,
    file_name="Example Audio",
    step=0,
)

# End the experiment
exp.end()

log_code

log_code(
    file_name=None, folder=None, code=None, code_name=None, overwrite=False
)

Logs additional source code files. This method allows logging code in different ways:

  • Without parameters, logs the code from the file calling this method.
  • With file_name, logs the code from the specified file.
  • Using folder, logs all files' code in that folder.
  • Using code, logs the content as source code.

Parameters:

  • file_name (str, default: None ) –

    File path to log.

  • folder (str, default: None ) –

    Folder path where the code files are stored.

  • code (str, default: None ) –

    Source code, either as text or a file-like object (such as StringIO). If passed, code_name is mandatory.

  • code_name (str, default: None ) –

    Name of the source code file.

  • overwrite (bool, default: False ) –

    If True, overwrites existing code with the same name.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

# Logs code from the file using this method.
exp.log_code()

# Logs code from the file 'dataset.py'.
exp.log_code(file_name="dataset.py")

# Logs all files code in the 'training/' folder.
exp.log_code(folder="training/")

# Logs any other code
exp.log_code(code=DYNAMIC_CODE_SOURCE, code_name="user_training_code.py")

log_confusion_matrix

log_confusion_matrix(
    y_true=None,
    y_predicted=None,
    matrix=None,
    labels=None,
    title="Confusion Matrix",
    row_label="Actual Category",
    column_label="Predicted Category",
    max_examples_per_cell=25,
    max_categories=25,
    winner_function=None,
    index_to_example_function=None,
    cache=True,
    file_name="confusion-matrix.json",
    overwrite=False,
    step=None,
    epoch=None,
    images=None,
    selected=None,
    **kwargs
)

Logs a confusion matrix.

Parameters:

  • y_true (list, default: None ) –

    List of vectors representing the targets, or a list of integers representing the correct label. If not provided, then matrix may be provided.

  • y_predicted (list, default: None ) –

    List of vectors representing predicted values, or a list of integers representing the output. If not provided, then matrix may be provided.

  • labels (list, default: None ) –

    A list of strings that name of the columns and rows, in order.

  • matrix (list, default: None ) –

    The confusion matrix (list of lists). Must be square, if given. If not given, then it is possible to provide y_true and y_predicted.

  • title (str, default: 'Confusion Matrix' ) –

    A custom name to be displayed. By default, it is "Confusion Matrix".

  • row_label (str, default: 'Actual Category' ) –

    Label for rows.

  • column_label (str, default: 'Predicted Category' ) –

    Label for columns.

  • max_examples_per_cell (int, default: 25 ) –

    Maximum number of examples per cell.

  • max_categories (int, default: 25 ) –

    Max number of columns and rows to use.

  • winner_function (callable, default: None ) –

    A function that takes in an entire list of rows of patterns, and returns the winning category for each row.

  • index_to_example_function (callable, default: None ) –

    A function that takes an index and returns either a number, a string, a URL, or a {"sample": str, "assetId": str} dictionary. See below for more info. If left blank, the function returns a number representing the index of the example.

  • cache (bool, default: True ) –

    Should the results of index_to_example_function be cached and reused?

  • file_name (str, default: 'confusion-matrix.json' ) –

    The name of the file that the confusion matrix will be saved as when logged.

  • overwrite (bool, default: False ) –

    If set to True, the existing confusion matrix with the same file name will be overwritten. If False, a new entry is created, preserving the previous versions.

  • step (int, default: None ) –

    The step parameter can be used to specify the particular step or iteration in the training process at which the confusion matrix is logged.

  • epoch (int, default: None ) –

    Similar to the step parameter, epoch specifically denotes the training epoch.

  • images (list, default: None ) –

    A list of data that can be passed to Experiment.log_image().

  • selected (list, default: None ) –

    A list of selected category indices. These are the rows/columns that will be shown. By default, select is None. If the number of categories is greater than max_categories, and selected is not provided, then selected will be computed automatically by selecting the most confused categories.

  • kwargs (Any, default: {} ) –

    Any extra keywords and their values will be passed onto the index_to_example_function.

Note

For more details and example uses, please see: Log confusion matrices. Uses winner_function to compute winning categories for y_true and y_predicted, if they are vectors.

Also, for more low-level information, see comet_ml.utils.ConfusionMatrix

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
import comet_ml
from sklearn.metrics import confusion_matrix

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

y_true = [2, 0, 2, 2, 0, 1]
y_pred = [0, 0, 2, 2, 0, 2]

confusion_matrix = exp.create_confusion_matrix(y_true, y_pred)
exp.log_confusion_matrix(matrix=confusion_matrix, step=0, file_name="confusion_matrix.json")

exp.end()

log_curve

log_curve(name, x, y, overwrite=False, step=None)

Log timeseries data.

Parameters:

  • name

    Name of data.

  • x

    list of x-axis values.

  • y

    list of y-axis values.

  • overwrite

    If True, overwrite previous log.

  • step

    The step value.

Example
1
2
3
4
5
6
7
8
import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name = "comet-docs")

exp.log_curve("my curve", x=[1, 2, 3, 4, 5],
                          y=[10, 20, 30, 40, 50])
exp.end()

log_dataframe_profile

log_dataframe_profile(
    dataframe: Any,
    name: Optional[str] = "dataframe",
    minimal: bool = False,
    log_raw_dataframe: bool = True,
    dataframe_format: str = "json",
    **format_kwargs: Any
) -> Optional[Dict[str, Optional[Dict[str, str]]]]

Log a pandas DataFrame profile as an asset. Optionally, can also log the dataframe.

Parameters:

  • dataframe (Any) –

    the dataframe to profile and/or log

  • name (str, default: 'dataframe' ) –

    the basename (without extension) of the dataframe assets.

  • minimal (bool, default: False ) –

    if True, create a minimal profile. Useful for large datasets.

  • log_raw_dataframe (bool, default: True ) –

    Log the dataframe as an asset (same as calling log_table())

  • dataframe_format (str, default: 'json' ) –

    The format. for optionally logging the dataframe.

  • format_kwargs (Any, default: {} ) –

    Keyword args for dataframe logging as an asset.

Example
1
2
3
4
5
6
7
8
9
import comet_ml
import pandas as pd

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")
df = pd.read_csv("https://people.sc.fsu.edu/~jburkardt/data/csv/airtravel.csv",
                 parse_dates=['year'], encoding='UTF-8')
exp.log_dataframe_profile(df)
exp.end()

See also: Experiment.log_table(pandas_dataframe)

log_dataset_hash

log_dataset_hash(data) -> None

Used to log the hash of the provided object.

Parameters:

  • data

    Any object that when casted to string (e.g str(data)) returns a value that represents the underlying data.

Note

This is a best-effort hash computation which is based on the md5 hash of the underlying string representation of the object data. Developers are encouraged to implement their own hash computation that's tailored to their underlying data source. That could be reported as experiment.log_parameter("dataset_hash", your_hash).

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
import comet_ml
import pandas as pd

# Initialize Comet ML Experiment
comet_ml.login()
exp = comet_ml.Experiment(project_name = "comet-docs")

# Example: Load a dataset (could be from a file, database, etc.)
data = pd.read_csv('path/to/your/dataset.csv')

# Function to create a consistent hash of a pandas DataFrame
def hash_dataframe(df):
    return pd.util.hash_pandas_object(df).sum()

# Compute the hash of the dataset
dataset_hash = hash_dataframe(data)

# Log the dataset hash to Comet ML
exp.log_dataset_hash(dataset_hash)

# End the experiment
exp.end()

log_dataset_info

log_dataset_info(
    name: Optional[str] = None,
    version: Optional[str] = None,
    path: Optional[str] = None,
) -> None

Used to log information about your dataset.

Parameters:

  • name (str, default: None ) –

    A string representing the name of the dataset.

  • version (str, default: None ) –

    A string representing a version identifier.

  • path (str, default: None ) –

    A string that represents the path to the dataset. Potential values could be a file system path, S3 path, or Database query.

Note

At least one argument should be included when calling this method. The logged values will show up in the Other tab of the Comet UI.

Example
1
2
3
4
5
6
7
8
import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

exp.log_dataset_info(name="Your dataset", version="1.0", path="./data.csv")

exp.end()

log_dependency

log_dependency(name, version) -> None

Reports name,version to the Installed Packages tab on Comet. Useful to track dependencies.

Parameters:

  • name

    Any type of key (str,int,float..)

  • version

    Any type of value (str,int,float..)

log_embedding

log_embedding(
    vectors,
    labels,
    image_data=None,
    image_size=None,
    image_preprocess_function=None,
    image_transparent_color=None,
    image_background_color_function=None,
    title="Comet Embedding",
    template_filename=None,
    group=None,
)

Log a multi-dimensional dataset and metadata for viewing with Comet's Embedding Projector (experimental).

Parameters:

  • vectors

    the tensors to visualize in 3D.

  • labels

    labels for each tensor, or a table for each tensor

  • image_data (optional, default: None ) –

    list of arrays or Images

  • image_size (optional, default: None ) –

    The size of each image, required if image_data is given.

  • image_preprocess_function (optional, default: None ) –

    If image_data is an array, apply this function to each element first

  • image_transparent_color

    A (red, green, blue) tuple.

  • image_background_color_function

    a function that takes an index, and returns a (red, green, blue) color tuple.

  • title

    Name of tensor.

  • template_filename

    Name of template JSON file.

  • group

    Name of group of embeddings.

See also: comet_ml.Embedding

Note: labels must be a list of strings, or a table where table is a list of lists of data, and the first row is a header followed by a list for each vector. See example below.

Examples:

from comet_ml import Experiment

import numpy as np
from keras.datasets import mnist

(x_train, y_train), (x_test, y_test) = mnist.load_data()

def label_to_color(index):
    label = y_test[index]
    if label == 0:
        return (255, 0, 0)
    elif label == 1:
        return (0, 255, 0)
    elif label == 2:
        return (0, 0, 255)
    elif label == 3:
        return (255, 255, 0)
    elif label == 4:
        return (0, 255, 255)
    elif label == 5:
        return (128, 128, 0)
    elif label == 6:
        return (0, 128, 128)
    elif label == 7:
        return (128, 0, 128)
    elif label == 8:
        return (255, 0, 255)
    elif label == 9:
        return (255, 255, 255)

experiment = Experiment(project_name="projector-embedding")

experiment.log_embedding(
    vectors=x_test,
    labels=y_test,
    image_data=x_test,
    image_preprocess_function=lambda matrix: np.round(matrix/255,0) * 2,
    image_transparent_color=(0, 0, 0),
    image_size=(28, 28),
    image_background_color_function=label_to_color,
)
# With a table:
experiment.log_embedding(
    vectors=[[3, 6, 2], [6, 1, 3], [9, 1, 1]],
    labels=[
        ["index", "label"],
        [      5, "apple"],
        [     16, "car"],
        [      2, "person"],
    ]
)

log_epoch_end

log_epoch_end(epoch_cnt, step=None) -> None

Logs that the epoch finished. Required for progress bars.

Parameters:

  • epoch_cnt (int) –

    The current count of completed epochs, indicating how many epochs have finished in the training cycle.

  • step (int, default: None ) –

    The step count at which the epoch ends.

Example
1
2
3
4
5
6
7
8
9
import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

num_epochs = 5
exp.log_epoch_end(num_epochs)

exp.end()

log_figure

log_figure(figure_name=None, figure=None, overwrite=False, step=None)

Log a visual representation of the provided figure to Comet in SVG format.

Parameters:

  • figure_name (str, default: None ) –

    A descriptive name for the figure.

  • figure (Any, default: None ) –

    The figure to be logged. Accepts Plotly Figures, Matplotlib Figures, or Seaborn simple plots. If not provided, the function will log the current global Matplotlib Pyplot figure.

  • overwrite (bool, default: False ) –

    Determines whether to overwrite an existing figure with the same name.

  • step (int, default: None ) –

    Associates the figure asset with a specific step in the Comet experiment.

Note

Kaleido is required to be installed in order to log plotly figures.

log_histogram_3d

log_histogram_3d(
    values, name=None, step=None, epoch=None, metadata=None, **kwargs
)

Logs a histogram of values for a 3D chart as an asset for this experiment. Calling this method multiple times with the same name and incremented steps will add additional histograms to the 3D chart on Comet.ml.

Parameters:

  • values

    A list, tuple, array (any shape) to summarize, or a Histogram object.

  • name

    Name of summary.

  • step

    Used as the Z axis when plotting on Comet.

  • epoch

    Used as the Z axis when plotting on Comet.

  • metadata

    Used for items like prefix for histogram name.

  • kwargs

    Additional keyword arguments for histogram.

Note

This method requires that step is either given here, or has been set elsewhere. For example, if you are using an auto- logger that sets step then you don't need to set it here.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
import comet_ml
import numpy as np

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

values = np.random.normal(0, 1, (1000, 3))

exp.log_histogram_3d(
    values=values,
    name="Histogram of Randomly Distributed Data",
    step=1,
    epoch=1,
    metadata={'description': 'This is a 3D histogram of three normally distributed random variables'}
)

exp.end()

log_html

log_html(html, clear=False) -> None

Reports any HTML blob to the HTML tab on Comet.ml. Useful for creating your own rich reports. The HTML will be rendered as an Iframe. Inline CSS/JS supported.

Parameters:

  • html

    Any html string.

  • clear

    When setting clear=True it will remove all previous html.

Example
1
2
3
4
5
6
7
import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

exp.log_html('<a href="www.comet.com"> I love Comet</a>')
exp.end()

log_html_url

log_html_url(url, text=None, label=None)

Easy to use method to add a link to a URL in the HTML tab on Comet.

Parameters:

  • url

    a link to a file or notebook, for example

  • text

    text to use a clickable word or phrase (optional; uses url if not given)

  • label

    text that precedes the link

Example

Logging a link:

1
2
3
4
5
6
7
import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

exp.log_html_url("https://my-company.com/file.txt")
exp.end()

Adds html similar to:

<a href="https://my-company.com/file.txt">
https://my-company.com/file.txt
</a>

Logging a link with text:

1
2
3
4
5
6
7
8
import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

exp.log_html_url("https://my-company.com/file.txt",
                            "File")
exp.end()

Adds html similar to:

<a href="https://my-company.com/file.txt">File</a>

Logging a link with text and a label:

1
2
3
4
5
import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")
exp.log_html_url("https://my-company.com/file.txt", "File", "Label")

Adds html similar to:

Label: <a href="https://my-company.com/file.txt">File</a>

log_image

log_image(
    image_data: Any,
    name: Optional[str] = None,
    overwrite: bool = False,
    image_format: str = "png",
    image_scale: float = 1.0,
    image_shape: Optional[Tuple[int, int]] = None,
    image_colormap: Optional[str] = None,
    image_minmax: Optional[Tuple[int, int]] = None,
    image_channels: str = "last",
    copy_to_tmp: bool = True,
    step: Optional[int] = None,
    annotations: Optional[List[Layer]] = None,
    metadata: Optional[Dict[str, Any]] = None,
) -> Optional[Dict[str, str]]

Logs the image. Images are displayed on the Graphics tab in Comet.

Parameters:

  • image_data (Any) –

    Is one of the following: - a path (string) to an image - a file-like object containing an image - a numpy matrix - a TensorFlow tensor - a PyTorch tensor - a list or tuple of values - a PIL Image

  • name (str, default: None ) –

    A custom name to be displayed on the dashboard. If not provided the filename from the image_data argument will be used if it is a path.

  • overwrite (bool, default: False ) –

    If another image with the same name exists, it will be overwritten if overwrite is set to True.

  • image_format (str, default: 'png' ) –

    If the image_data is actually something that can be turned into an image, this is the format used. Typical values include 'png' and 'jpg'.

  • image_scale (float, default: 1.0 ) –

    If the image_data is actually something that can be turned into an image, this will be the new scale of the image.

  • image_shape (tuple, default: None ) –

    If the image_data is actually something that can be turned into an image, this is the new shape of the array. Dimensions are (width, height) or (width, height, colors) where colors is 3 (RGB) or 1 (grayscale).

  • image_colormap (str, default: None ) –

    If the image_data is actually something that can be turned into an image, this is the colormap used to colorize the matrix.

  • image_minmax (tuple, default: None ) –

    If the image_data is actually something that can be turned into an image, this is the (min, max) used to scale the values. Otherwise, the image is autoscaled between (array.min, array.max).

  • image_channels (str, default: 'last' ) –

    If the image_data is actually something that can be turned into an image, this is the setting that indicates where the color information is in the format of the 2D data. 'last' indicates that the data is in (rows, columns, channels) where 'first' indicates (channels, rows, columns).

  • copy_to_tmp (bool, default: True ) –

    (Deprecated) If image_data is not a file path, then this flag determines if the image is first copied to a temporary file before upload. If copy_to_tmp is False, then it is sent directly to the cloud.

  • step (int, default: None ) –

    Used to associate the image asset to a specific step.

  • annotations (list, default: None ) –

    Annotations contains a list of annotation layers. Each annotation layer can contain bounding boxes and/or polygons. See example below.

  • metadata (dict, default: None ) –

    Additional metadata to be associated with logged image.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
import comet_ml
from comet_ml.annotations import Layer, Box, Polygon
import numpy as np
from PIL import Image

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")


# Create a dummy image using numpy
width, height = 256, 256
array = np.tile(np.linspace(0, 255, width, dtype=np.uint8), (height, 1))
image = Image.fromarray(array)

# Convert image to grayscale
image = image.convert("L")

# One layer with a box and polygon:
annotations = [
    Layer(
        boxes=[Box([x, y, w, h], label="dog")],
        polygons=[Polygon([(x1, y1), ...], label="cat")],
        name="Predictions",
    )
]

# Log the image
exp.log_image(
    image_data=image,
    name="Gradient.png",
    image_format="png",
    image_scale=0.5,
    image_shape=(128, 128),
    image_colormap='gray',
    step=1,
    annotations=annotations,
    metadata={"description": "A simple gradient image from black to white"}
)

# End the experiment
exp.end()

log_metric

log_metric(name, value, step=None, epoch=None, include_context=True) -> None

Logs a general metric (i.e accuracy, f1).

See also log_metrics.

Parameters:

  • name

    Name of your metric.

  • value

    The value of your metric, we recommend that this is a numerical value.

  • step

    Used as the X axis when plotting on Comet.

  • epoch

    Used as the X axis when plotting on Comet.

  • include_context

    If set to True, the current context will be logged along the metric.

Note

Comet guarantees to store 15,000 data points for each metric. If more than 15,000 data points are reported we perform a form of reservoir sub-sampling and download sample the metric values.

Example
1
2
3
y_pred_train = model.predict(X_train)
acc = compute_accuracy(y_pred_train, y_train)
experiment.log_metric("accuracy", acc)

log_metrics

log_metrics(
    dic: Dict[str, Any],
    prefix: Optional[str] = None,
    step: Optional[int] = None,
    epoch: Optional[int] = None,
)

Logs a key,value dictionary of metrics. See also log_metric

log_model

log_model(
    name,
    file_or_folder,
    file_name=None,
    overwrite=False,
    metadata=None,
    copy_to_tmp=True,
    prepend_folder_name=True,
)

Logs the model data under the name. Data can be a file path, a folder path or a file-like object.

Parameters:

  • name (str) –

    The name of the model

  • file_or_folder (str | dict) –

    The model data; can be a file path, a folder path or a file-like object.

  • file_name (str, default: None ) –

    The name of the model data. Used with file-like objects or files only.

  • overwrite (bool, default: False ) –

    If True, then overwrite previous versions Does not apply to folders.

  • metadata (dict, default: None ) –

    Some additional data to attach to the the data. Must be a JSON-encodable dict.

  • copy_to_tmp (bool, default: True ) –

    Depreacted. For file name or file-like; if True copy to temporary location before uploading; if False, then upload from current location

  • prepend_folder_name (bool, default: True ) –

    If True and logging a folder, prepend file path by the folder name.

Returns:

  • dict

    Dictionary of model URLs

log_notebook

log_notebook(filename, overwrite=False)

Log a Jupyter Notebook file as an asset.

Parameters:

  • filename (str) –

    The path and name of the notebook to be logged.

  • overwrite (bool, default: False ) –

    If True, overwrites the previously logged notebook asset.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
import comet_ml
import nbformat
from nbformat.v4 import new_notebook

# Create a dummy notebook to log
nb = new_notebook()

with open('example_notebook.ipynb', 'w', encoding='utf-8') as f:
    nbformat.write(nb, f)

# Log the new notebook to Comet
comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

exp.log_notebook("./example_notebook.ipynb")

exp.end()

log_optimization

log_optimization(
    optimization_id: Optional[str] = None,
    metric_name: Optional[str] = None,
    metric_value: Any = None,
    parameters: Optional[Dict] = None,
    objective: Optional[str] = None,
) -> None

Logs an existing optimization result.

Parameters:

  • optimization_id (str, default: None ) –

    The id of the optimization result.

  • metric_name (str, default: None ) –

    The name of your metric

  • metric_value (Any, default: None ) –

    The value of the given metric.

  • parameters (dict, default: None ) –

    Additional parameters to be logged.

  • objective (str, default: None ) –

    The objective of the optimization, could be either maximum/minimum.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

# Define a hypothetical scenario for optimization results
optimization_id = "opt123"
metric_name = "accuracy"
metric_value = 0.95
parameters = {
    'C': 1,
    'kernel': 'linear',
    'gamma': 'scale'
}
objective = "maximum"

# Log the optimization
exp.log_optimization(
    optimization_id=optimization_id,
    metric_name=metric_name,
    metric_value=metric_value,
    parameters=parameters,
    objective=objective
)

# End the experiment
exp.end()

log_other

log_other(key, value)

Reports a key and value to the Other tab on Comet.ml. Useful for reporting datasets attributes, datasets path, unique identifiers etc.

Other metadata can also be defined as environment variables. All environment variables that starts with COMET_LOG_OTHER_ will be logged automatically to the Experiment.

Parameters:

  • key

    Any type of key (str,int,float..)

  • value

    Any type of value (str,int,float..)

log_others

log_others(dictionary)

Reports dictionary of key/values to the Other tab on Comet.ml. Useful for reporting datasets attributes, datasets path, unique identifiers etc.

See log_other

Parameters:

  • dictionary (dict) –

    dict of key/values where value is any type of value (str,int,float..)

log_parameter

log_parameter(name: Any, value: Any, step: Optional[Any] = None) -> None

Logs a parameter.

Parameters:

  • name (Any) –

    Name of your parameter.

  • value (Any) –

    The value of your parameter.

  • step (Optional[Any], default: None ) –

    Used as the x-axis when plotting on Comet.

Note

It's encouraged to use log_other for additional values that are not hyper parameters.

If the same key is reported multiple times only the last reported value will be saved.

If this method is called inside a context, like test,train, validate or context_manager, the parameter will be stored with the current context name as a prefix.

See also log_parameters.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
import comet_ml

# Initialize an experiment
comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

# Log a parameter
exp.log_parameter("batch_size", 64)

# End the experiment
exp.end()

log_parameters

log_parameters(
    parameters: Dict[str, Any],
    prefix: Optional[str] = None,
    step: Optional[int] = None,
    nested_support: bool = True,
)

Logs a dictionary (or dictionary-like object) of multiple parameters.

Parameters:

  • parameters (dict) –

    This is a dictionary where each key is a string representing the name of the parameter, and the value is the parameter value itself, which can be of any data type that is serializable. The method logs each key-value pair as a parameter in the experiment.

  • prefix (str, default: None ) –

    This optional parameter allows you to add a prefix to the keys in the parameters dictionary when they are logged. This can be helpful for organizing or grouping parameters under a common namespace, making them easier to identify and analyze in the Comet UI.

  • step (int, default: None ) –

    This optional parameter can be used to associate the logged parameters with a particular step or iteration in an experiment. This is useful for tracking how parameters change over time, particularly in iterative processes like training a machine learning model over multiple epochs.

  • nested_support (bool, default: True ) –

    This parameter controls whether the method should support nested dictionaries. If set to True, the method can handle parameters that are dictionaries themselves and will flatten these into a format suitable for logging. Each key in a nested dictionary will be combined with its parent keys to create a single, flat key.

Note

See also log_parameter.

If you call this method multiple times with the same keys your values would be overwritten.

If this method is called inside a context, like test,train, validate or context_manager, the parameter will be stored with the current context name as a prefix.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

params = {
    "batch_size": 64,
    "layer1": "LSTM(128)",
    "layer2": "LSTM(128)",
    "MAX_LEN":200
}

exp.log_parameters(params)

exp.end()

log_points_3d

log_points_3d(
    scene_name: str,
    points: Optional[List[Point3D]] = None,
    boxes: Optional[List[Dict[str, Any]]] = None,
    step: Optional[int] = None,
    metadata: Optional[Dict] = None,
) -> None

Log 3d points and bounding boxes as an asset.

Parameters:

  • scene_name (str) –

    A string identifying the 3d scene to render. A same scene name could be logged across different steps.

  • points (list, default: None ) –

    A list of points, each point being a list (or equivalent like Numpy array). Each point length should be either 3, if only the position is given: [X, Y, Z]. The length could also be 6, if color is passed as well: [X, Y, Z, R, G, B]. Red, Green and Blue should be a number between 0 and 1. Either points or boxes are required.

  • boxes (list, default: None ) –

    List of box definition Dict.

  • step (int, default: None ) –

    Used to associate the asset to a specific step.

  • metadata (dict, default: None ) –

    Additional custom metadata can be associated with the logged asset.

Note

You can learn more about logging 3d point clouds here. You can also visualize these assets with the 3D Points panels.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
import comet_ml
import numpy as np

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

# Generate some synthetic 3D points
np.random.seed(42)
points = np.random.rand(10, 6)  # 10 points, with XYZ and RGB values

# Normalize the color values to be between 0 and 1
points[:, 3:] = points[:, 3:] / np.max(points[:, 3:])

# Define a bounding box as a list of segments (lines)
# Each segment is a pair of points (x, y, z)
box_segments = [
    # Base of the box
    [[0, 0, 0], [1, 0, 0]],
    [[1, 0, 0], [1, 1, 0]],
    [[1, 1, 0], [0, 1, 0]],
    [[0, 1, 0], [0, 0, 0]],
    # Top of the box
    [[0, 0, 1], [1, 0, 1]],
    [[1, 0, 1], [1, 1, 1]],
    [[1, 1, 1], [0, 1, 1]],
    [[0, 1, 1], [0, 0, 1]],
    # Vertical lines
    [[0, 0, 0], [0, 0, 1]],
    [[1, 0, 0], [1, 0, 1]],
    [[1, 1, 0], [1, 1, 1]],
    [[0, 1, 0], [0, 1, 1]],
]

# Create a box dictionary with the segments
box = {
    "segments": box_segments,
    "color": [255, 0, 0],  # Red color
    "name": "example_box",
}

# Log 3D points and the bounding box to Comet
exp.log_points_3d(
    scene_name="Simple3DExample",
    points=points.tolist(),
    boxes=[box],
)

# End the experiment
exp.end()

log_remote_asset

log_remote_asset(
    uri,
    remote_file_name=None,
    overwrite=False,
    asset_type="asset",
    step=None,
    metadata=None,
)

Logs a Remote Asset identified by an URI. A Remote Asset is an asset but its content is not uploaded and stored on Comet. Rather a link for its location is stored, so you can identify and distinguish between two experiment using different version of a dataset stored somewhere else.

Parameters:

  • uri (str) –

    The remote asset location, there is no imposed format, and it could be a private link.

  • remote_file_name (str, default: None ) –

    The "name" of the remote asset, could be a dataset name, a model file name.

  • overwrite (bool, default: False ) –

    If True will overwrite all existing assets with the same name.

  • asset_type (str, default: 'asset' ) –

    Specifies the type of the asset being logged.

  • step (int, default: None ) –

    Used to associate the asset to a specific step.

  • metadata (dict, default: None ) –

    Some additional data to attach to the remote asset. Must be a JSON-encodable dict.

Example
1
2
3
4
5
6
7
8
9
import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

exp.log_remote_asset("s3://bucket/folder/file")
exp.log_remote_asset("dataset:701bd06b43b7423296fb626027d02198")

exp.end()

log_remote_model

log_remote_model(
    model_name: str,
    uri: str,
    metadata: Any = None,
    sync_mode: bool = True,
    max_synced_objects: int = 10000,
) -> None

Logs metadata about a model that is stored elsewhere, such as remote storage solutions like AWS S3, Google Cloud Storage, Azure Blob Storage, etc. It allows you to keep track of model metadata without moving the actual model files into Comet.

Parameters:

  • model_name (str) –

    The name of the model.

  • uri (str) –

    The remote model location, there is no imposed format and it could be a private link. Can log a single file and a folder.

  • metadata (dict, default: None ) –

    some additional data to attach to the the data. Must be a JSON-encodable dict.

  • sync_mode (bool, default: True ) –

    If True and the URI begins with s3:// or gs://, Comet attempts to list all objects in the given bucket and path. All the objects will be logged under the given model name. If False, Comet just logs a single remote model with the provided URI as the remote URI.

  • max_synced_objects (int, default: 10000 ) –

    When sync_mode is True and the URI begins with s3:// or gs://, set the maximum number of S3/GCP objects to log. If there are more matching S3/GCP objects than max_synced_objects, a warning will be displayed and the provided URI will be logged as a single remote model.

Note

If the URI begins with s3:// or gs://, Comet attempts to list all objects in the given bucket and path and logs them individually.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

model_name = 'remote_model'
model_uri = 'gs://demo-remote-model/'

exp.log_remote_model(model_name=model_name, uri=model_uri, sync_mode=True)

exp.end()

log_system_info

log_system_info(key, value) -> None

Reports the key and value to the System metrics tab in the single experiments view of the Comet UI.

Parameters:

  • key

    Any type of key.

  • value

    Any type of value.

Note

Useful to track general system information. This information can be added to the table on the Project view. You can retrieve this information via the Python API.

Example
1
2
3
4
5
6
7
import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

exp.log_system_info(key="info-about-system", value="debian-based")
exp.end()

log_table

log_table(filename, tabular_data=None, headers=False, **format_kwargs)

Log tabular data, including data, csv files, tsv files, and Pandas dataframes.

Parameters:

  • filename (str) –

    A filename ending in ".csv", or ".tsv" (for tabular data) or ".json", ".csv", ".md", or ".html" (for Pandas dataframe data).

  • tabular_data (Any, default: None ) –

    Data that can be interpreted as 2D tabular data or a Pandas dataframe.

  • headers (bool | list, default: False ) –

    If True, will add column headers automatically if tabular_data is given; if False, no headers will be added; if list then it will be used as headers. Only useful with tabular data (csv, or tsv).

  • format_kwargs (Any, default: {} ) –

    When passed a Pandas dataframe these keyword arguments are used in the conversion to "json", "csv", "md", or "html". See Pandas Dataframe conversion methods (like to_json()) for more information.

See also:

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
import comet_ml
import pandas as pd

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

data = {
    'Name': ['Alice', 'Bob', 'Charlie'],
    'Age': [25, 30, 35],
    'City': ['New York', 'Los Angeles', 'Chicago']
}
example_table = pd.DataFrame(data)

exp.log_table("example_table.csv", example_table)

log_tensorflow_folder

log_tensorflow_folder(folder) -> None

Logs all the tensorflow log files located in the given folder as assets.

Parameters:

  • folder (str) –

    The path to the folder you want to log.

Note

Use comet_ml.APIExperiment.download_tensorflow_folder to get the contents of a previously logged folder.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
import comet_ml
import tensorflow as tf
from tensorflow.keras.layers import Dense
from tensorflow.keras.models import Sequential
import json

# Initialize Comet.ml
comet_ml.init(project_name="comet-docs")
exp = comet_ml.Experiment(project_name="comet-docs")

# Setup TensorFlow model
model = Sequential([
    Dense(64, activation='relu', input_shape=(10,)),
    Dense(1)
])

model.compile(optimizer='adam', loss='mean_squared_error')

# Create a directory and save the TensorFlow model there
model_dir = "saved_model"
model.save(model_dir)

# Save hyperparameters
params = {
    'batch_size': 32,
    'epochs': 10,
    'layers': [64, 1],
    'activation': ['relu', 'linear'],
    'optimizer': 'adam',
    'loss': 'mean_squared_error'
}
with open(f"{model_dir}/params.json", 'w') as f:
    json.dump(params, f)

# Log TensorFlow model folder to Comet.ml
exp.log_tensorflow_folder(model_dir)

# End the experiment
exp.end()

log_text

log_text(text, step=None, metadata=None) -> None

Logs the text. These strings appear on the Text Tab in the Comet UI.

Parameters:

  • text

    String of text to be stored.

  • step

    Used to associate the asset to a specific step.

  • metadata

    Some additional data to attach to the text. Must be a JSON-encodable dict.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
import comet_ml

# Initialize an experiment
comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

# Example of logging text at different steps of an experiment
exp.log_text("Starting an experiment", step=0)

# End the experiment
exp.end()

log_video

log_video(
    file: Union[str, IO],
    name: Optional[str] = None,
    overwrite: bool = False,
    format: Optional[str] = None,
    step: Optional[int] = None,
    epoch: Optional[int] = None,
    metadata: Optional[Dict[str, Any]] = None,
) -> Optional[Dict[str, str]]

Logs a video to Comet. Videos are displayed on the assets tab in Comet and support the following formats: MP4, MOV, WMV, and GIF.

Parameters:

  • file (str) –

    The path to the video or a file-like object that contains the video.

  • name (str, default: None ) –

    A custom name can be provided to be displayed on the assets tab. If not provided, the filename from the file argument will be used if it is a path.

  • overwrite (bool, default: False ) –

    If another video with the same name exists, it will be overwritten if overwrite is set to True.

  • format (str, default: None ) –

    The extension of the file is used to check if the asset is of a supported format. If the extension of the file is not correct or if you are uploading a file-like object, you can indicate the format of the video through this parameter.

  • step (int, default: None ) –

    This is used to associate the video asset with a specific step.

  • epoch (int, default: None ) –

    Used to associate the asset to a specific epoch.

  • metadata (dict, default: None ) –

    dditional custom metadata can be associated with the logged video.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
import comet_ml
import cv2
import numpy as np

# Initialize Comet.ml
comet_ml.init(project_name="comet-docs")
exp = comet_ml.Experiment(project_name="comet-docs")

# Create a dummy video file in MP4 format
video_filename = 'output.mp4'
frame_count = 60  # Number of frames in the video
frame_width = 640
frame_height = 480
frame_rate = 10  # Frames per second

fourcc = cv2.VideoWriter_fourcc(*'mp4v')
out = cv2.VideoWriter(video_filename, fourcc, frame_rate, (frame_width, frame_height))

for i in range(frame_count):
    frame = np.random.randint(0, 255, (frame_height, frame_width, 3), dtype=np.uint8)
    out.write(frame)

out.release()

# Log video file to Comet
exp.log_video(video_filename, overwrite=True)

# End the experiment
exp.end()

register_model

register_model(
    model_name: str,
    version: Optional[str] = None,
    workspace: Optional[str] = None,
    registry_name: Optional[str] = None,
    public: Optional[bool] = None,
    description: Optional[str] = None,
    comment: Optional[str] = None,
    tags: Optional[list] = None,
    stages: Optional[list] = None,
    status: Optional[str] = None,
    sync: Optional[bool] = False,
    timeout: float = 10.0,
) -> None

Register an experiment's model to the registry.

Parameters:

  • model_name (str) –

    The name of the experiment model.

  • workspace (str, default: None ) –

    This argument is deprecated and ignored. Models are registered to the workspace the experiment belongs to.

  • version (str, default: None ) –

    A proper semantic version string; defaults to "1.0.0".

  • registry_name (str, default: None ) –

    The name of the registered workspace model, if not provided the model_name will be used instead.

  • public (bool, default: None ) –

    If True, then the model will be publicly viewable.

  • description (str, default: None ) –

    A textual description of the model.

  • comment (str, default: None ) –

    A textual comment about the model

  • tags (list, default: None ) –

    A list of textual tags such as ["tag1", "tag2"], etc.

  • stages (list, default: None ) –

    This argument is deprecated and will be ignored. Please use tags instead.

  • status (str, default: None ) –

    A string describing the status of this model version.

  • sync (bool, default: False ) –

    Whether this function is synchronous and will be finished only once the model was registered.

  • timeout (float, default: 10.0 ) –

    Maximum time (In seconds) before the function would end if called with sync = True.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
import comet_ml
import pandas as pd
import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

data = pd.DataFrame({
    'Feature1': np.random.rand(100),
    'Feature2': np.random.rand(100),
    'Target': np.random.rand(100)
})
X = data[['Feature1', 'Feature2']]
y = data['Target']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

model = LinearRegression()
model.fit(X_train, y_train)

from joblib import dump
dump(model, "ols_model.pickle")
exp.log_model("ols_model", "ols_model.pickle")

exp.register_model(model_name="ols_model", version="1.0.1")

exp.end()

send_notification

send_notification(
    title: str,
    status: Optional[str] = None,
    additional_data: Optional[Dict[str, Any]] = None,
) -> None

Send yourself a notification through email when an experiment ends.

Parameters:

  • title (str) –

    The email subject.

  • status (str, default: None ) –

    The final status of the experiment. Typically, something like "finished", "completed" or "aborted".

  • additional_data (dict[str, Any], default: None ) –

    A dictionary of key/values to notify.

Note

In order to receive the notification, you need to have turned on Notifications in your Settings in the Comet user interface.

If you wish to have the additional_data saved with the experiment, you should also call Experiment.log_other() with this data as well.

This method uses the email address associated with your account.

Example
import comet_ml

comet_ml.login()
exp = comet_ml.Experiment()

exp.send_notification(
    "Experiment %s" % experiment.get_key(),
    "started"
)
try:
    train(...)
    exp.send_notification(
        "Experiment %s" % experiment.get_key(),
        "completed successfully"
    )
except Exception:
    exp.send_notification(
        "Experiment %s" % experiment.get_key(),
        "failed"
    )

set_cmd_args

set_cmd_args()

Logs command-line arguments used to run the script.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
import comet_ml

# Initialize Comet experiment
comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

exp.set_cmd_args()

# Continue on with your experiment
# ...

# End the experiment
exp.end()

set_code

set_code(code=None, overwrite=False, filename=None) -> None

This method is now deprecated, use comet_ml.Experiment.log_code instead.

Sets the current experiment script's code. Should be called once per experiment.

Parameters:

  • code (Any, default: None ) –

    The experiment's source code.

  • overwrite (bool, default: False ) –

    If True, will overwrite previously set code.

  • filename (str, default: None ) –

    name of file to get source code from

set_epoch

set_epoch(epoch) -> None

Sets the current epoch in the training process.

Parameters:

  • epoch

    Integer value

Note

You can also pass the epoch directly when reporting comet_ml.Experiment.log_metric.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
import comet_ml
import torch
from torch import nn
from torch.utils.data import DataLoader, TensorDataset

# Initialize Comet.ml experiment
comet_ml.init(project_name="comet-docs")
exp = comet_ml.Experiment(project_name="comet-docs")

# Create a dummy dataset, model, and dataloader
x = torch.randn(100, 10)
y = torch.randn(100, 1)
dataset = TensorDataset(x, y)
data_loader = DataLoader(dataset, batch_size=10)
model = nn.Sequential(
    nn.Linear(10, 5),
    nn.ReLU(),
    nn.Linear(5, 1)
)
loss_function = nn.MSELoss()

optimizer = torch.optim.Adam(model.parameters(), lr=0.01)

# Training loop
for epoch in range(10):
    exp.set_epoch(epoch)

    for data, target in data_loader:
        optimizer.zero_grad()
        output = model(data)
        loss = loss_function(output, target)
        loss.backward()
        optimizer.step()

        # Log loss every epoch
        exp.log_metric("loss", loss.item(), epoch=epoch)

# End the experiment
exp.end()

set_filename

set_filename(fname) -> None

Sets the current experiment filename.

Parameters:

  • fname (str) –

    The script's filename.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
import comet_ml

# Initialize Comet experiment
comet_ml.init(project_name="comet-docs")
exp = comet_ml.Experiment(project_name="comet-docs")

exp.set_filename('new_experiment.py')

# End the experiment
exp.end()

set_model_graph

set_model_graph(graph, overwrite=False) -> None

Sets the current experiment's computational graph.

Parameters:

  • graph (Any) –

    A model's computational graph.

  • overwrite (bool, default: False ) –

    If True, will overwrite a previously logged computational graph definition.

Note

The computational graph will be defined in the Graph definition tab within the Comet UI of the logged experiment.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
import comet_ml
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense

# Initialize Comet.ml experiment
comet_ml.init(project_name="comet-docs")
exp = comet_ml.Experiment(project_name="comet-docs")

# Define a simple Sequential model
model = Sequential([
    Dense(64, activation='relu', input_shape=(10,)),
    Dense(10, activation='softmax')
])

# Convert the model to its TensorFlow graph representation
graph = model.to_json()
exp.set_model_graph(graph)

# End the experiment
exp.end()

set_name

set_name(name)

Set a name for the experiment. Useful for filtering and searching on Comet.ml. Will shown by default under the Other tab.

Parameters:

  • name (str) –

    A name for the experiment.

set_offline_zip_uploader

set_offline_zip_uploader(upload_callback: UploadCallback) -> None

This method allows you to specify a callback function that will be invoked to upload the offline ZIP archive created if a connectivity issue occurs. The callback function receives the file path to the offline ZIP archive and perform the upload before the job ends.

We also provide a callback to upload to S3 directly, see: comet_ml.get_s3_uploader.

Parameters:

  • upload_callback (UploadCallback) –

    A user-defined function that takes a single argument, file_path (str), which is the path to the offline ZIP archive. The function should handle the upload process to the desired location.

Example
def custom_uploader(file_path: str) -> None:
    # Implement your upload logic here
    print(f"Uploading {file_path} to the cloud storage.")
    # Upload logic goes here

experiment.set_offline_zip_uploader(custom_uploader)
Notes
  • The callback function is invoked at the end of the job if an offline fallback occured after a connectivity issue and the SDK couldn't recover from it.
  • The callback function is responsible for handling any errors that occur during the upload process.
  • If the callback function fails to upload the ZIP archive, an error log message will be printed without retrying the upload.

set_os_packages

set_os_packages() -> None

Reads the installed os packages and reports them to server as a message.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
import comet_ml

# Initialize Comet experiment
comet_ml.init(project_name="os_package_logging")
exp = comet_ml.Experiment(project_name="comet-docs")

# Log the OS packages installed on the system
exp.set_os_packages()

# End the experiment
exp.end()

set_pip_packages

set_pip_packages() -> None

Get the installed pip packages using pkg resources and reports them to server as a message.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
import comet_ml

# Initialize Comet experiment
comet_ml.init(project_name="comet-docs")
exp = comet_ml.Experiment(project_name="comet-docs")

# Log the Python packages and their versions
exp.set_pip_packages()

# End the experiment
exp.end()

set_step

set_step(step) -> None

Sets the current step in a training process. In Deep Learning, each step is after feeding a single batch into the network. This is used to generate correct plots on Comet.

Parameters:

  • step

    The current step number.

Note

You can also pass the step directly when reporting comet_ml.Experiment.log_metric, and comet_ml.Experiment.log_parameter.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
import comet_ml
import numpy as np
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense

# Initialize Comet experiment
comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

# Create a simple neural network model
model = Sequential([
    Dense(64, activation='relu', input_shape=(10,)),
    Dense(1, activation='sigmoid')
])
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

# Generate dummy data
X_train = np.random.random((1000, 10))
y_train = np.random.randint(2, size=(1000, 1))

# Train the model with detailed step tracking
for epoch in range(10):
    for batch in range(100):
        step = epoch * 100 + batch
        exp.set_step(step)  # Set step for detailed tracking
        model.train_on_batch(X_train, y_train)

# End the experiment
exp.end()

stop_early

stop_early(epoch)

Used to programmatically stop an experiment early.

Parameters:

  • epoch

    The epoch number at early stop.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
import comet_ml
import numpy as np
import tensorflow as tf
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import OneHotEncoder
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.losses import CategoricalCrossentropy

# Initialize Comet experiment
comet_ml.init(project_name="comet-docs")
exp = comet_ml.Experiment(project_name="comet-docs")

# Set up simple neural network example
data = load_iris()
X = data.data
y = data.target.reshape(-1, 1)

encoder = OneHotEncoder(sparse=False)
y = encoder.fit_transform(y)

X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)

model = Sequential([
    Dense(10, activation='relu', input_shape=(X_train.shape[1],)),
    Dense(3, activation='softmax')
])

model.compile(optimizer='adam', loss=CategoricalCrossentropy(), metrics=['accuracy'])

# Train model with manual early stopping
for epoch in range(100):
    history = model.fit(X_train, y_train, epochs=1, validation_data=(X_val, y_val))
    val_loss = history.history['val_loss'][0]
    exp.log_metric("validation_loss", val_loss, epoch=epoch)

    # Check for early stopping condition
    if val_loss < 0.5:  # Example threshold for early stopping
        print(f"Stopping early at epoch {epoch} due to reaching loss threshold.")
        exp.stop_early(epoch=epoch)
        break

test

test()

A context manager to mark the beginning and the end of the testing phase. This allows you to provide a namespace for metrics/params.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
import comet_ml
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

# Initialize Comet experiment
comet_ml.init(project_name="comet-docs")
exp = comet_ml.Experiment(project_name="comet-docs")

# Load a dataset and create a model
data = load_iris()
X = data.data
y = data.target
X_train, X_test, y_train, y_test = train_test_split(X, y,
                                                    test_size=0.2,
                                                    random_state=42)
model = LogisticRegression(max_iter=200)
model.fit(X_train, y_train)

# Start test context using a with statement
with exp.test():
    # Perform predictions on the test set
    pred = model.predict(X_test)
    test_acc = accuracy_score(y_test, pred)

    # Log test accuracy
    exp.log_metric("test_accuracy", test_acc)

# End the experiment
exp.end()

train

train()

A context manager to mark the beginning and the end of the training phase. This allows you to provide a namespace for metrics/params.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
import comet_ml
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

# Initialize Comet experiment
comet_ml.init(project_name="comet-docs")
exp = comet_ml.Experiment(project_name="comet-docs")

# Load a dataset and create a model
data = load_iris()
X = data.data
y = data.target
X_train, X_test, y_train, y_test = train_test_split(X, y,
                                                    test_size=0.2,
                                                    random_state=42)
model = LogisticRegression(max_iter=200)

# Start train context using a with statement
with exp.train():

    # Train model
    model.fit(X_train, y_train)

    # Perform predictions on the train set
    pred = model.predict(X_train)
    train_acc = accuracy_score(y_train, pred)

    # Log train accuracy
    exp.log_metric("train_accuracy", train_acc)

# End the experiment
exp.end()

validate

validate()

A context manager to mark the beginning and the end of the validating phase. This allows you to provide a namespace for metrics/params.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
import comet_ml
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

# Initialize Comet.ml experiment
comet_ml.init(project_name="comet-docs")
exp = comet_ml.Experiment(project_name="comet-docs")

# Load a dataset and create a model
data = load_iris()
X = data.data
y = data.target
X_train, X_validation, y_train, y_validation = train_test_split(X, y,
                                                                test_size=0.2,
                                                                random_state=42)
model = LogisticRegression(max_iter=200)
model.fit(X_train, y_train)

# Start validation context using a with statement
with exp.validate():
    # Perform predictions on the validation set
    pred = model.predict(X_validation)
    val_acc = accuracy_score(y_validation, pred)

    # Log validation accuracy
    exp.log_metric("validation_accuracy", val_acc)

# End the experiment
exp.end()
Sep. 30, 2024