comet_ml.OfflineExperiment ¶
Attributes¶
artifact_name_max_length instance-attribute
¶
artifact_name_max_length: Optional[int] = None
asset_name_max_length instance-attribute
¶
asset_name_max_length: Optional[int] = None
image_name_max_length instance-attribute
¶
image_name_max_length: Optional[int] = None
url property
¶
url
Get the url of the experiment.Example
1
2
3
4
5
import comet_ml
comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")
print(exp.url)
Functions¶
__init__ ¶
__init__(
project_name: Optional[str] = None,
workspace: Optional[str] = None,
log_code: Optional[bool] = True,
log_graph: Optional[bool] = True,
auto_param_logging: Optional[bool] = True,
auto_metric_logging: Optional[bool] = True,
parse_args: Optional[bool] = True,
auto_output_logging: Optional[str] = "default",
log_env_details: Optional[bool] = True,
log_git_metadata: Optional[bool] = True,
log_git_patch: Optional[bool] = True,
disabled: Optional[bool] = False,
offline_directory: Optional[str] = None,
log_env_gpu: Optional[bool] = True,
log_env_host: Optional[bool] = True,
api_key: Optional[str] = None,
display_summary: Optional[bool] = None,
log_env_cpu: Optional[bool] = True,
log_env_network: Optional[bool] = True,
log_env_disk: Optional[bool] = True,
display_summary_level: Optional[int] = None,
auto_weight_logging: Optional[bool] = None,
auto_log_co2: Optional[bool] = False,
auto_metric_step_rate: Optional[int] = 10,
auto_histogram_tensorboard_logging: Optional[bool] = False,
auto_histogram_epoch_rate: Optional[int] = 1,
auto_histogram_weight_logging: Optional[bool] = False,
auto_histogram_gradient_logging: Optional[bool] = False,
auto_histogram_activation_logging: Optional[bool] = False,
experiment_key: Optional[str] = None,
distributed_node_identifier: Optional[str] = None,
**kwargs
) -> None
Creates a new experiment and serialize it on disk. The experiment file will need to be uploaded manually later to appear on the frontend.
Parameters:
project_name
(str
, default:None
) –Send your experiment to a specific project. Otherwise, will be sent to
Uncategorized Experiments
. If project name does not already exist Comet.ml will create a new project.workspace
(str
, default:None
) –Attach an experiment to a project that belongs to this workspace
log_code
(bool
, default:True
) –Allows you to enable/disable code logging
log_graph
(bool
, default:True
) –Allows you to enable/disable automatic computation graph logging.
auto_param_logging
(bool
, default:True
) –Allows you to enable/disable hyper-parameters logging
auto_metric_logging
(bool
, default:True
) –Allows you to enable/disable metrics logging
auto_metric_step_rate
(int
, default:10
) –Controls how often batch metrics are logged
auto_histogram_tensorboard_logging
(bool
, default:False
) –Allows you to enable/disable automatic histogram logging
auto_histogram_epoch_rate
(int
, default:1
) –Controls how often histograms are logged
auto_histogram_weight_logging
(bool
, default:False
) –Allows you to enable/disable automatic histogram logging of biases and weights
auto_histogram_gradient_logging
(bool
, default:False
) –Allows you to enable/disable automatic histogram logging of gradients
auto_histogram_activation_logging
(bool
, default:False
) –Allows you to enable/disable automatic histogram logging of activations
auto_output_logging
(str
, default:'default'
) –Allows you to select which output logging mode to use. You can pass
"native"
which will log all output even when it originated from a C native library. You can also pass"simple"
which will work only for output made by Python code. If you want to disable automatic output logging, you can passFalse
. The default is"default"
which will detect your environment and deactivate the output logging for IPython and Jupyter environment and sets"native"
in the other cases.auto_log_co2
(bool
, default:False
) –Automatically tracks the CO2 emission of this experiment if
codecarbon
package is installed in the environmentparse_args
(bool
, default:True
) –Allows you to enable/disable automatic parsing of CLI arguments
log_env_details
(bool
, default:True
) –Log various environment information in order to identify where the script is running
log_env_gpu
(bool
, default:True
) –Allow you to enable/disable the automatic collection of gpu details and metrics (utilization, memory usage etc..).
log_env_details
must also be true.log_env_cpu
(bool
, default:True
) –Allow you to enable/disable the automatic collection of cpu details and metrics (utilization, memory usage etc..).
log_env_details
must also be true.log_env_network
(bool
, default:True
) –Allow you to enable/disable the automatic collection of network details and metrics (sent, receive rates, etc..).
log_env_details
must also be true.log_env_disk
(bool
, default:True
) –Allow you to enable/disable the automatic collection of disk utilization metrics (usage, IO rates, etc.).
log_env_details
must also be true.log_env_host
(bool
, default:True
) –Allow you to enable/disable the automatic collection of host information (ip, hostname, python version, user etc...).
log_env_details
must also be true.log_git_metadata
(bool
, default:True
) –Allow you to enable/disable the automatic collection of git details
display_summary_level
(int
, default:None
) –Control the summary detail that is displayed on the console at end of experiment. If 0, the summary notification is still sent. Valid values are 0 to 2.
disabled
(bool
, default:False
) –Allows you to disable all network communication with the Comet.ml backend. It is useful when you want to test to make sure everything is working, without actually logging anything.
offline_directory
(str
, default:None
) –The directory used to save the offline archive for the experiment.
experiment_key
(str
, default:None
) –If provided, will be used as the experiment key. If an experiment with the same key already exists, it will raise an Exception during upload. Could be set through configuration as well. Must be an alphanumeric string whose length is between 32 and 50 characters.
add_tag ¶
add_tag(tag: str) -> bool
Add a tag to the experiment. Tags will be shown in the dashboard.
Parameters:
tag
(str
) –A tag to add to the experiment.
add_tags ¶
add_tags(tags: List[str]) -> bool
Add several tags to the experiment. Tags will be shown in the dashboard. Args: tags (list): Tags list to add to the experiment.
clean ¶
clean()
Clean the experiment loggers, useful in case you want to debug your scripts with IPDB.
create_confusion_matrix ¶
create_confusion_matrix(
y_true=None,
y_predicted=None,
labels=None,
matrix=None,
title="Confusion Matrix",
row_label="Actual Category",
column_label="Predicted Category",
max_examples_per_cell=25,
max_categories=25,
winner_function=None,
index_to_example_function=None,
cache=True,
selected=None,
images=None,
**kwargs
)
Create a confusion matrix for use over multiple epochs.
Parameters:
y_true
(list
, default:None
) –List of vectors representing the targets, or a list of integers representing the correct label. If not provided, then matrix may be provided.
y_predicted
(list
, default:None
) –List of vectors representing predicted values, or a list of integers representing the output. If not provided, then matrix may be provided.
labels
(list
, default:None
) –A list of strings that name of the columns and rows, in order.
matrix
(list
, default:None
) –The confusion matrix (list of lists). Must be square, if given. If not given, then it is possible to provide y_true and y_predicted.
title
(str
, default:'Confusion Matrix'
) –A custom name to be displayed. By default, it is "Confusion Matrix".
row_label
(str
, default:'Actual Category'
) –Label for rows.
column_label
(str
, default:'Predicted Category'
) –Label for columns.
max_examples_per_cell
(int
, default:25
) –Maximum number of examples per cell.
max_categories
(int
, default:25
) –Max number of columns and rows to use.
winner_function
(callable
, default:None
) –A function that takes in an entire list of rows of patterns, and returns the winning category for each row.
index_to_example_function
(callable
, default:None
) –A function that takes an index and returns either a number, a string, a URL, or a {"sample": str, "assetId": str} dictionary. See below for more info. If left blank, the function returns a number representing the index of the example.
cache
(bool
, default:True
) –Should the results of index_to_example_function be cached and reused?
selected
(list
, default:None
) –A list of selected category indices. These are the rows/columns that will be shown. By default, select is None. If the number of categories is greater than max_categories, and selected is not provided, then selected will be computed automatically by selecting the most confused categories.
images
(list
, default:None
) –A list of data that can be passed to Experiment.log_image().
kwargs
(optional
, default:{}
) –Any extra keywords and their values will be passed onto the index_to_example_function.
Note
For more details and example uses, please see: Log confusion matrices. Uses winner_function to compute winning categories for y_true and y_predicted, if they are vectors.
Also, for more low-level information, see comet_ml.utils.ConfusionMatrix
Example
1 2 3 4 5 6 7 8 9 10 11 12 |
|
create_embedding_image ¶
create_embedding_image(
image_data,
image_size,
image_preprocess_function=None,
image_transparent_color=None,
image_background_color_function=None,
)
Create an embedding image (a sprite sheet). Returns the image and the url to the image.
Parameters:
image_data
(Any
) –List of arrays or Images.
image_size
(list
) –The size of each image.
image_preprocess_function
(callable
, default:None
) –If image_data is an array, apply this function to each element first.
image_transparent_color
(list
, default:None
) –A (red, green, blue) tuple.
image_background_color_function
(callable
, default:None
) –A function that takes an index, and returns a (red, green, blue) color tuple.
Returns:
tuple
–Returns the image and url
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
|
disable_mp ¶
disable_mp()
Disabling the auto-collection of metrics and monkey-patching of the Machine Learning frameworks.Example
1
2
3
4
5
6
7
8
9
10
import comet_ml
comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")
exp.disable_mp()
exp.log_parameter("example_parameter", 123)
exp.end()
end ¶
end()
The Only one Experiment can be alive in a Python process, if you wish to start a new Experiment you should first end the previous Experiment by calling the end()
method finalizes an experiment by ensuring all data (i.e., parameters, metrics, asset, etc.) are uploaded to Comet before it returns. Unless you are running in a Jupyter Notebook, using the end()
method is optional as it will be called automatically by Comet when the script ends.Note
end()
method and then start the new Experiment.Example
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
import comet_ml
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
# Initialize Comet
comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")
# Run an experiment
data = load_iris()
X = data.data
y = data.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
model = LogisticRegression()
model.fit(X_train, y_train)
y_pred_train = model.predict(X_train)
acc = accuracy_score(y_train, y_pred_train)
# Log accuracy metric to Comet
exp.log_metric("accuracy", acc)
# End the experiment
exp.end()
flush ¶
flush() -> bool
Flush all pending data to the Comet server. It works similar to end but without ending the run. This is a blocking operation that will wait for all the data logged so far to be delivered to the Comet server.
Returns:
bool
–Returns a Boolean indicating whether the flush was successful or not.
get_artifact ¶
get_artifact(*args, **kwargs)
Getting an artifact is only supported for Online Experiment.
get_callback ¶
get_callback(framework, *args, **kwargs)
Get a callback for a particular framework.
Parameters:
framework
(str
) –Specifies the machine learning framework for which you want a callback.
Returns:
callable
–Callback object that is compatible with the framework specified in the call.
Note
When framework == 'keras' then return an instance of Comet.ml's Keras callback.
When framework == 'tf-keras' then return an instance of Comet.ml's TensorflowKeras callback.
When framework == "tf-estimator-train" then return an instance of Comet.ml's Tensorflow Estimator Train callback.
When framework == "xgboost" then return an instance of Comet.ml's xgboost.callback.TrainingCallback implementation
When framework == "fastai" then return an instance of Comet.ml's fastai.learner.Callback implementation.
The keras callbacks are added to your Keras model.fit()
callbacks list automatically to report model training metrics to Comet.ml so you do not need to add them manually.
The lightgbm callback is added to the lightgbm.train()
callbacks list automatically to report model training metrics to Comet.ml so you do not need to add it manually.
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
|
get_keras_callback ¶
get_keras_callback()
This method is deprecated. See Experiment.get_callback("keras"). Get a callback for the Keras framework.
get_key ¶
get_key()
Returns the experiment key, useful for using with the ExistingExperiment class
Returns:
str
–Experiment Key (String)
Example
1 2 3 4 5 6 7 8 |
|
get_metric ¶
get_metric(name: str) -> Any
Get the last value for a metric from those logged.
Parameters:
name
(str
) –The name of the metric to get. This method will return the last value logged to this metric.
get_name ¶
get_name() -> str
Get the name of the experiment, if one.Example
1
2
3
4
5
6
import comet_ml
comet_ml.login()
exp = comet_ml.OfflineExperiment()
exp.set_name("My Name")
print(exp.get_name())
get_other ¶
get_other(name: str) -> Any
Get an other from those logged.
Parameters:
name
(str
) –The name of the other parameter to get.
get_parameter ¶
get_parameter(name)
Get a parameter that was logged previously in an Experiment.
Parameters:
name
(str
) –The name of the parameter to get.
Returns:
str
–Parameter value
Note
If this method is called inside a context, like test, train, validate or context_manager, the current context name will be automatically added at the front of parameter name.
Raises a KeyError if parameter with given name not found.
Example
import comet_ml
comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")
exp.log_parameter("learning_rate", 0.0001)
exp.log_parameter("batch_size", 64)
learning_rate = exp.get_parameter("learning_rate")
print(f"Learning Rate: {learning_rate}")
assert float(learning_rate) == 0.0001
batch_size = exp.get_parameter("batch_size")
print(f"Batch Size: {batch_size}")
assert int(batch_size) == 64
exp.end()
get_tags ¶
get_tags() -> List[str]
Return the tags of this experiment.
Returns:
List[str]
–List[str]: The list of tags.
log_asset ¶
log_asset(
file_data,
file_name=None,
overwrite=False,
copy_to_tmp=True,
step=None,
metadata=None,
)
Logs the Asset determined by file_data.
Parameters:
file_data
(Any
) –Either the file path of the file you want to log, or a file-like asset.
file_name
(str
, default:None
) –A custom file name to be displayed. If not provided the filename from the
file_data
argument will be used.overwrite
(bool
, default:False
) –If True will overwrite all existing assets with the same name.
copy_to_tmp
(bool
, default:True
) –(Deprecated) If
file_data
is a file-like object, then this flag determines if the file is first copied to a temporary file before upload. Ifcopy_to_tmp
is False, then it is sent directly to the cloud.step
(int
, default:None
) –Used to associate the asset to a specific step.
metadata
(Any
, default:None
) –Some additional data to attach to the the audio asset. Must be a JSON-encodable dict.
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
|
log_asset_data ¶
log_asset_data(
data,
name=None,
overwrite=False,
step=None,
metadata=None,
file_name=None,
epoch=None,
)
Logs the data given (str, binary, or JSON).
Parameters:
data
–data to be saved as asset.
name
–A custom file name to be displayed. If not provided the filename from the temporary saved file will be used.
overwrite
–If True will overwrite all existing assets with the same name.
step
–Used to associate the asset to a specific step.
epoch
–Used to associate the asset to a specific epoch.
metadata
–Some additional data to attach to the the asset data. Must be a JSON-encodable dict.
Note
See also comet_ml.APIExperiment.get_asset_list, comet_ml.APIExperiment.get_asset, and comet_ml.APIExperiment.get_asset_by_name.
Example
1 2 3 4 5 6 7 8 9 10 |
|
log_asset_folder ¶
log_asset_folder(folder, step=None, log_file_name=None, recursive=False)
Logs all the files located in the given folder as assets.
Parameters:
folder
(str
) –the path to the folder you want to log.
step
(int
, default:None
) –used to associate the asset to a specific step.
log_file_name
(bool
, default:None
) –if True, log the file path with each file.
recursive
(bool
, default:False
) –if True, recurse folder and save file names.
Note
If log_file_name is set to True, each file in the given folder will be logged with the following name schema: FOLDER_NAME/RELPATH_INSIDE_FOLDER
. Where FOLDER_NAME
is the basename of the given folder and RELPATH_INSIDE_FOLDER
is the file path relative to the folder itself.
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
|
log_audio ¶
log_audio(
audio_data: Any,
sample_rate: Optional[int] = None,
normalize_data: str = "PCM_16_BIT",
file_name: Optional[str] = None,
metadata: Optional[Dict[str, Any]] = None,
overwrite: bool = False,
copy_to_tmp: bool = True,
step: Optional[int] = None,
) -> Optional[Dict[str, Optional[str]]]
Logs the audio Asset determined by audio data.
Parameters:
audio_data
(str | array
) –Either the file path of the file you want to log, or a numpy array given to
scipy.io.wavfile.write
for wav conversion.sample_rate
(int
, default:None
) –The sampling rate given to
scipy.io.wavfile.write
for creating the wav file.normalize_data
(str
, default:'PCM_16_BIT'
) –The parameter that determines whether
audio_data
should be normalized to fit within the specific numerical bounds (e.g. 16-bit PCM). If you set this parameter to 'OFF', it is your responsibility to ensure thataudio_data
is compatible with the format supported byscipy.io.wavfile.write
. Supported values: 'PCM_16_BIT', 'OFF'.file_name
(str
, default:None
) –A custom file name to be displayed. If not provided, the filename from the
audio_data
argument will be used.metadata
(dict
, default:None
) –Some additional data to attach to the audio asset. Must be a JSON-encodable dict.
overwrite
(bool
, default:False
) –If True will overwrite all existing assets with the same name.
copy_to_tmp
(bool
, default:True
) –(Deprecated) If
audio_data
is a numpy array, then this flag determines if the WAV file is first copied to a temporary file before upload. Ifcopy_to_tmp
is False, then it is sent directly to the cloud.step
(int
, default:None
) –Used to associate the audio asset to a specific step.
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
|
log_code ¶
log_code(
file_name=None, folder=None, code=None, code_name=None, overwrite=False
)
Logs additional source code files. This method allows logging code in different ways:
- Without parameters, logs the code from the file calling this method.
- With
file_name
, logs the code from the specified file. - Using
folder
, logs all files' code in that folder. - Using
code
, logs the content as source code.
Parameters:
file_name
(str
, default:None
) –File path to log.
folder
(str
, default:None
) –Folder path where the code files are stored.
code
(str
, default:None
) –Source code, either as text or a file-like object (such as StringIO). If passed,
code_name
is mandatory.code_name
(str
, default:None
) –Name of the source code file.
overwrite
(bool
, default:False
) –If True, overwrites existing code with the same name.
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
|
log_confusion_matrix ¶
log_confusion_matrix(
y_true=None,
y_predicted=None,
matrix=None,
labels=None,
title="Confusion Matrix",
row_label="Actual Category",
column_label="Predicted Category",
max_examples_per_cell=25,
max_categories=25,
winner_function=None,
index_to_example_function=None,
cache=True,
file_name="confusion-matrix.json",
overwrite=False,
step=None,
epoch=None,
images=None,
selected=None,
**kwargs
)
Logs a confusion matrix.
Parameters:
y_true
(list
, default:None
) –List of vectors representing the targets, or a list of integers representing the correct label. If not provided, then matrix may be provided.
y_predicted
(list
, default:None
) –List of vectors representing predicted values, or a list of integers representing the output. If not provided, then matrix may be provided.
labels
(list
, default:None
) –A list of strings that name of the columns and rows, in order.
matrix
(list
, default:None
) –The confusion matrix (list of lists). Must be square, if given. If not given, then it is possible to provide y_true and y_predicted.
title
(str
, default:'Confusion Matrix'
) –A custom name to be displayed. By default, it is "Confusion Matrix".
row_label
(str
, default:'Actual Category'
) –Label for rows.
column_label
(str
, default:'Predicted Category'
) –Label for columns.
max_examples_per_cell
(int
, default:25
) –Maximum number of examples per cell.
max_categories
(int
, default:25
) –Max number of columns and rows to use.
winner_function
(callable
, default:None
) –A function that takes in an entire list of rows of patterns, and returns the winning category for each row.
index_to_example_function
(callable
, default:None
) –A function that takes an index and returns either a number, a string, a URL, or a {"sample": str, "assetId": str} dictionary. See below for more info. If left blank, the function returns a number representing the index of the example.
cache
(bool
, default:True
) –Should the results of index_to_example_function be cached and reused?
file_name
(str
, default:'confusion-matrix.json'
) –The name of the file that the confusion matrix will be saved as when logged.
overwrite
(bool
, default:False
) –If set to True, the existing confusion matrix with the same file name will be overwritten. If False, a new entry is created, preserving the previous versions.
step
(int
, default:None
) –The step parameter can be used to specify the particular step or iteration in the training process at which the confusion matrix is logged.
epoch
(int
, default:None
) –Similar to the step parameter, epoch specifically denotes the training epoch.
images
(list
, default:None
) –A list of data that can be passed to Experiment.log_image().
selected
(list
, default:None
) –A list of selected category indices. These are the rows/columns that will be shown. By default, select is None. If the number of categories is greater than max_categories, and selected is not provided, then selected will be computed automatically by selecting the most confused categories.
kwargs
(Any
, default:{}
) –Any extra keywords and their values will be passed onto the index_to_example_function.
Note
For more details and example uses, please see: Log confusion matrices. Uses winner_function to compute winning categories for y_true and y_predicted, if they are vectors.
Also, for more low-level information, see comet_ml.utils.ConfusionMatrix
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 |
|
log_curve ¶
log_curve(name, x, y, overwrite=False, step=None)
Log timeseries data.
Parameters:
name
–Name of data.
x
–list of x-axis values.
y
–list of y-axis values.
overwrite
–If True, overwrite previous log.
step
–The step value.
Example
1 2 3 4 5 6 7 8 |
|
log_dataframe_profile ¶
log_dataframe_profile(
dataframe: Any,
name: Optional[str] = "dataframe",
minimal: bool = False,
log_raw_dataframe: bool = True,
dataframe_format: str = "json",
**format_kwargs: Any
) -> Optional[Dict[str, Optional[Dict[str, str]]]]
Log a pandas DataFrame profile as an asset. Optionally, can also log the dataframe.
Parameters:
dataframe
(Any
) –the dataframe to profile and/or log
name
(str
, default:'dataframe'
) –the basename (without extension) of the dataframe assets.
minimal
(bool
, default:False
) –if True, create a minimal profile. Useful for large datasets.
log_raw_dataframe
(bool
, default:True
) –Log the dataframe as an asset (same as calling
log_table()
)dataframe_format
(str
, default:'json'
) –The format. for optionally logging the dataframe.
format_kwargs
(Any
, default:{}
) –Keyword args for dataframe logging as an asset.
Example
1 2 3 4 5 6 7 8 9 |
|
See also: Experiment.log_table(pandas_dataframe)
log_dataset_hash ¶
log_dataset_hash(data) -> None
Used to log the hash of the provided object.
Parameters:
data
–Any object that when casted to string (e.g str(data)) returns a value that represents the underlying data.
Note
This is a best-effort hash computation which is based on the md5 hash of the underlying string representation of the object data. Developers are encouraged to implement their own hash computation that's tailored to their underlying data source. That could be reported as experiment.log_parameter("dataset_hash", your_hash)
.
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
|
log_dataset_info ¶
log_dataset_info(
name: Optional[str] = None,
version: Optional[str] = None,
path: Optional[str] = None,
) -> None
Used to log information about your dataset.
Parameters:
name
(str
, default:None
) –A string representing the name of the dataset.
version
(str
, default:None
) –A string representing a version identifier.
path
(str
, default:None
) –A string that represents the path to the dataset. Potential values could be a file system path, S3 path, or Database query.
Note
At least one argument should be included when calling this method. The logged values will show up in the Other
tab of the Comet UI.
Example
1 2 3 4 5 6 7 8 |
|
log_dependency ¶
log_dependency(name, version) -> None
Reports name,version to the Installed Packages
tab on Comet. Useful to track dependencies.
Parameters:
name
–Any type of key (str,int,float..)
version
–Any type of value (str,int,float..)
log_embedding ¶
log_embedding(
vectors,
labels,
image_data=None,
image_size=None,
image_preprocess_function=None,
image_transparent_color=None,
image_background_color_function=None,
title="Comet Embedding",
template_filename=None,
group=None,
)
Logging embedding is only supported for Online Experiment at the moment
log_epoch_end ¶
log_epoch_end(epoch_cnt, step=None) -> None
Logs that the epoch finished. Required for progress bars.
Parameters:
epoch_cnt
(int
) –The current count of completed epochs, indicating how many epochs have finished in the training cycle.
step
(int
, default:None
) –The step count at which the epoch ends.
Example
1 2 3 4 5 6 7 8 9 |
|
log_figure ¶
log_figure(figure_name=None, figure=None, overwrite=False, step=None)
Log a visual representation of the provided figure to Comet in SVG format.
Parameters:
figure_name
(str
, default:None
) –A descriptive name for the figure.
figure
(Any
, default:None
) –The figure to be logged. Accepts Plotly Figures, Matplotlib Figures, or Seaborn simple plots. If not provided, the function will log the current global Matplotlib Pyplot figure.
overwrite
(bool
, default:False
) –Determines whether to overwrite an existing figure with the same name.
step
(int
, default:None
) –Associates the figure asset with a specific step in the Comet experiment.
Note
Kaleido is required to be installed in order to log plotly figures.
log_histogram_3d ¶
log_histogram_3d(
values, name=None, step=None, epoch=None, metadata=None, **kwargs
)
Logs a histogram of values for a 3D chart as an asset for this experiment. Calling this method multiple times with the same name and incremented steps will add additional histograms to the 3D chart on Comet.ml.
Parameters:
values
–A list, tuple, array (any shape) to summarize, or a Histogram object.
name
–Name of summary.
step
–Used as the Z axis when plotting on Comet.
epoch
–Used as the Z axis when plotting on Comet.
metadata
–Used for items like prefix for histogram name.
kwargs
–Additional keyword arguments for histogram.
Note
This method requires that step is either given here, or has been set elsewhere. For example, if you are using an auto- logger that sets step then you don't need to set it here.
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
|
log_html ¶
log_html(html, clear=False) -> None
Reports any HTML blob to the HTML
tab on Comet.ml. Useful for creating your own rich reports. The HTML will be rendered as an Iframe. Inline CSS/JS supported.
Parameters:
html
–Any html string.
clear
–When setting clear=True it will remove all previous html.
Example
1 2 3 4 5 6 7 |
|
log_html_url ¶
log_html_url(url, text=None, label=None)
Easy to use method to add a link to a URL in the HTML
tab on Comet.
Parameters:
url
–a link to a file or notebook, for example
text
–text to use a clickable word or phrase (optional; uses url if not given)
label
–text that precedes the link
Example
Logging a link:
1 2 3 4 5 6 7 |
|
Adds html similar to:
<a href="https://my-company.com/file.txt">
https://my-company.com/file.txt
</a>
Logging a link with text:
1 2 3 4 5 6 7 8 |
|
Adds html similar to:
<a href="https://my-company.com/file.txt">File</a>
Logging a link with text and a label:
1 2 3 4 5 |
|
Adds html similar to:
Label: <a href="https://my-company.com/file.txt">File</a>
log_image ¶
log_image(
image_data: Any,
name: Optional[str] = None,
overwrite: bool = False,
image_format: str = "png",
image_scale: float = 1.0,
image_shape: Optional[Tuple[int, int]] = None,
image_colormap: Optional[str] = None,
image_minmax: Optional[Tuple[int, int]] = None,
image_channels: str = "last",
copy_to_tmp: bool = True,
step: Optional[int] = None,
annotations: Optional[List[Layer]] = None,
metadata: Optional[Dict[str, Any]] = None,
) -> Optional[Dict[str, str]]
Logs the image. Images are displayed on the Graphics tab in Comet.
Parameters:
image_data
(Any
) –Is one of the following: - a path (string) to an image - a file-like object containing an image - a numpy matrix - a TensorFlow tensor - a PyTorch tensor - a list or tuple of values - a PIL Image
name
(str
, default:None
) –A custom name to be displayed on the dashboard. If not provided the filename from the
image_data
argument will be used if it is a path.overwrite
(bool
, default:False
) –If another image with the same name exists, it will be overwritten if overwrite is set to True.
image_format
(str
, default:'png'
) –If the image_data is actually something that can be turned into an image, this is the format used. Typical values include 'png' and 'jpg'.
image_scale
(float
, default:1.0
) –If the image_data is actually something that can be turned into an image, this will be the new scale of the image.
image_shape
(tuple
, default:None
) –If the image_data is actually something that can be turned into an image, this is the new shape of the array. Dimensions are (width, height) or (width, height, colors) where
colors
is 3 (RGB) or 1 (grayscale).image_colormap
(str
, default:None
) –If the image_data is actually something that can be turned into an image, this is the colormap used to colorize the matrix.
image_minmax
(tuple
, default:None
) –If the image_data is actually something that can be turned into an image, this is the (min, max) used to scale the values. Otherwise, the image is autoscaled between (array.min, array.max).
image_channels
(str
, default:'last'
) –If the image_data is actually something that can be turned into an image, this is the setting that indicates where the color information is in the format of the 2D data. 'last' indicates that the data is in (rows, columns, channels) where 'first' indicates (channels, rows, columns).
copy_to_tmp
(bool
, default:True
) –(Deprecated) If
image_data
is not a file path, then this flag determines if the image is first copied to a temporary file before upload. Ifcopy_to_tmp
is False, then it is sent directly to the cloud.step
(int
, default:None
) –Used to associate the image asset to a specific step.
annotations
(list
, default:None
) –Annotations contains a list of annotation layers. Each annotation layer can contain bounding boxes and/or polygons. See example below.
metadata
(dict
, default:None
) –Additional metadata to be associated with logged image.
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 |
|
log_metric ¶
log_metric(name, value, step=None, epoch=None, include_context=True) -> None
Logs a general metric (i.e accuracy, f1).
See also log_metrics.
Parameters:
name
–Name of your metric.
value
–The value of your metric, we recommend that this is a numerical value.
step
–Used as the X axis when plotting on Comet.
epoch
–Used as the X axis when plotting on Comet.
include_context
–If set to True, the current context will be logged along the metric.
Note
Comet guarantees to store 15,000 data points for each metric. If more than 15,000 data points are reported we perform a form of reservoir sub-sampling and download sample the metric values.
Example
1 2 3 |
|
log_metrics ¶
log_metrics(
dic: Dict[str, Any],
prefix: Optional[str] = None,
step: Optional[int] = None,
epoch: Optional[int] = None,
)
Logs a key,value dictionary of metrics. See also log_metric
log_model ¶
log_model(
name,
file_or_folder,
file_name=None,
overwrite=False,
metadata=None,
copy_to_tmp=True,
prepend_folder_name=True,
)
Logs the model data under the name. Data can be a file path, a folder path or a file-like object.
Parameters:
name
(str
) –The name of the model
file_or_folder
(str | dict
) –The model data; can be a file path, a folder path or a file-like object.
file_name
(str
, default:None
) –The name of the model data. Used with file-like objects or files only.
overwrite
(bool
, default:False
) –If True, then overwrite previous versions Does not apply to folders.
metadata
(dict
, default:None
) –Some additional data to attach to the the data. Must be a JSON-encodable dict.
copy_to_tmp
(bool
, default:True
) –Depreacted. For file name or file-like; if True copy to temporary location before uploading; if False, then upload from current location
prepend_folder_name
(bool
, default:True
) –If True and logging a folder, prepend file path by the folder name.
Returns:
dict
–Dictionary of model URLs
log_notebook ¶
log_notebook(filename, overwrite=False)
Log a Jupyter Notebook file as an asset.
Parameters:
filename
(str
) –The path and name of the notebook to be logged.
overwrite
(bool
, default:False
) –If True, overwrites the previously logged notebook asset.
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
|
log_optimization ¶
log_optimization(
optimization_id: Optional[str] = None,
metric_name: Optional[str] = None,
metric_value: Any = None,
parameters: Optional[Dict] = None,
objective: Optional[str] = None,
) -> None
Logs an existing optimization result.
Parameters:
optimization_id
(str
, default:None
) –The id of the optimization result.
metric_name
(str
, default:None
) –The name of your metric
metric_value
(Any
, default:None
) –The value of the given metric.
parameters
(dict
, default:None
) –Additional parameters to be logged.
objective
(str
, default:None
) –The objective of the optimization, could be either maximum/minimum.
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
|
log_other ¶
log_other(key, value)
Reports a key and value to the Other
tab on Comet.ml. Useful for reporting datasets attributes, datasets path, unique identifiers etc.
Other
metadata can also be defined as environment variables. All environment variables that starts with COMET_LOG_OTHER_
will be logged automatically to the Experiment.
Parameters:
key
–Any type of key (str,int,float..)
value
–Any type of value (str,int,float..)
log_others ¶
log_others(dictionary)
Reports dictionary of key/values to the Other
tab on Comet.ml. Useful for reporting datasets attributes, datasets path, unique identifiers etc.
See log_other
Parameters:
dictionary
(dict
) –dict of key/values where value is any type of value (str,int,float..)
log_parameter ¶
log_parameter(name: Any, value: Any, step: Optional[Any] = None) -> None
Logs a parameter.
Parameters:
name
(Any
) –Name of your parameter.
value
(Any
) –The value of your parameter.
step
(Optional[Any]
, default:None
) –Used as the x-axis when plotting on Comet.
Note
It's encouraged to use log_other for additional values that are not hyper parameters.
If the same key is reported multiple times only the last reported value will be saved.
If this method is called inside a context, like test,train, validate or context_manager, the parameter will be stored with the current context name as a prefix.
See also log_parameters.
Example
1 2 3 4 5 6 7 8 9 10 11 |
|
log_parameters ¶
log_parameters(
parameters: Dict[str, Any],
prefix: Optional[str] = None,
step: Optional[int] = None,
nested_support: bool = True,
)
Logs a dictionary (or dictionary-like object) of multiple parameters.
Parameters:
parameters
(dict
) –This is a dictionary where each key is a string representing the name of the parameter, and the value is the parameter value itself, which can be of any data type that is serializable. The method logs each key-value pair as a parameter in the experiment.
prefix
(str
, default:None
) –This optional parameter allows you to add a prefix to the keys in the parameters dictionary when they are logged. This can be helpful for organizing or grouping parameters under a common namespace, making them easier to identify and analyze in the Comet UI.
step
(int
, default:None
) –This optional parameter can be used to associate the logged parameters with a particular step or iteration in an experiment. This is useful for tracking how parameters change over time, particularly in iterative processes like training a machine learning model over multiple epochs.
nested_support
(bool
, default:True
) –This parameter controls whether the method should support nested dictionaries. If set to True, the method can handle parameters that are dictionaries themselves and will flatten these into a format suitable for logging. Each key in a nested dictionary will be combined with its parent keys to create a single, flat key.
Note
See also log_parameter.
If you call this method multiple times with the same keys your values would be overwritten.
If this method is called inside a context, like test,train, validate or context_manager, the parameter will be stored with the current context name as a prefix.
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
|
log_points_3d ¶
log_points_3d(
scene_name: str,
points: Optional[List[Point3D]] = None,
boxes: Optional[List[Dict[str, Any]]] = None,
step: Optional[int] = None,
metadata: Optional[Dict] = None,
) -> None
Log 3d points and bounding boxes as an asset.
Parameters:
scene_name
(str
) –A string identifying the 3d scene to render. A same scene name could be logged across different steps.
points
(list
, default:None
) –A list of points, each point being a list (or equivalent like Numpy array). Each point length should be either 3, if only the position is given: [X, Y, Z]. The length could also be 6, if color is passed as well: [X, Y, Z, R, G, B]. Red, Green and Blue should be a number between 0 and 1. Either points or boxes are required.
boxes
(list
, default:None
) –List of box definition Dict.
step
(int
, default:None
) –Used to associate the asset to a specific step.
metadata
(dict
, default:None
) –Additional custom metadata can be associated with the logged asset.
Note
You can learn more about logging 3d point clouds here. You can also visualize these assets with the 3D Points panels.
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 |
|
log_remote_asset ¶
log_remote_asset(
uri,
remote_file_name=None,
overwrite=False,
asset_type="asset",
step=None,
metadata=None,
)
Logs a Remote Asset identified by an URI. A Remote Asset is an asset but its content is not uploaded and stored on Comet. Rather a link for its location is stored, so you can identify and distinguish between two experiment using different version of a dataset stored somewhere else.
Parameters:
uri
(str
) –The remote asset location, there is no imposed format, and it could be a private link.
remote_file_name
(str
, default:None
) –The "name" of the remote asset, could be a dataset name, a model file name.
overwrite
(bool
, default:False
) –If True will overwrite all existing assets with the same name.
asset_type
(str
, default:'asset'
) –Specifies the type of the asset being logged.
step
(int
, default:None
) –Used to associate the asset to a specific step.
metadata
(dict
, default:None
) –Some additional data to attach to the remote asset. Must be a JSON-encodable dict.
Example
1 2 3 4 5 6 7 8 9 |
|
log_remote_model ¶
log_remote_model(
model_name: str,
uri: str,
metadata: Any = None,
sync_mode: bool = True,
max_synced_objects: int = 10000,
) -> None
Logs metadata about a model that is stored elsewhere, such as remote storage solutions like AWS S3, Google Cloud Storage, Azure Blob Storage, etc. It allows you to keep track of model metadata without moving the actual model files into Comet.
Parameters:
model_name
(str
) –The name of the model.
uri
(str
) –The remote model location, there is no imposed format and it could be a private link. Can log a single file and a folder.
metadata
(dict
, default:None
) –some additional data to attach to the the data. Must be a JSON-encodable dict.
sync_mode
(bool
, default:True
) –If True and the URI begins with s3:// or gs://, Comet attempts to list all objects in the given bucket and path. All the objects will be logged under the given model name. If False, Comet just logs a single remote model with the provided URI as the remote URI.
max_synced_objects
(int
, default:10000
) –When sync_mode is True and the URI begins with s3:// or gs://, set the maximum number of S3/GCP objects to log. If there are more matching S3/GCP objects than max_synced_objects, a warning will be displayed and the provided URI will be logged as a single remote model.
Note
If the URI begins with s3:// or gs://, Comet attempts to list all objects in the given bucket and path and logs them individually.
Example
1 2 3 4 5 6 7 8 9 10 11 |
|
log_system_info ¶
log_system_info(key, value) -> None
Reports the key and value to the System metrics
tab in the single experiments view of the Comet UI.
Parameters:
key
–Any type of key.
value
–Any type of value.
Note
Useful to track general system information. This information can be added to the table on the Project view. You can retrieve this information via the Python API.
Example
1 2 3 4 5 6 7 |
|
log_table ¶
log_table(filename, tabular_data=None, headers=False, **format_kwargs)
Log tabular data, including data, csv files, tsv files, and Pandas dataframes.
Parameters:
filename
(str
) –A filename ending in ".csv", or ".tsv" (for tabular data) or ".json", ".csv", ".md", or ".html" (for Pandas dataframe data).
tabular_data
(Any
, default:None
) –Data that can be interpreted as 2D tabular data or a Pandas dataframe.
headers
(bool | list
, default:False
) –If True, will add column headers automatically if tabular_data is given; if False, no headers will be added; if list then it will be used as headers. Only useful with tabular data (csv, or tsv).
format_kwargs
(Any
, default:{}
) –When passed a Pandas dataframe these keyword arguments are used in the conversion to "json", "csv", "md", or "html". See Pandas Dataframe conversion methods (like
to_json()
) for more information.
See also:
- pandas.DataFrame.to_json documentation
- pandas.DataFrame.to_csv documentation
- pandas.DataFrame.to_html documentation
- pandas.DataFrame.to_markdown documentation
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
|
log_tensorflow_folder ¶
log_tensorflow_folder(folder) -> None
Logs all the tensorflow log files located in the given folder as assets.
Parameters:
folder
(str
) –The path to the folder you want to log.
Note
Use comet_ml.APIExperiment.download_tensorflow_folder to get the contents of a previously logged folder.
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 |
|
log_text ¶
log_text(text, step=None, metadata=None) -> None
Logs the text. These strings appear on the Text Tab in the Comet UI.
Parameters:
text
–String of text to be stored.
step
–Used to associate the asset to a specific step.
metadata
–Some additional data to attach to the text. Must be a JSON-encodable dict.
Example
1 2 3 4 5 6 7 8 9 10 11 |
|
log_video ¶
log_video(
file: Union[str, IO],
name: Optional[str] = None,
overwrite: bool = False,
format: Optional[str] = None,
step: Optional[int] = None,
epoch: Optional[int] = None,
metadata: Optional[Dict[str, Any]] = None,
) -> Optional[Dict[str, str]]
Logs a video to Comet. Videos are displayed on the assets tab in Comet and support the following formats: MP4, MOV, WMV, and GIF.
Parameters:
file
(str
) –The path to the video or a file-like object that contains the video.
name
(str
, default:None
) –A custom name can be provided to be displayed on the assets tab. If not provided, the filename from the file argument will be used if it is a path.
overwrite
(bool
, default:False
) –If another video with the same name exists, it will be overwritten if overwrite is set to True.
format
(str
, default:None
) –The extension of the file is used to check if the asset is of a supported format. If the extension of the file is not correct or if you are uploading a file-like object, you can indicate the format of the video through this parameter.
step
(int
, default:None
) –This is used to associate the video asset with a specific step.
epoch
(int
, default:None
) –Used to associate the asset to a specific epoch.
metadata
(dict
, default:None
) –dditional custom metadata can be associated with the logged video.
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
|
register_model ¶
register_model(
model_name: str,
version: Optional[str] = None,
workspace: Optional[str] = None,
registry_name: Optional[str] = None,
public: Optional[bool] = None,
description: Optional[str] = None,
comment: Optional[str] = None,
tags: Optional[list] = None,
stages: Optional[list] = None,
status: Optional[str] = None,
sync: Optional[bool] = False,
timeout: float = 10.0,
) -> None
Register an experiment's model to the registry.
Parameters:
model_name
(str
) –The name of the experiment model.
workspace
(str
, default:None
) –This argument is deprecated and ignored. Models are registered to the workspace the experiment belongs to.
version
(str
, default:None
) –A proper semantic version string; defaults to "1.0.0".
registry_name
(str
, default:None
) –The name of the registered workspace model, if not provided the model_name will be used instead.
public
(bool
, default:None
) –If True, then the model will be publicly viewable.
description
(str
, default:None
) –A textual description of the model.
comment
(str
, default:None
) –A textual comment about the model
tags
(list
, default:None
) –A list of textual tags such as ["tag1", "tag2"], etc.
stages
(list
, default:None
) –This argument is deprecated and will be ignored. Please use
tags
instead.status
(str
, default:None
) –A string describing the status of this model version.
sync
(bool
, default:False
) –Whether this function is synchronous and will be finished only once the model was registered.
timeout
(float
, default:10.0
) –Maximum time (In seconds) before the function would end if called with sync = True.
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
|
send_notification ¶
send_notification(
title: str,
status: Optional[str] = None,
additional_data: Optional[Dict[str, Any]] = None,
) -> None
With an Experiment
, this method will send you a notification through email when an experiment ends. However, with other experiment types, this method does nothing.
set_cmd_args ¶
set_cmd_args()
Logs command-line arguments used to run the script.Example
1
2
3
4
5
6
7
8
9
10
11
12
13
import comet_ml
# Initialize Comet experiment
comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")
exp.set_cmd_args()
# Continue on with your experiment
# ...
# End the experiment
exp.end()
set_code ¶
set_code(code=None, overwrite=False, filename=None) -> None
This method is now deprecated, use comet_ml.Experiment.log_code instead.
Sets the current experiment script's code. Should be called once per experiment.
Parameters:
code
(Any
, default:None
) –The experiment's source code.
overwrite
(bool
, default:False
) –If True, will overwrite previously set code.
filename
(str
, default:None
) –name of file to get source code from
set_epoch ¶
set_epoch(epoch) -> None
Sets the current epoch in the training process.
Parameters:
epoch
–Integer value
Note
You can also pass the epoch directly when reporting comet_ml.Experiment.log_metric.
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 |
|
set_filename ¶
set_filename(fname) -> None
Sets the current experiment filename.
Parameters:
fname
(str
) –The script's filename.
Example
1 2 3 4 5 6 7 8 9 10 |
|
set_model_graph ¶
set_model_graph(graph, overwrite=False) -> None
Sets the current experiment's computational graph.
Parameters:
graph
(Any
) –A model's computational graph.
overwrite
(bool
, default:False
) –If True, will overwrite a previously logged computational graph definition.
Note
The computational graph will be defined in the Graph definition
tab within the Comet UI of the logged experiment.
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
|
set_name ¶
set_name(name)
Set a name for the experiment. Useful for filtering and searching on Comet.ml. Will shown by default under the Other
tab.
Parameters:
name
(str
) –A name for the experiment.
set_os_packages ¶
set_os_packages() -> None
Reads the installed os packages and reports them to server as a message.Example
1
2
3
4
5
6
7
8
9
10
11
import comet_ml
# Initialize Comet experiment
comet_ml.init(project_name="os_package_logging")
exp = comet_ml.Experiment(project_name="comet-docs")
# Log the OS packages installed on the system
exp.set_os_packages()
# End the experiment
exp.end()
set_pip_packages ¶
set_pip_packages() -> None
Get the installed pip packages using pkg resources and reports them to server as a message.Example
1
2
3
4
5
6
7
8
9
10
11
import comet_ml
# Initialize Comet experiment
comet_ml.init(project_name="comet-docs")
exp = comet_ml.Experiment(project_name="comet-docs")
# Log the Python packages and their versions
exp.set_pip_packages()
# End the experiment
exp.end()
set_step ¶
set_step(step) -> None
Sets the current step in a training process. In Deep Learning, each step is after feeding a single batch into the network. This is used to generate correct plots on Comet.
Parameters:
step
–The current step number.
Note
You can also pass the step directly when reporting comet_ml.Experiment.log_metric, and comet_ml.Experiment.log_parameter.
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
|
stop_early ¶
stop_early(epoch)
Used to programmatically stop an experiment early.
Parameters:
epoch
–The epoch number at early stop.
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 |
|
test ¶
test()
A context manager to mark the beginning and the end of the testing phase. This allows you to provide a namespace for metrics/params.Example
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
import comet_ml
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
# Initialize Comet experiment
comet_ml.init(project_name="comet-docs")
exp = comet_ml.Experiment(project_name="comet-docs")
# Load a dataset and create a model
data = load_iris()
X = data.data
y = data.target
X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.2,
random_state=42)
model = LogisticRegression(max_iter=200)
model.fit(X_train, y_train)
# Start test context using a with statement
with exp.test():
# Perform predictions on the test set
pred = model.predict(X_test)
test_acc = accuracy_score(y_test, pred)
# Log test accuracy
exp.log_metric("test_accuracy", test_acc)
# End the experiment
exp.end()
train ¶
train()
A context manager to mark the beginning and the end of the training phase. This allows you to provide a namespace for metrics/params.Example
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
import comet_ml
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
# Initialize Comet experiment
comet_ml.init(project_name="comet-docs")
exp = comet_ml.Experiment(project_name="comet-docs")
# Load a dataset and create a model
data = load_iris()
X = data.data
y = data.target
X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.2,
random_state=42)
model = LogisticRegression(max_iter=200)
# Start train context using a with statement
with exp.train():
# Train model
model.fit(X_train, y_train)
# Perform predictions on the train set
pred = model.predict(X_train)
train_acc = accuracy_score(y_train, pred)
# Log train accuracy
exp.log_metric("train_accuracy", train_acc)
# End the experiment
exp.end()
validate ¶
validate()
A context manager to mark the beginning and the end of the validating phase. This allows you to provide a namespace for metrics/params.Example
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
import comet_ml
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
# Initialize Comet.ml experiment
comet_ml.init(project_name="comet-docs")
exp = comet_ml.Experiment(project_name="comet-docs")
# Load a dataset and create a model
data = load_iris()
X = data.data
y = data.target
X_train, X_validation, y_train, y_validation = train_test_split(X, y,
test_size=0.2,
random_state=42)
model = LogisticRegression(max_iter=200)
model.fit(X_train, y_train)
# Start validation context using a with statement
with exp.validate():
# Perform predictions on the validation set
pred = model.predict(X_validation)
val_acc = accuracy_score(y_validation, pred)
# Log validation accuracy
exp.log_metric("validation_accuracy", val_acc)
# End the experiment
exp.end()