If you are in a Jupyter Notebook, you can install comet_ml directly in the notebook by using this magic:

%pip install comet_ml --upgrade --quiet

If you have a complex Python environment, you can use this format to install comet_ml at the command line:

bash /path/to/python -m pip install comet_ml

Other useful pip variations at the command line:

  1. python3 -m pip install comet_ml --user : using Python 3, installs comet_ml to ~/.local
  2. pip install comet_ml --upgrade : using the default Python, upgrades comet_ml to latest version
  3. pip3 install comet_ml --upgrade --upgrade-strategy eager : using Python 3, upgrades comet_ml and all of its dependencies

The flags --user, --upgrade, and --upgrade-strategy can be used in many combinations.


If you're installing the SDK on shared servers or as part of a CI/CD system it's encouraged to use the --upgrade flag for pip as our SDK is constantly evolving.

If you are using conda or would like to log CPU usage, please see our additional Installation Instructions.

Python Configuration

In order to interact with you will need to let it know what your Comet API key is. Recall that you can get (and reset) your Comet API key via this URL:

If using an onprem version of Comet, please adjust the URL for your installation location.

Interactive Setup

If you are in an interactive environment (such as a Jupyter Notebook) connected to the internet, the easiest way to get started is the following lines in a cell near the top of the notebook:

python import comet_ml comet_ml.init()

The first time you use this it will ask you for a valid Comet API key, and save it in your ~/.comet.config file (see below). Subsequent uses will not ask for your Comet API key.

In addtion, you can temporarily override Comet configuration options, like this:

python import comet_ml comet_ml.init(project_name="nlp-1267")

In addition setting and saving the API key, this variation will temporarily set the COMET_PROJECT_NAME to "nlp-1267" so that all of the Experiments created in this session will be logged in that project. See Experiment Configuration Parameters below for additional configuration options you could set in this manner.

You can also run comet init --api-key at the command-line to set and save the Comet API key. See Comet Command-Line Utilities for additional information.

Non-interactive Setup

There are a number of methods for you to set your API key prior to running a script. They are in order of highest precedence:

  1. An argument passed to Experiment("YOUR-API-KEY")
  2. An environment variable named COMET_API_KEY
  3. A setting in the .comet.config file in your current folder
  4. A setting in the .comet.config file in your HOME directory

That is, an explicit API key passed to Experiment() has top priority, and would override any other method. To get started immediately, you can use the following:

python import comet_ml experiment = comet_ml.Experiment("YOUR-API-KEY") Alternatively, an environment variable named COMET_API_KEY has the second-highest priority, and would override any configuration file setting. In a Unix-like environment, you could set it like this:


However, in general, we recommend using a file named .comet.config (note the initial dot in the filename) in your home directory. That way, it will always be found, and you'll never have to set it again. The file format of the .comet.config file is as follows:

ini [comet] api_key=YOUR-API-KEY

After you create the file, you can then create an Experiment() without having to put your private Comet API key in your source code.

You can get (or reset) your API key under your personal settings at

NOTE: the COMET_REST_API_KEY has been deprecated. You can now simply use your COMET_API_KEY. For more information, please see REST API, Getting Started.

Debugging with Comet

When you run a Python script creating Experiments with comet_ml, your script will run as it normally would. That is, even if you, say, attempt to log something invalid, your script will run without interruption to completion. Let's take a specific look at this case, and explore how to debug it.

Consider a script that attempts to log an invalid image:



from comet_ml import Experiment

e = Experiment()

image = [] e.log_image(image)

further processing...

print("Done!") ```

This script will run without stopping or crashing even though the image is not a proper image. Here is the output of running the script:

$ python ... COMET ERROR: Could not convert image_data into an image; ignored Done! ...

For many cases, the ERROR message might be enough of a hint to know what to fix. But what if it isn't obvious why the image didn't log? You have two options:

  1. Create a Comet Debug Log file. This is especially useful to send to Comet for detailed analysis. See Troubleshooting below on how to create a detailed log file.
  2. Turn on tracebacks while you run the script.

The second option will run your script exactly as before, but will also show any tracebacks from comet_ml operations that failed. It won't stop processing, but it will show the traceback of any errors. Because it could be confusing to see a traceback without crashing, we don't show them by default. But it is easy to enable this setting:

You can set the COMET_LOGGING_CONSOLE on the command line, like:



export COMET_LOGGING_CONSOLE=info python

When you run the script this way, you will see output similar to:

$ COMET_LOGGING_CONSOLE=info python ... COMET ERROR: Could not convert image_data into an image; ignored Traceback (most recent call last): File "/home/dblank/comet/python/comet-client-lib/comet_ml/", line 659, in process_upload_to_be_converted self.image_channels, File "/home/dblank/comet/python/comet-client-lib/comet_ml/", line 284, in image_data_to_file_like_object image_channels, File "/home/dblank/comet/python/comet-client-lib/comet_ml/", line 322, in array_to_image_fp minmax = [array.min(), array.max()] File "/home/dblank/.local/lib/python3.6/site-packages/numpy/core/", line 34, in _amin return umr_minimum(a, axis, None, out, keepdims, initial, where) ValueError: zero-size array to reduction operation minimum which has no identity Done! ...

As you can see, it shows the stacktrace that caused the problem, but also still continues with processing.

You can also set the COMET_LOGGING_CONSOLE variable in your Comet Configuration file as described in the following section.

Comet Configuration Variables

Here is a list of the main configuration variables (followed by details on specific sections):

Configuration item Comet Config Section/Name Environment variable Description
API key [comet]
COMET_API_KEY The API key used for creating Experiments and using the Python API
Current Workspace [comet]
COMET_WORKSPACE Use this workspace when creating new experiments
Current Project [comet]
COMET_PROJECT_NAME Use this project when creating new experiments
Offline Directory [comet]
COMET_OFFLINE_DIRECTORY Set the offline directory for OfflineExperiment()
Git Directory [comet]
COMET_GIT_DIRECTORY Set the directory that contains your git repo, if you are not under it
Display Summary [comet]
COMET_DISPLAY_SUMMARY If False, do not display upload summary for experiments
Hide API key in logs [comet_logging]
COMET_LOGGING_HIDE_API_KEY If True, redact the Comet API key in logs
Logging file [comet_logging]
COMET_LOGGING_FILE Use the given file for storing log messages. This can also contain patterns such as "comet-{project}.log" (See below)
Logging file level [comet_logging]
COMET_LOGGING_FILE_LEVEL By default, the log outputs will contains INFO and higher level (WARNING and ERROR) log messages. This configuration item can be used to change the level of the logged messages.
Logging file overwrite [comet_logging]
COMET_LOGGING_FILE_OVERWRITE Overwrites the log file on each run, if True
Console logging [comet_logging]
COMET_LOGGING_CONSOLE Set the logging level for console messages (e.g., INFO, DEBUG)
Logging auto values [comet_logging]
COMET_LOGGING_METRICS_IGNORE List of framework:metric names to not auto-log
Logging auto values [comet_logging]
COMET_LOGGING_PARAMETERS_IGNORE List of framework:parameter names to not auto-log
Logging auto values [comet_logging]
COMET_LOGGING_OTHERS_IGNORE List of framework:other names to not auto-log
Experiment end timeout [comet_timeout]
COMET_TIMEOUT_CLEANING How many times (in seconds) does the Python SDK wait until all experiment data has been uploaded. Default is 3600 seconds (i.e. 1 hour).

Experiment Configuration Parameters

In addition to the above, you can also control exactly what each experiment logs with the following configuration variables in the section [comet_auto_log]:

Experiment Loggable Item Comet Config [comet_auto_log] Section Name Experiment parameter Environment variable
code, Jupyter Notebook code log_code COMET_AUTO_LOG_CODE
git metadata (log_code must be True) git_metadata log_git_metadata COMET_AUTO_LOG_GIT_METADATA
git patch (log_code must be True) git_patch log_git_patch COMET_AUTO_LOG_GIT_PATCH
model graph representation graph log_graph COMET_AUTO_LOG_GRAPH
model parameters parameters auto_param_logging COMET_AUTO_LOG_PARAMETERS
model weights and biases weights auto_weight_logging COMET_AUTO_LOG_WEIGHTS
training metrics metrics auto_metric_logging COMET_AUTO_LOG_METRICS
command-line arguments cli_arguments parse_args COMET_AUTO_LOG_CLI_ARGUMENTS
console and Jupyter Notebook standard output and error output_logger auto_output_logging COMET_AUTO_LOG_OUTPUT_LOGGER
CO2 emissions co2 auto_log_co2 COMET_AUTO_LOG_CO2
environment details env_details log_env_details COMET_AUTO_LOG_ENV_DETAILS
environment GPU env_gpu log_env_gpu COMET_AUTO_LOG_ENV_GPU
environment hostname env_host log_env_host COMET_AUTO_LOG_ENV_HOST
environment CPU env_cpu log_env_cpu COMET_AUTO_LOG_ENV_CPU
everything; if True, everything above is disabled disable disable COMET_AUTO_LOG_DISABLE

Remember: any configuration setting defined as an environment variable will override the value set in the .comet.config file.

The auto_log_co2=True setting automatically tracks the CO2 emission of the experiment if the codecarbon package is installed in the environment. For more details please see Introducing CodeCarbon.

Comet File Logging

As mentioned, you can use a number of patterns in the COMET_LOGGING_FILE name ([comet_logging] file value). These are keywords surrounded by curly braces as follows:

COMET_LOGGING_FILE Pattern Example Expanded Form Meaning
{user} comet-{user}.log comet-jones.log Uses the OS username
{project} comet-{project}.log comet-general.log Uses the COMET_PROJECT_NAME if set, else "general"
{pid} {project}-{pid}.log general-2635.log Uses the OS Process ID
{datetime} comet-{datetime}.log comet-20190814-151135.log Uses a Year, Month, Day, Hour, Minute, Seconds time format

Other Configuration Logging

Configuration item Comet Config Section/Name Environment variable Description
CPU/GPU system prefix [comet]
COMET_DISTRIBUTED_NODE_IDENTIFIER A prefix to be used on all GPU/CPU logged system metrics

There is also an additional flag that can only be set in the environment: COMET_DISABLE_AUTO_LOGGING (see below for addition environment-only settings). Use this to disable Comet’s automatic framework tracking functionality. For a complete list of items logged per Python framework, see Automatic Logging. will not interfere with your run, unless you attempt to use one of the auto logging Python libraries, but you have imported comet_ml after the library. For example:

```python import tensorflow import comet_ml

experiment = comet_ml.Experiment() ```

When you create an Experiment then you will get an ImportError. To fix, simply move the import comet_ml above the other import statement.

In some rare cases, something will fail (such as the connection between your computer and In that case, your code will keep on running even though the run will not be logged with If you wish for your code to stop if it is not being logged, we suggest the following pattern:

```python from comet_ml import Experiment

exp = Experiment() if exp.alive is False: raise Exception("Something went wrong") ```

That is, after an Experiment has been created, you may check the alive property. If alive is False then you know something has failed, and you could stop your run. Of course, there is also the OfflineExperiment to use on computers with flaky (or non-existent) internet connections.

Comet Environment-Only Variables

In addition to the configuration variables, there are a few special-purpose Comet environment-only variables. These only operate in the environment, and cannot be set in the configuration:

Environment Variable Description
COMET_INI The path to your .comet.config file
COMET_OPTIMIZER_ID Current optimizer ID; for use with the comet optimize CLI command
COMET_OPTIMIZER_PROCESS_ID Current job number; set when using comet optimize CLI command
COMET_OPTIMIZER_PROCESS_JOBS Total number of parallel jobs; set when using comet optimize CLI command
COMET_DISABLE_AUTO_LOGGING Disable all auto-logging (see next section)

Comet Auto Logging

Comet can automatically log certain values when using one of the following machine learning frameworks:

Framework What is automatically logged
tensorflow steps and metrics; see examples
tensorflow model analysis time series, plots, and slicing metrics
tensorboard summary scalars (as metrics) and summary histograms
sklearn hyperparameters; see examples
keras graph description, steps, metrics, hyperparameters, weights and biases, activations, and gradients as histograms, optimizer config, and number of trainable parameters; see examples
mlflow hyperparameters, assets, models, plus lower-level framework items (e.g., tensorflow's metrics, tensorboard summaries)
fbprophet hyperparameters, model, and figures
pytorch graph description, steps, and loss; see examples
pytorch-lightning loss and accuracy, with refinements in progress; see examples
pytorch + apex loss; see examples
fastai all pytorch items, plus epochs, and metrics; see examples
xgboost metrics, hyperparameters; see examples

See the framework page for more details on its specifics.


Also check out the comet init command-line method of creating best-practices code with Comet.

However, there may be items that are logged that you wish to ignore, or items ignored that you wish to log. You can control exactly what is auto-logged using the appropriate logging config variables. Each of the ignore variables should be set to a list of framework:name items, separated by commas. Here is an example showing the default values:

ini [comet_logging] metrics_ignore = keras:batch_size,keras:batch_batch others_ignore = parameters_ignore = keras:verbose,keras:do_validation,keras:validation_steps

The metrics_ignore line indicates that the metric keras:batch_size will not be auto-logged to Comet. The reason that it is not logged as a metric is because it does not typically change over time, and is usually regarded as a parameter. However, if it did change over time, you can remove it from this list and it then would be logged to Comet.

When you run a script using Comet, it will print out any auto-logged framework:name that is ignored. You can then adjust these configuration variables so that you have complete control over what is logged.

HTTP Proxy

You can instruct comet_ml to use an HTTP proxy by setting one of the common HTTPS variables in an appropriate manner for your operating system. Comet will look for environment variables named HTTPS_PROXY or https_proxy, for all operating systems first, and if it cannot find them, it looks for proxy information from Mac OSX System Configuration for Mac OS X and Windows Systems Registry for Windows. If both lowercase and uppercase environment variables exist (and disagree), the lowercase is preferred.

You can use any standard format for setting the proxy:

  • ""
  • ""
  • ""
  • "socks4://"
  • "socks5://"

For example, in bash you can:

bash export HTTPS_PROXY="" python


Note that you must use the HTTPS version (not HTTP). Also, to use the socks4 or socks5 protocol, you must have the "PySocks" Python library installed.

You may also define the NO_PROXY environment variable (all capital letters) to define a list of domains that should not be consulted. For example:

bash export NO_PROXY=",localhost," python

For more information on using proxies, please see Setting Up Clients To Use a Proxy.

Context Managers

In addition to the default method of logging shown in Getting Started, you can also use the Python contextual managers with experiment.train(), with experiment.validate(), or with experiment.test() which will allow metrics and hyperparameters will be prefixed accordingly. For example, "accuracy" will be reported as "train_accuracy" in the with experiment.train() context manager. Here is an example using all three context managers:

```python import comet_ml experiment = comet_ml.Experiment()

model = build_model()

with experiment.train(): train_model(model)

with experiment.test(): test_model(model)

with experiment.validate(): validate_model(model) ```

The Experiment object logs various parameters of your experiment to For example:

```python from comet_ml import Experiment experiment = Experiment() batch_size = 4 # A hyperparameter used somewhere in the code.

experiment.log_parameter("batch_size", batch_size) ```

By default your experiment will be added to the project for Uncategorized Experiments named general. You can also log your experiment to a specific project using the project_name argument of the Experiment class:

```python from comet_ml import Experiment

if "my project name" does not already exist, it will be created.

experiment = Experiment(project_name="my project name") batch_size = 4

experiment.log_parameter("batch_size", batch_size) ``` You can also log a custom list of hyperparameters to your experiment via a dictionary.

```python from comet_ml import Experiment experiment = Experiment(project_name="my project name", auto_param_logging=False) batch_size = 128 num_classes = 10 epochs = 20

params = { "batch_size":batch_size, "epochs":epochs, "num_classes":num_classes }

experiment.log_parameters(params) ```

Reporting from SDK to a workspace project

When initializing a team experiment provide the workspace argument. This will assign the experiment to the correct workspace.

```python from comet_ml import Experiment experiment = Experiment(workspace="my-workspace-name", project_name="a project")

Your code.



Every workspace member can use their own API key. If workspace is set it will be reported to that workspace. Remember, you can set your API Key in your config file. See Quick Start for more details.


Recall from above that you can set the configuration variable COMET_LOGGING_CONSOLE to "info" to see tracebacks for any Comet-based issues. This is often enough information to help track down a problem (for example, the reason why an image is not logged). However, if you need the maximum amount of debug information, you'll want to create a "Comet debug log file."

The first step in creating a "Comet debug log file" is to set two configuration variables: COMET_LOGGING_FILE and COMET_LOGGING_FILE_LEVEL as described above. For example, you can define the variables in the environment, or in your .comet.config file.

Here is how you can set them in the bash environment:

bash $ export COMET_LOGGING_FILE=/tmp/comet.log $ export COMET_LOGGING_FILE_LEVEL=debug

Here is the contents of a sample .comet.config file:

[comet_logging] file = /tmp/comet.log file_level = debug

Or, you can also define them at the same time as you run your script:

bash $ COMET_LOGGING_FILE_LEVEL=debug \ COMET_LOGGING_FILE=/tmp/comet.log \ python

Finally, you can also put them right into the script itself before you import comet_ml:

```python import os os.environ["COMET_LOGGING_FILE"] = "/tmp/comet.log" os.environ["COMET_LOGGING_FILE_LEVEL"] = "debug"

import comet_ml ... ```

In these examples, the debugging logs will be in /tmp/comet.log, but you can put it where you like, and name it whatever you prefer. This log will show details on all of the steps of your experiment, and any details about failures. If you still have problems, please share this file with us via the Slack channel.

Another item to check is to make sure that your comet_ml version is up to date. You can find the latest version number on the Python Packaging comet_ml page. To upgrade, you can use the command:

bash $ pip install comet_ml --upgrade

In some cases, you may want to also update all of the packages that comet_ml depends on. You can do that with:

bash $ pip install comet_ml --upgrade --upgrade-strategy eager

System Check

If you are unable to successfully log data, or are on a new onprem installation and want to ensure that everything is set up correctly, you can run the Comet command-line script:

bash comet check

The comet check command will make sure that your system is properly configured for talking to the servers. More more information, see Comet Command-Line Utilities.

Running with Distributed and Parallel Processes

The Comet Python SDK can be used when the training process is distributed across several processes and potentially several GPUs and/or servers. For details, caveats, and examples, please see Using the Python SDK in distributed mode.