Skip to content

Configuration

This page presents the full range of configuration options for controlling how your data is logged to Comet.

There are additional configuration parameters that are specific to Comet integrations with other tools and frameworks. For details, see the Comet documentation on the relevant integration.

Order of precedence for configuration parameters

Comet prioritizes setting configuration parameters in the following order:

  • Arguments passed to the Experiment object, for example, Experiment(api_key="Your API Key", disable_logging=True)
  • Environment parameters, for example, export COMET_API_KEY=<Your API Key>
  • A setting in the .comet.config file in your current folder
  • A setting in the .comet.config file in your HOME directory

In other words, an explicit parameter passed to the Experiment object has top priority, followed by parameters set through environment parameters, followed by a .comet.config file in your current working directory, followed by the .comet.config file in your HOME directory.

Configuration parameters

These parameters control behaviors of the SDK that are not related to logging Experiment data.

The table below outlines all the available configuration environment parameters and their corresponding names in the .comet.config file.

Comet Config Section/NameEnvironment Parameter NameDescription
[comet]
api_key
COMET_API_KEYThe API key used for creating Experiments and using the Python API
[comet]
workspace
COMET_WORKSPACEUse this workspace when creating new experiments
[comet]
project_name
COMET_PROJECT_NAMEUse this project when creating new experiments
[comet]
offline_directory
COMET_OFFLINE_DIRECTORYSet the offline directory for OfflineExperiment()
[comet]
git_directory
COMET_GIT_DIRECTORYSet the directory that contains your git repo, if you are not under it
[comet]
display_summary
COMET_DISPLAY_SUMMARYIf False, do not display upload summary for experiments
[comet_logging]
hide_api_key
COMET_LOGGING_HIDE_API_KEYIf True, redact the Comet API key in logs
[comet_logging]
file
COMET_LOGGING_FILEUse the given file for storing log messages. This can also contain patterns such as "comet-{project}.log" (See below)
[comet_logging]
file_level
COMET_LOGGING_FILE_LEVELBy default, the log outputs will contains INFO and higher level (WARNING and ERROR) log messages. This configuration item can be used to change the level of the logged messages.
[comet_logging]
file_overwrite
COMET_LOGGING_FILE_OVERWRITEOverwrites the log file on each run, if True
[comet_logging]
console
COMET_LOGGING_CONSOLESet the logging level for console messages (for example, INFO, DEBUG)
[comet_logging]
metrics_ignore
COMET_LOGGING_METRICS_IGNOREList of framework:metric names to not log automatically
[comet_logging]
parameters_ignore
COMET_LOGGING_PARAMETERS_IGNOREList of framework:parameter names to not log automatically
[comet_logging]
others_ignore
COMET_LOGGING_OTHERS_IGNOREList of framework:other names do not log automatically
[comet_timeout]
cleaning
COMET_TIMEOUT_CLEANINGHow many times (in seconds) does the Python SDK wait until all experiment data has been uploaded. Default is 3600 seconds / 1 hour.

Experiment configuration parameters

In addition to the above, you can also control exactly what each Experiment logs with the following configuration parameters in the section [comet_auto_log].

Comet Config Section/NameEnvironment Parameter NameDescription
[comet_auto_log]
code
COMET_AUTO_LOG_CODEAuto Log source code for the current Experiment Run. Defaults to True.
[comet_auto_log]
git_metadata
COMET_AUTO_LOG_GIT_METADATAAuto Log git metadata for the current Experiment Run. Defaults to True.
[comet_auto_log]
git_patch
COMET_AUTO_LOG_GIT_PATCHAuto Log the git patch for current Experiment Run. Set to True to enable.
[comet_auto_log]
graph
COMET_AUTO_LOG_GRAPHAuto Log the model graph. This will depend on the framework being used. Set to True to enable.
[comet_auto_log]
parameters
COMET_AUTO_LOG_PARAMETERSAuto Log the model parameters. This will depend on the framework being used. Set to True to enable.
[comet_auto_log]
weights
COMET_AUTO_LOG_WEIGHTSAuto Log the model weights. This will depend on the framework being used. Set to True to enable.
[comet_auto_log]
metrics
COMET_AUTO_LOG_METRICSAuto Log the model metrics. This will depend on the framework being used. Set to True to enable.
[comet_auto_log]
cli_arguments
COMET_AUTO_LOG_CLI_ARGUMENTSAuto Log CLI arguments passed to the training script. Set to True to enable.
[comet_auto_log]
output_logger
COMET_AUTO_LOG_OUTPUT_LOGGERAuto Log console and Jupyter Notebook standard output and error. Set to True to enable
[comet_auto_log]
co2
COMET_AUTO_LOG_CO2Auto Log CO2 emissions from the run using the code-carbon package. Set to True to enable
[comet_auto_log]
env_details
COMET_AUTO_LOG_ENV_DETAILSAuto Log details about the compute environment. Set to True to enable, set to False to quickly disable all environment details logging.
[comet_auto_log]
env_gpu
COMET_AUTO_LOG_ENV_GPUAuto Log the collection of GPU details and metrics (utilization, memory usage etc..). Logging Environment Details must also be set to True
[comet_auto_log]
env_host
COMET_AUTO_LOG_ENV_HOSTAuto Log the host information (ip, host name, python version, user etc...). Logging Environment Details must also be set to True
[comet_auto_log]
env_cpu
COMET_AUTO_LOG_ENV_CPUAuto Log the collection of CPU details and metrics (utilization, memory usage etc..). Logging Environment Details must also be set to True
[comet_auto_log]
disable
COMET_AUTO_LOG_DISABLEDisable all network communication with the Comet backend. It is useful when you want to test to make sure everything is working, without actually logging anything. Defaults to False.

Environment-only parameters

In addition to the configuration parameters, there are a few special-purpose Comet environment-only parameters. These only operate in the environment, and cannot be set in the configuration:

Environment ParameterDescription
COMET_INIThe path to your .comet.config file
COMET_OPTIMIZER_IDCurrent optimizer ID; for use with the comet optimize CLI command
COMET_OPTIMIZER_PROCESS_IDCurrent job number; set when using comet optimize CLI command
COMET_OPTIMIZER_PROCESS_JOBSTotal number of parallel jobs; set when using comet optimize CLI command
COMET_DISABLE_AUTO_LOGGINGDisable all automatic logging (see next section)

Configure Comet file logging

Comet lets you log the console output and errors to a log file. You can use a number of patterns to name your Comet log file. These are keywords that are surrounded by curly braces.

To set this pattern through an enviroment parameter:

export COMET_LOGGING_FILE={user}.log

To set this pattern through the .comet.config file:

[comet_logging]
file={user}.log

The following table shows the full list of available patterns:

COMET_LOGGING_FILE PatternExampleExpanded FormMeaning
{user}comet-{user}.logcomet-jones.logUses the OS username
{project}comet-{project}.logcomet-general.logUses the COMET_PROJECT_NAME if set, else "general"
{pid}{project}-{pid}.loggeneral-2635.logUses the OS Process ID
{datetime}comet-{datetime}.logcomet-20190814-151135.logUses a Year, Month, Day, Hour, Minute, Seconds time format

Other configuration logging

Configuration itemComet Config Section/NameEnvironment variableDescription
CPU/GPU system prefix[comet]
distributed_node_identifier
COMET_DISTRIBUTED_NODE_IDENTIFIERA prefix to be used on all GPU/CPU logged system metrics.
Jul. 20, 2022