Skip to content

Configuration

This page presents the full range of configuration options for controlling how your data is logged to Comet.

There are additional configuration parameters that are specific to Comet integrations with other tools and frameworks. For details, see the Comet documentation on the relevant integration.

Order of precedence for configuration parameters

Comet prioritizes setting configuration parameters in the following order:

  • Arguments passed to the Experiment object, for example, Experiment(api_key="Your API Key", disable_logging=True)
  • Environment parameters, for example, export COMET_API_KEY=<Your API Key>
  • A setting in the .comet.config file in your current folder
  • A setting in the .comet.config file in your HOME directory

In other words, an explicit parameter passed to the Experiment object has top priority, followed by parameters set through environment parameters, followed by a .comet.config file in your current working directory, followed by the .comet.config file in your HOME directory.

Configuration parameters

These parameters control behaviors of the SDK that are not related to logging Experiment data.

The table below outlines all the available configuration environment parameters and their corresponding names in the .comet.config file.

Comet Config Section/NameEnvironment Parameter NameDescription
[comet]
api_key
COMET_API_KEYThe API key used for creating Experiments and using the Python API
[comet]
workspace
COMET_WORKSPACEUse this workspace when creating new experiments
[comet]
project_name
COMET_PROJECT_NAMEUse this project when creating new experiments
[comet]
url_override
COMET_URL_OVERRIDEURL to upload the experiments to, this needs to be updated for on-premise deployments. Defaults to https://www.comet.com/clientlib/.
[comet]
offline_directory
COMET_OFFLINE_DIRECTORYSet the offline directory for OfflineExperiment(). Defaults to .cometml-runs.
[comet]
git_directory
COMET_GIT_DIRECTORYSet the directory that contains your git repo, if you are not under it
[comet]
display_summary_level
COMET_DISPLAY_SUMMARY_LEVELControls the level of detail displayed in the experiment summary. If set to 0, the summary is hidden, but the summary notification is still sent. If set to 2, the experiment summary shows everything that has been logged. Defaults to 1.
[comet]
rich_output
COMET_RICH_OUTPUTWhen enabled, Comet console log messages will be rendered using formatted methods from the Rich library. Defaults to True.
[comet]
system_cpu_interval
COMET_SYSTEM_CPU_INTERVALControls the interval at which system CPU metrics (CPU utilization and memory utilization) are logged. The value must be in seconds and cannot be less than 1s. Defaults to 60.
[comet]
system_gpu_interval
COMET_SYSTEM_GPU_INTERVALControls the interval at which system GPU metrics are logged. The value must be in seconds and cannot be less than 1s. Defaults to 60.
[comet]
system_network_interval
COMET_SYSTEM_NETWORK_INTERVALControls the interval at which system network metrics are logged. The value must be in seconds and cannot be less than 1s. Defaults to 60.
[comet]
system_disk_interval
COMET_SYSTEM_DISK_INTERVALControls the interval at which system disk metrics are logged. The value must be in seconds and cannot be less than 1s. Defaults to 60.
[comet]
http_session.retry_total
COMET_HTTP_SESSION_RETRY_TOTALNumber of times the SDK will try to upload a message before triggering the fallback to offline mechanism. Defaults to 3.
[comet]
http_session.retry_backoff_factor
COMET_HTTP_SESSION_RETRY_BACKOFF_FACTORControls how much time is applied between two different attempts to send data to Comet. Comet will wait for backoff_factor * (2 ** (attempt_number - 1)) seconds between attemps. Defaults to 2.
[comet_fallback_streamer]
keep_offline_zip
COMET_FALLBACK_STREAMER_KEEP_OFFLINE_ZIPIf True, save a local copy of experiment data for all training runs, not the just the ones that encountered connectivity issues. Defaults to False.
[comet_logging]
hide_api_key
COMET_LOGGING_HIDE_API_KEYIf True, redact the Comet API key in logs. Defaults to True.
[comet_logging]
file
COMET_LOGGING_FILEUse the given file for storing log messages. This can also contain patterns such as "comet-{project}.log" (See below)
[comet_logging]
file_level
COMET_LOGGING_FILE_LEVELBy default, the log outputs will contains INFO and higher level (WARNING and ERROR) log messages. This configuration item can be used to change the level of the logged messages. Defaults to INFO.
[comet_logging]
file_overwrite
COMET_LOGGING_FILE_OVERWRITEOverwrites the log file on each run, if True. Defaults to False.
[comet_logging]
console
COMET_LOGGING_CONSOLESet the logging level for console messages (for example, INFO, DEBUG).
[comet_logging]
metrics_ignore
COMET_LOGGING_METRICS_IGNOREList of framework:metric names to not log automatically. Defaults to keras:batch_size,keras:batch_batch
[comet_logging]
parameters_ignore
COMET_LOGGING_PARAMETERS_IGNOREList of framework:parameter names to not log automatically. Defaults to keras:verbose,keras:do_validation,keras:validation_steps
[comet_logging]
others_ignore
COMET_LOGGING_OTHERS_IGNOREList of framework:other names do not log automatically.
[comet_timeout]
cleaning
COMET_TIMEOUT_CLEANINGHow many times (in seconds) does the Python SDK wait until all experiment data has been uploaded. Default is 3600 seconds / 1 hour.

Experiment configuration parameters

In addition to the above, you can also control exactly what each Experiment logs with the following configuration parameters in the section [comet_auto_log].

Comet Config Section/NameEnvironment Parameter NameDescription
[comet_auto_log]
code
COMET_AUTO_LOG_CODEAuto Log source code for the current Experiment Run. Defaults to True.
[comet_auto_log]
git_metadata
COMET_AUTO_LOG_GIT_METADATAAuto Log git metadata for the current Experiment Run. Defaults to True.
[comet_auto_log]
git_patch
COMET_AUTO_LOG_GIT_PATCHAuto Log the git patch for current Experiment Run. Set to True to enable.
[comet_auto_log]
graph
COMET_AUTO_LOG_GRAPHAuto Log the model graph. This will depend on the framework being used. Set to True to enable.
[comet_auto_log]
parameters
COMET_AUTO_LOG_PARAMETERSAuto Log the model parameters. This will depend on the framework being used. Set to True to enable.
[comet_auto_log]
weights
COMET_AUTO_LOG_WEIGHTSAuto Log the model weights. This will depend on the framework being used. Set to True to enable.
[comet_auto_log]
metrics
COMET_AUTO_LOG_METRICSAuto Log the model metrics. This will depend on the framework being used. Set to True to enable.
[comet_auto_log]
cli_arguments
COMET_AUTO_LOG_CLI_ARGUMENTSAuto Log CLI arguments passed to the training script. Set to True to enable.
[comet_auto_log]
output_logger
COMET_AUTO_LOG_OUTPUT_LOGGERAuto Log console and Jupyter Notebook standard output and error. Set to True to enable
[comet_auto_log]
co2
COMET_AUTO_LOG_CO2Auto Log CO2 emissions from the run using the code-carbon package. Set to True to enable
[comet_auto_log]
env_details
COMET_AUTO_LOG_ENV_DETAILSAuto Log details about the compute environment. Set to True to enable, set to False to quickly disable all environment details logging.
[comet_auto_log]
env_gpu
COMET_AUTO_LOG_ENV_GPUAuto Log the collection of GPU details and system metrics (utilization, memory usage etc..). Logging Environment Details must also be set to True
[comet_auto_log]
env_host
COMET_AUTO_LOG_ENV_HOSTAuto Log the host information (ip, host name, python version, user etc...). Logging Environment Details must also be set to True
[comet_auto_log]
env_cpu
COMET_AUTO_LOG_ENV_CPUAuto Log the collection of CPU details and system metrics (utilization, memory usage etc..). Logging Environment Details must also be set to True
[comet_auto_log]
env_cpu_per_core
COMET_AUTO_LOG_ENV_CPU_PER_COREAuto Log the collection of CPU details and system metrics (utilization etc..) for each CPU core. Logging Environment Details must also be set to True. Defaults to False
[comet_auto_log]
env_network
COMET_AUTO_LOG_ENV_NETWORKAuto Log the collection of network system metrics. Logging Environment Details must also be set to True. Defaults to True
[comet_auto_log]
env_disk
COMET_AUTO_LOG_ENV_DISKAuto Log the collection of disk system metrics. Logging Environment Details must also be set to True. Defaults to True
[comet_auto_log]
disable
COMET_AUTO_LOG_DISABLEDisable all network communication with the Comet backend. It is useful when you want to test to make sure everything is working, without actually logging anything. Defaults to False.

Environment-only parameters

In addition to the configuration parameters, there are a few special-purpose Comet environment-only parameters. These only operate in the environment, and cannot be set in the configuration:

Environment ParameterDescription
COMET_INIA path to another .comet.config file to take into account
COMET_OPTIMIZER_IDCurrent optimizer ID; for use with the comet optimize CLI command
COMET_OPTIMIZER_PROCESS_IDCurrent job number; set when using comet optimize CLI command
COMET_OPTIMIZER_PROCESS_JOBSTotal number of parallel jobs; set when using comet optimize CLI command
COMET_DISABLE_AUTO_LOGGINGDisable all automatic logging (see next section)

Configure Comet file logging

Comet lets you log the console output and errors to a log file. You can use a number of patterns to name your Comet log file. These are keywords that are surrounded by curly braces.

To set this pattern through an enviroment parameter:

export COMET_LOGGING_FILE={user}.log

To set this pattern through the .comet.config file:

[comet_logging]
file={user}.log

The following table shows the full list of available patterns:

COMET_LOGGING_FILE PatternExampleExpanded FormMeaning
{user}comet-{user}.logcomet-jones.logUses the OS username
{project}comet-{project}.logcomet-general.logUses the COMET_PROJECT_NAME if set, else "general"
{pid}{project}-{pid}.loggeneral-2635.logUses the OS Process ID
{datetime}comet-{datetime}.logcomet-20190814-151135.logUses a Year, Month, Day, Hour, Minute, Seconds time format

Other configuration logging

Configuration itemComet Config Section/NameEnvironment variableDescription
System metrics prefix[comet]
distributed_node_identifier
COMET_DISTRIBUTED_NODE_IDENTIFIERA prefix to be used on all logged system metrics.
Feb. 24, 2024