Experiment

Description¶

A comet experiment object can be used to modify or get information about an active experiment. All methods documented here are the different ways to interact with an experiment. Use create_experiment() to create or get_experiment() to retrieve a Comet experiment object.

Examples¶

library(cometr)
# Assuming you have COMET_API_KEY, COMET_WORKSPACE, COMET_PROJECT_NAME variables define
exp <- create_experiment()
exp$get_key()
exp$get_metadata()
exp$add_tags(c("test", "tag2"))
exp$get_tags()
exp$log_metric("metric1", 5)
exp$get_metric("metric1")
exp$get_metrics_summary()
exp$stop()



## ------------------------------------------------
## Method `Experiment$get_artifact`
## ------------------------------------------------


library(cometr)
# Assuming you have COMET_API_KEY, COMET_WORKSPACE, COMET_PROJECT_NAME variables define
exp <- create_experiment()

# Get a Comet Artifact
logged_artifact <- exp$get_artifact("workspace/artifact-name:version_or_alias")

# Which is equivalent to
logged_artifact = exp$get_artifact(artifact_name="artifact-name",
                                   workspace="workspace",
                                   version_or_alias="version_or_alias")

Methods¶

Public Methods¶

• Experiment$new()
• Experiment$get_key()
• Experiment$get_workspace_name()
• Experiment$get_project_name()
• Experiment$get_dynamic()
• Experiment$get_url()
• Experiment$get_metadata()
• Experiment$archive()
• Experiment$restore()
• Experiment$delete()
• Experiment$stop()
• Experiment$log_metric()
• Experiment$get_metric()
• Experiment$get_metrics_summary()
• Experiment$log_graph()
• Experiment$get_graph()
• Experiment$log_parameter()
• Experiment$get_parameters()
• Experiment$log_other()
• Experiment$get_other()
• Experiment$add_tags()
• Experiment$get_tags()
• Experiment$log_html()
• Experiment$get_html()
• Experiment$upload_asset()
• Experiment$log_remote_asset()
• Experiment$get_asset_list()
• Experiment$get_asset()
• Experiment$create_symlink()
• Experiment$log_git_metadata()
• Experiment$get_git_metadata()
• Experiment$get_git_patch()
• Experiment$get_output()
• Experiment$log_code()
• Experiment$get_code()
• Experiment$log_system_details()
• Experiment$get_system_details()
• Experiment$log_artifact()
• Experiment$get_artifact()
• Experiment$set_start_end_time()
• Experiment$print()

Method new()¶

Do not call this function directly. Use create_experiment() or get_experiment() instead.

Usage

Experiment$new(
  experiment_key,
  experiment_url = NULL,
  api_key = NULL,
  keep_active = FALSE,
  log_output = FALSE,
  log_error = FALSE,
  dynamic = TRUE,
  workspace_name = NULL,
  project_name = NULL
)

Arguments:

• experiment_key The key of the Experiment.
• experiment_url The URL of the Experiment.
• api_key Comet API key (can also be specified using the COMET_API_KEY parameter as an environment variable or in a comet config file).
• keep_active If TRUE, automatically send Comet a status update every few seconds until the experiment is stopped to mark the experiment as active on the Comet web dashboard.
• log_output If TRUE, all standard output will automatically be sent to the Comet servers to display as message logs for the experiment. The output will still be shown in the console as well.
• log_error If TRUE, all output from 'stderr' (which includes errors, warnings, and messages) will be redirected to the Comet servers to display as message logs for the experiment. Note that unlike auto_log_output, if this option is on then these messages will not be shown in the console and instead they will only be logged to the Comet experiment. This option is set to FALSE by default because of this behavior.
• dynamic If TRUE the Experiment was created rather than retrieved.
• workspace_name The workspace name (can also be specified using the COMET_WORKSPACE parameter as an environment variable or in a comet config file).
• project_name The project name (can also be specified using the COMET_PROJECT_NAME parameter as an environment variable or in a comet config file).

Method get_key()¶

Get the experiment key of an experiment.

Usage

Experiment$get_key()

Method get_workspace_name()¶

Get the workspace name of an experiment.

Usage

Experiment$get_workspace_name()

Method get_project_name()¶

Get the project name of an experiment.

Usage

Experiment$get_project_name()

Method get_dynamic()¶

Get the dynamic status of an experiment.

Usage

Experiment$get_dynamic()

Method get_url()¶

Get the URL to view an experiment in the browser.

Usage

Experiment$get_url()

Method get_metadata()¶

Get an experiment's metadata.

Usage

Experiment$get_metadata()

Method archive()¶

Archive an experiment.

Usage

Experiment$archive()

Method restore()¶

Restore an archived experiment.

Usage

Experiment$restore()

Method delete()¶

Delete an experiment.

Usage

Experiment$delete()

Method stop()¶

Stop an experiment. Always call this method before creating a new experiment.

Usage

Experiment$stop()

Method log_metric()¶

Log a metric name and value. Metrics are the only items that are logged as a full time series. However, even metrics can be throttled if too much data (either by rate or by count) is attempted to be logged.

Usage

Experiment$log_metric(name, value, step = NULL, epoch = NULL, context = NULL)

Arguments:

• name (Required) Name of the metric.
• value (Required) Value of the metric.
• step Step number.
• epoch Epoch.
• context Context.

Method get_metric()¶

Get All Metrics For Name

Usage

Experiment$get_metric(name)

Arguments:

• name (Required) Name of metric.

Method get_metrics_summary()¶

Get an experiment's metrics summary.

Usage

Experiment$get_metrics_summary()

Method log_graph()¶

Log an experiment's associated model graph.

Usage

Experiment$log_graph(graph)

Arguments:

• graph (Required) JSON representation of a graph.

Method get_graph()¶

Get an experiment's model graph.

Usage

Experiment$get_graph()

Method log_parameter()¶

Log a parameter name and value. Note that you can only retrieve parameters summary data (e.g., this is not recorded as a full time series).

Usage

Experiment$log_parameter(name, value, step = NULL)

Arguments:

• name (Required) Name of the parameter.
• value (Required) Value of the parameter.
• step Step number.

Method get_parameters()¶

Get an experiment's parameters summary.

Usage

Experiment$get_parameters()

Method log_other()¶

Log a key/value `other`` data (not a metric or parameter). Note that you can only retrieve others summary data (e.g., this is not recorded as a full time series).

Usage

Experiment$log_other(key, value)

Arguments:

• key (Required) The key.
• value (Required) The value.

Method get_other()¶

Get an experiment's others (logged with log_other()) summary.

Usage

Experiment$get_other()

Method add_tags()¶

Add a list of tags to an experiment.

Usage

Experiment$add_tags(tags)

Arguments:

• tags (Required) List of tags.

Method get_tags()¶

Get an experiment's tags.

Usage

Experiment$get_tags()

Method log_html()¶

Set (or append onto) an experiment's HTML.

Usage

Experiment$log_html(html, override = FALSE)

Arguments:

• html (Required) An HTML string to add to the experiment.
• override If TRUE, override the previous HTML. If FALSE, append to it.

Method get_html()¶

Get an experiment's HTML.

Usage

Experiment$get_html()

Method upload_asset()¶

Upload a file to the experiment.

Usage

Experiment$upload_asset(
  file,
  step = NULL,
  overwrite = NULL,
  context = NULL,
  type = NULL,
  name = NULL,
  metadata = NULL
)

Arguments:

• file (Required) Path to the file to upload.
• step Step number.
• overwrite If TRUE, overwrite any uploaded file with the same name.
• context The context.
• type The type of asset.
• name Name of the file on comet. By default the name of the file will match the file that you upload, but you can use this parameter to use a different name.
• metadata Metadata to upload along with the file.

Method log_remote_asset()¶

Logs a Remote Asset identified by an URI. A Remote Asset is an asset but its content is not uploaded and stored on Comet. Rather a link for its location is stored, so you can identify and distinguish between two experiment using different version of a dataset stored somewhere else.

Usage

Experiment$log_remote_asset(
  uri,
  remote_file_name = NULL,
  step = NULL,
  overwrite = FALSE,
  type = "asset",
  metadata = NULL
)

Arguments:

• uri (Required) The remote asset location, there is no imposed format, and it could be a private link.
• remote_file_name The "name" of the remote asset, could be a dataset name, a model file name.
• step Step number.
• overwrite If TRUE, overwrite any logged asset with the same name.
• type The type of asset, default: "asset".
• metadata Metadata to log along with the asset

Method get_asset_list()¶

Get an experiment's asset list.

Usage

Experiment$get_asset_list(type = NULL)

Arguments:

• type The type of assets to retrieve (by default, all assets are returned).

Method get_asset()¶

Get an asset.

Usage

Experiment$get_asset(assetId)

Arguments:

• assetId (Required) The asset ID to retrieve.

Method create_symlink()¶

Add a symlink to an experiment in another project.

Usage

Experiment$create_symlink(project_name)

Arguments:

• project_name (Required) Project that the experiment to should linked to.

Method log_git_metadata()¶

Log an experiment's git metadata. This should only be called once and it can be done automatically by enabling log_git_info in create_experiment() or get_experiment(). This will replace any previous git metadata that was logged.

Usage

Experiment$log_git_metadata(
  branch = NULL,
  origin = NULL,
  parent = NULL,
  user = NULL,
  root = NULL
)

Arguments:

• branch Git branch name.
• origin Git repository origin.
• parent Git commit SHA.
• user Git username.
• root Git root.

Method get_git_metadata()¶

Get the git metadata of an experiment.

Usage

Experiment$get_git_metadata()

Method get_git_patch()¶

Get the git patch of an experiment.

Usage

Experiment$get_git_patch()

Method get_output()¶

Get an experiment's standard output and error.

Usage

Experiment$get_output()

Method log_code()¶

Log an experiment's source code. This should only be called once and it can be done automatically by enabling log_code in create_experiment() or get_experiment(). This will replace any previous code that was logged.

Usage

Experiment$log_code(code)

Arguments:

• code The code to set as the source code.

Method get_code()¶

Get an experiment's source code.

Usage

Experiment$get_code()

Method log_system_details()¶

Log system details. This can be done automatically by enabling log_system_details in create_experiment() or get_experiment().

Usage

Experiment$log_system_details(
  command = NULL,
  executable = NULL,
  hostname = NULL,
  installed_packages = NULL,
  gpu_static_info = NULL,
  ip = NULL,
  network_interface_ips = NULL,
  additional_system_info = NULL,
  os = NULL,
  os_packages = NULL,
  os_type = NULL,
  pid = NULL,
  user = NULL,
  r_version = NULL,
  r_version_verbose = NULL
)

Arguments:

• command Script and optional arguments.
• executable Executable.
• hostname Hostname.
• installed_packages List of installed R packages.
• gpu_static_info List of GPU information, where each GPU is a list() with fields gpuIndex, name, powerLimit, totalMemory, uuid.
• ip IP address.
• network_interface_ips List of network interface IPs.
• additional_system_info List of additional parameters to log, where each parameter is a list() with key and value pairs.
• os Full details about operating system.
• os_packages List of operating system packages installed.
• os_type Operating system type.
• pid Process ID.
• user User.
• r_version Short form R version.
• r_version_verbose Long form R version.

Method get_system_details()¶

Get an experiment's system details.

Usage

Experiment$get_system_details()

Method log_artifact()¶

Log an Artifact object, synchronously create a new Artifact Version and upload all local and remote assets attached to the Artifact object.

Usage

Experiment$log_artifact(artifact)

Arguments:

• artifact an Artifact object.

Method get_artifact()¶

Returns a logged artifact object that can be used to access the artifact version assets and download them locally.

If no version or alias is provided, the latest version for that artifact is returned.

Usage

Experiment$get_artifact(
  artifact_name,
  workspace = NULL,
  version_or_alias = NULL
)

Arguments:

• artifact_name (Required) Retrieve an artifact with that name. This could either be a fully qualified artifact name like workspace/artifact-name:versionOrAlias or just the name of the artifact like artifact-name.
• workspace Retrieve an artifact belonging to that workspace.
• version_or_alias Retrieve the artifact by the given alias or version.

Example:

library(cometr)
# Assuming you have COMET_API_KEY, COMET_WORKSPACE, COMET_PROJECT_NAME variables define
exp <- create_experiment()

# Get a Comet Artifact
logged_artifact <- exp$get_artifact("workspace/artifact-name:version_or_alias")

# Which is equivalent to
logged_artifact = exp$get_artifact(artifact_name="artifact-name",
                                   workspace="workspace",
                                   version_or_alias="version_or_alias")

Method set_start_end_time()¶

Set an experiment's start and end time.

Usage

Experiment$set_start_end_time(start = NULL, end = NULL)

Arguments:

• start Start time for the experiment (milliseconds since the Epoch)
• end End time for the experiment (milliseconds since the Epoch)

Method print()¶

Print the experiment.

Usage

Experiment$print()