Skip to content

Use Artifacts

Comet Artifacts is a tool that provides a convenient way to log, version, and browse data from all parts of the experimentation pipeline.

An Artifact is a versioned object, where each version is an immutable snapshot of files and assets, arranged in a folder-like logical structure. This snapshot can be tracked using metadata, a version number, tags, aliases and visualization. An Artifact also tracks which experiments consumed it, and which experiment produced it.

This means that with Artifacts, you can structure your experiments as multi-stage pipelines or directed acyclic graphs (DAGs) and ensure centralized, managed and versioned access to any of the intermediate data produced in the process.

Artifacts live at the Workspace level and can be accessed across Projects.

This need to track the data used or created by an experiment or model arises frequently. In many cases, machine learning teams must:

  • Reuse data produced by intermediate or exploratory steps in their experimentation pipeline, and allow it to be tracked, versioned, consumed, and analyzed in a managed way.
  • Track and reproduce complex multi-experiment scenarios, where the output of one experiment would be used as the input of another experiment.
  • Iterate on their datasets over time, track which model used which version of the dataset, and schedule model re-training.

Create an Artifact

It takes only a few lines of code to register an Artifact of any size to Comet.

from comet_ml import Experiment, Artifact

experiment = Experiment()

artifact = Artifact(artifact_name="artifact-name", artifact_type="dataset")
artifact.add("path/to/my/file.csv")
experiment.log_artifact(artifact)

The logged Artifact can be viewed in your Workspace in the Artifacts tab. You will notice that an Artifact contain versions. This makes it easy to fetch a snapshot of data from a particular point in time, or reproduce an experiment based on its specific inputs.

Artifacts in the Workspace

Use an Artifact

The process of using an Artifact is similar to the one used to create it.

from comet_ml import Experiment, Artifact

experiment = Experiment()

logged_artifact = experiment.get_artifact("artifact-name")
local_artifact = logged_artifact.download()

Artifacts also link to the Experiments that produce them. Additionally, the Experiment view will display the Artifacts that were produced and consumed by an Experiment.

Artifacts in the Experiment View

Try out Artifacts!

May. 25, 2022