Use the model registry

Comet has a sophisticated system for logging, registering, versioning, and deploying machine learning models. A model can be anything you want it to be---Comet makes no assumptions of what comprises a model.

The general steps in creating a Comet model pipeline are:

1. Log a model using a Python SDK Experiment.
2. Register an Experiment model.
3. Track model versions of the registered model.
4. Deploy a registered model.

These steps are described in detail in the following sections.

Log a model¶

In Comet, a model can be composed of any files or folders in the file system. The files can be local or stored remotely in s3 for example.

The first step in the model pipeline is to log all of the associated files with a model name through an Experiment (for example, Experiment, OfflineExperiment, or ExistingExperiment).

If all of the files connected to a model are in a single folder, you can simply log the model with the path to the folder:

from comet_ml import Experiment

experiment = Experiment()

experiment.log_model("MNIST CNN", "../models/run-026")

However, if the files that compose a model are separated, you can also log each file individually. In fact, you can also log multiple folders and multiple files as a single model. For more information, see Experiment.log_model().

from comet_ml import Experiment

experiment = Experiment()

experiment.log_model("MNIST CNN", "run4/layer1-weights.pickle")
experiment.log_model("MNIST CNN", "run6/layer2-weights.pickle")
experiment.log_model("MNIST CNN", "run9/layer3-weights.pickle")

from comet_ml import Experiment

experiment = Experiment()

experiment.log_remote_model("MNIST CNN", "s3://bucket/my-model/run-4/layer1-weights.pickle")

After you log a model, you will see the model and all of its individual files in the Experiment's Assets tab.

From the Experiment's Assets tab, you can download the model, see if it has been registered before, or register it as a new model, or a new version of an existing model.

You can see exactly the registry models that have been made with this Experiment model by clicking the View Existing link next to the asset model entry.

Clicking the name of the registered model takes you to the Registry view.

Register a model¶

After you log a model through an Experiment through the SDK, you can then register it. Registered models belong to a workspace, and are shared with your team there.

You can register an Experiment's model in three ways:

1. Through the Register Model button on the Experiment UI's top right corner
2. Through the Experiment's Asset tab in the Comet UI.
3. Programmatically through the Comet Python SDK.

After clicking on the Register Model button, a side bar will open up where you can choose a previously logged model to register into the Comet Model Registry.

Here, you can give the registered model a name, a version, and set its visibility to Public or Private. Note that registered model names must be all lowercase, and version strings must use proper semantic versioning. Semantic versioning are strings like "1.0.0", "2.1.5", or "1.0.0-alpha". For more examples of "semver", see, for example, Understanding Semantic Versioning Specification.

On the other hand, if you have registered a model in this Workspace before, you see the following dialog:

Here, you select one of the previous registered model names from the list, and again provide proper semantic versioning. Click Register your model.

You can also register an Experiment's model through the Python API. If you are registering the model in the same script where you logged the model, you can use Experiment.register_model(). If you are registering the model after the Experiment finished, you can use APIExperiment.register_model().

That would look similar to:

#..previosly logged model
experiment.register_model("MNIST CNN")

from comet_ml import API

api = API()

experiment = api.get("workspace-name/project-name/experiment-name")
experiment.register_model("MNIST CNN")

That's it! The "MNIST CNN" is the name of the Experiment model, and if you don't provide a registry model name, it will serve as the foundation for it as well. However, since registered model names have more constraints than Experiment model names, the registered model name becomes "mnist-cnn". Likewise, since we did not provide a version number, the string "1.0.0" was used as the default.

Track model versions¶

Registered models belong to the workspace. In this manner, you can share your models with the workspace team. At the workspace view, you can select between viewing the Projects or the registered models.

Selecting a registered model's card in the Model Registry takes you to the detailed view of the registered model.

Here you can see the latest version of the registered model, and all previous versions as well. Comet Models are closely tied to Experiments and you can access all of the Experiment data in the registry by customizing your columns. You can manage your models by adjusting their tags or formally changing the model status.

You can also alter these settings programmatically using the Python API:

from comet_ml import API
api = API()
model = api.get_model(workspace=workspace, model_name=model_name)

model.set_status(version='1.1.0', status="Production")
model.add_tag(version='1.1.0', tag='new_tag')

Programmatically, you can delete these as well:

api.delete_registry_model(workspace, registry_name)
api.delete_registry_model_version(workspace, registry_name, version)

Deploy models¶

To download a registered model as a zip file simply click the download icon next to the model name or download a specific version of the model through the table side menu. The zip file contains all of the files that were logged with the Experiment model through the Python SDK.

You can also download and deploy your model programmatically, using the Python API:

from comet_ml import API

api = API()

#get the Model object
model = api.get_model(workspace=workspace, model_name=model_name)

# Download a Registry Model:
model.download("1.0.0", output_path="./", expand=True)

You can select where you would like to download it to using the output_path parameter (using a relative or an absolute path), and determine if the zip file is expanded or not using the expand parameter.