September 22, 2023
Photo by Andrea Piacquadio: https://www.pexels.com/photo/collage-photo-of-woman-3812743/ Computer vision is one of the most widely used and…
A comprehensive Model Registry is one of the most overlooked components when machine learning teams are building out their MLOps toolset. A Model Registry serves as the bridge between the training and production phases of the model lifecycle.
In Training, researchers are iterating across parameters and data, trying to find their champion model. Once found, the model is moved to a model registry. Only models stored in the registry see the light of production.
While it may feel as if the Model Registry is just a file and versioning store, a comprehensive Model Registry offers a lot more.
Comet’s Model Registry helps Machine Learning teams implement best practices by:
The Model Registry is used by different personas at different stages of the Model Lifecycle. Oftentimes a model trained by a Data Scientist is handed off to a ML Engineer who is responsible for deploying and maintaining a model. Typically ML Engineers are not involved in the model training process, giving room for miscommunication and confusion. To ensure a clean handoff, it is important that the Model Registry provides the complete context of how a model was trained. The following screenshot is a sample of how Machine Learning teams of all shapes and sizes utilize Comet’s Model registry today
From the Model Registry above it’s very easy to answer the following questions
Having a one-stop shop of all the relevant information for a model allows Machine Learning organizations to operate at high efficiency.
A model’s journey doesn’t stop at the Model Registry. Model’s are meant to be eventually downloaded and deployed to make predictions on real-time data. Oftentimes, the process for retrieving a production-grade model however is disjointed or undocumented . Comet reduces such friction by laying out explicit instructions on how to download a specific model version in different flavors (Command Line, REST, Python).
It is inevitable that models will change over time. Data and Concepts drifts are some of the biggest reasons why models start to under-perform in production. To tackle this, teams often re-train their models on new data-sets to create newer and more accurate models. Model Versioning starts to become incredibly important in such situations. The History tab in Comet’s Model Registry allows users to see all the changes for a particular model.
Not only does the History Tab capture when a new model version was added or when a model was moved to staging, but it also can track who downloaded the model. This visibility is extremely useful if things go wrong. Say it’s known that a particular model version has a bug. The model maintainer can come to the History tab, see all the individuals who downloaded the model, and alert them of the issue.
Having a Model Registry that is integrated with your other tooling allows the team to automate their internal process. Comet’s Model Registry can be integrated with your deployment infrastructure through Webhooks. By simply changing a model’s stage, users can trigger their CI/CD pipelines automatically, reducing the need for manual human intervention.
Overall, a Model Registry can provide that single point of control for managing Machine Learning models, which helps to ensure that models are used in a consistent, compliant, and efficient manner. It’s imperative for organizations to treat their models as first class citizens. It is as simple as adding a few lines of code with your model info and you are ready to get started.
Check out the example below and if you’re ready to start logging, check out our documentation!