Comet Platform¶
The powerful and intuitive Comet UI serves as a valued workbench among many ML teams for managing and tracking their research and experimentation.
All Comet products share the same Comet UI, as per the mapping below.
| Comet product | Comet UI page |
|---|---|
| Experiment Management, Comet Optimizer | Project pages, Single Experiment page |
| Comet Artifacts | Artifacts page |
| Comet Model Registry | Model Registry page |
| CometLLM | Prompts page |
| Comet Production Monitoring | MPM page |
Note that Comet UI pages for all Comet products are on the way!
Comet automatically reports and keeps up-to-date all your Comet experiments, artifacts, models, prompts and chains, and production metrics in the Comet UI.
Within one centralized platform, Comet UI unlocks for you:
Automated lineage: Comet supports out-of-the-box and custom tracking across the AI lifecycle, without having to query databases or set up manual and static visualizations.
Collaboration: The Comet UI is shared across an Organization and can be broken down into Workspaces, allowing teams to easily manage and share experiments with granularity.
Customization: Comet is built with an API-first approach. This allows you to create any visualizations via the Python panel and introduce any integrations without waiting for Comet engineering cycles.
The structure of Comet¶
The Comet platform follows an hierarchical organization structure.
By defining a four-layer platform architecture (organization → workspace → project → experiments), Comet offers extra flexibility in managing your AI experimentation and how your team members can access them.
Introduction to Comet concepts¶
Below we provide an high-level definition of the foundational concepts behind Comet's platform structure before diving into each concept in more detail.
| Term | Definition |
|---|---|
| Organization | The top level of a Comet account. It contains one or more workspaces. Each user belongs to one and only one organization. Billing is done at an organization level. |
| Workspace | A collection of projects, used to manage user access to projects. It is often used to differentiate between teams. Within each organization there are a number of workspaces that are used to manage user permissions within an organization. They often map to teams within a company, but not necessarily. Models and datasets are available at a workspace level. Workspace names are globally unique. |
| Project | Either an ML Project, i.e. a collection of experiments, or an LLM Project, i.e. a collection of prompts and chains. A project is often used to differentiate experiments per use case. Within each workspace, users can define projects that are used to manage each modeling project. |
| Experiment | A unit of measurable research that defines a single run with some data, parameters, code, and results. |
| Model Registry | A centralized collection of all your trained models at different lifecycle stages. Find more info in the Comet Model Registry guide. |
| Artifact | A set of assets represented as a folder that can be sorted and versioned in Comet. Find more info in the Comet Artifacts guide. |
| MPM | Model Production Monitoring - a Comet feature that allows you to track models in production. Find more info in the Comet Production Monitoring guide. |
| View | Each project can have multiple views that correspond to different dashboards. Find more info in the Project Panels page UI docs. |
Note
MPM is a custom feature. To enable it, contact our sales team.
Organization¶
An Organization contains one or more workspaces.
A default organization is created on user sign-up. From the admin dashboard, an admin can edit the organization name, invite other users to the organization and remove them from the organization. Learn more from the Admin dashboard UI docs.
Workspace¶
A Workspace is a collection of projects.
When you first create your account, Comet creates a default workspace for you with your username. This default workspace contains your personal private and public projects. In addition, you can create additional workspaces, where each workspace can have its own set of members.
The default workspace is used in two main ways:
- Comet Dashboard: When you first log in you will be sent to your default workspace. You can always change the workspace using the dropdown in the user menu or on the project list page.
- Comet SDK: When logging an experiment to Comet specifying the workspace that the experiment should be logged to is optional. If you don't specify one then the experiment goes to the default workspace, otherwise it goes to the specified workspace.
A Workspace has four tabs:
- Projects: Visualizes the performance of machine learning training runs.
- Model Registry: Visualizes trained models and manage their deployment.
- Artifacts: Tracks and versions large datasets.
- Model Production Monitoring: Tracks the performance of models deployed to production.
Note
You can allow collaborators on your personal workspace and create private projects by upgrading your plan. If you ever have any questions, feel free to send us a note at support@comet.com or on our Slack channel.
View a workspace¶
The Workspace can be displayed as either a grid of Project cards, or as a Project list.
You can toggle between the two views by clicking on the Grid icon or the List icon by the top right of the Projects page.
In both views, you can:
- Filter projects by owner or visibility (public or private) or category.
- Search projects by name or description.
- Sort projects by name, description, last updated, experiment counts, or creation date.
In the Projects list view, a configuration button in the right-most column lets you:
- Edit a project's name, description, or visibility (public versus private).
- Delete a project.
- Share a project.
- Change icon image for the project.
In the Projects grid view, these options are accessible through the vertical ellipsis in the upper-left corner of each card.
Project¶
Comet supports two types of projects:
- ML Projects track your ML experiments including both training and fine-tuning.
- LLM Projects track your LLM prompt engineering workflows.
A Project is either private (viewable and editable by all collaborators with the proper permission level) or public (editable by the owner, and viewable by everyone). You can set a project’s public or private visibility by clicking on the Gear icon by the project's entry in the Projects page.
ML Project¶
The ML Project UI pages allows you to manage, select, and analyze all, or subsets of, the project's experiments.
The Project UI pages consist of:
- Panels: Visualizations of important metrics that are used in your Experiments. Please refer to the Visualizations section for information on how you can add visualizations that serve your needs.
- Experiments: Displays the Experiment table.
- Notes: Allows you to make notes in this project.
- Archive: Shows all experiments for this project that have been archived. From here, you can permanently delete or restore an experiment or group of experiments.
- Manage: For managing Projects. See below for more details.
- Reports: Shows all the reports for this Project.
Tip
You can group visualizations in the Panels page into sections for easier management!
The Project Panels page also presents the following useful shortcuts:
- Current Project view name. If you are using the default view generated by Comet, Auto Generated is displayed. Click to display a list of possible views.
- Save View lets you turn changes in your current Project into a custom Project view. For more information, refer to the Save, edit, delete, and update dashboard views section below.
- Edit Layout lets you customize the size and position of the panels.
- Add lets you add an experiment, report, or panel.
- Options provides a shareable URL for this project, and controls for decimal precision and auto refresh.
Find more information in the Project Pages UI docs.
Manage access¶
The Manage page allows you to control a project's visibility. Here, you can make a project public (anyone can see the project) or private (you choose who can see the project).
By clicking Create Shareable Link, you can create a link to the project. The link works, even if a project is private. Each project can have a single, shareable link.
At any time, you can delete a shareable link by clicking the Recycle icon next to the link.
Find more information in the The Manage Page UI docs.
Create visualizations¶
Project visualizations, known as panels, allow you to view and compare performance across multiple experiments.
Comet supports many types of visualizations, including line charts, bar charts, scatter charts, and parallel coordinates charts. In addition, there is an entire marketplace of additional reports, exporters, and tools available in the Panel Gallery.
Find more information in the Project Panels page docs page.
Tip
All panels in the Project page are editable! Simply click the three vertical dots in the top right of the panel, and select Edit Panel.
LLM Project¶
The LLM Prompts page allows you to manage, select, and analyze all, or subsets of, the project's prompts and chains.
The Prompts page visualizes the Prompts table, i.e. a table view of all logged prompts and chains with relevant information.
The Prompts page also presents the following useful shortcuts:
- Search lets you filter across prompts and chains via exact, approximate, or wildcard search.
- Columns allows you to edit the table view to visualize the metadata and details of your choice.
- Group by provides you with the ability to group table entries by some metadata and details of your choice.
Find more information in the Prompts Page UI docs.
Experiment¶
An Experiment is a unit of measurable research that defines a single run with data, parameters, code, and results.
Tip
Under the hood, LLM prompts are also Experiment objects!
The Single Experiment page for ML Projects¶
Comet automatically tracks and collects all parameters, metadata, and other assets that you log after creating an Experiment object in your code.
Other functionalities supported by the Comet UI for your single experiment include:
- Manage experiment metadata and tags
- Register a model
- Reproduce an experiment
- Archive the experiment
- Move the experiment
Find more information in the Single Experiment page UI docs.
The Prompt sidebar for LLM Project¶
You can access all information for a single logged prompt or chain by clicking on the desired entry in the Prompts table.
For any prompt engineering workflows, the Comet UI reports for you:
- Inputs / Outputs
- Metadata
- Images
- Details
- For chains only:
- Gantt chart of each chain step with associated execution time
Find more information in The Prompt sidebar UI docs.