Skip to content

cometx

   _________  ____ ___  ___  / /__  __
  / ___/ __ \/ __ `__ \/ _ \/ __/ |/_/
 / /__/ /_/ / / / / / /  __/ /__>  <
 \___/\____/_/ /_/ /_/\___/\__/_/|_|

Open source extensions for the Comet SDK.

These extensions are created and supported by the community and are not an official project of Comet ML. We welcome contributions!

Installation

pip install cometx --upgrade

To use these command-line functions, you can set your Comet API key and URL override using command-line flags:

cometx --api-key="YOUR-COMET-API-KEY" COMMAND

If you are a Comet on-prem user, and your installation does not use smart-keys, you'll also need to set the URL override:

cometx --api-key="YOUR-COMET-API-KEY" --url-override="https://your-companys-comet.com/clientlib/" COMMAND

Usage

cometx is composed of a series of commands that are useful independently, and can be used together to create sophisticated tools for ML management.

Commands

For all commands, use the --help flag to get additional information.

cometx also includes a Python API to upload panels and to create and update dashboards. See Panels and Dashboards (Python API).

Global Options

These flags are availble before a command:

  • --api-key API_KEY - Set the COMET_API_KEY
  • --url-override URL_OVERRIDE - Set the COMET_URL_OVERRIDE

This command shows the cometx version:

  • --version - Display comet_ml version

This command can be used globally, or for individual commands:

  • -h, --help - Show help message

cometx list

This command is used to:

  • get a list of all workspaces that you are a member of
  • get a list of all projects in a workspace
  • get a list of all experiments (by name or key) in a project

cometx list examples:

cometx list WORKSPACE/PROJECT/EXPERIMENT-KEY-OR-NAME
cometx list WORKSPACE/PROJECT
cometx list WORKSPACE
cometx list

Flags

  • -u, --use-name - Use experiment names for experiment folders and listings
  • --query QUERY - Only list experiments that match this Comet query string
  • --debug - Provide debug info

For more information, cometx list --help

cometx download

This command is used to:

  • download all workspaces, projects, and experiments of workspaces that you are a member of
  • download all projects, and experiments of a given workspace
  • download all experiments of a given workspace/project
  • download artifacts and models from the registry
  • download panels

Note: For detailed information on copying experiments from one Comet installation to another, see MIGRATIONS.md.

cometx download examples:

cometx download WORKSPACE/PROJECT/EXPERIMENT-KEY-OR-NAME [RESOURCE ...] [FLAGS ...]
cometx download WORKSPACE/PROJECT [RESOURCE ...] [FLAGS ...]
cometx download WORKSPACE [RESOURCE ...] [FLAGS ...]
cometx download [RESOURCE ...] [FLAGS ...]

Where [RESOURCE ...] is zero or more of the following names:

  • run - alias for: code, git, output, graph, and requirements
  • system
  • others
  • parameters
  • metadata
  • metrics
  • assets
  • html
  • project - alias for: project_notes, project_metadata

If no RESOURCE is given it will download all of them.

Flags

  • --from from - Source of data to copy. Should be: comet, wandb, or neptune
  • -i IGNORE [IGNORE ...], --ignore IGNORE [IGNORE ...] - Resource(s) (or 'experiments') to ignore
  • -j PARALLEL, --parallel PARALLEL - The number of threads to use for parallel downloading; default (None) is based on CPUs
  • -o OUTPUT, --output OUTPUT - Output directory for downloads
  • -u, --use-name - Use experiment names for experiment folders and listings
  • -l, --list - List the items at this level (workspace, project, experiment, artifacts, or model-registry) rather than download
  • --flat - Download the files without subfolders
  • -f, --ask - Queries the user; if flag not included system will answer yes for all queries
  • --filename FILENAME - Only get resources ending with this
  • --query QUERY - Only download experiments that match this Comet query string
  • --asset-type ASSET_TYPE - Only get assets with this type
  • --sync SYNC - What level to sync at: all, experiment, project, or workspace
  • --debug - Provide debug info

To download artifacts:

cometx download WORKSPACE/artifacts/NAME [FLAGS ...]
cometx download WORKSPACE/artifacts/NAME/VERSION-OR-ALIAS [FLAGS ...]

To download models from the model registry:

cometx download WORKSPACE/model-registry/NAME [FLAGS ...]
cometx download WORKSPACE/model-registry/NAME/VERSION-OR-STAGE [FLAGS ...]

To download panels:

cometx download WORKSPACE/panels/NAME-OR-ID [FLAGS ...]
cometx download WORKSPACE/panels [FLAGS ...]

For more information, cometx download --help

cometx copy

This command is used to:

  • copy downloaded data to a new experiment
  • create a symlink from one project to existing experiments
  • copy panels

Note: For detailed information on copying experiments from one Comet installation to another, see MIGRATIONS.md.

cometx copy examples:

cometx copy SOURCE DESTINATION
cometx copy --symlink SOURCE DESTINATION

where SOURCE is:

  • if not --symlink, "WORKSPACE/PROJECT/EXPERIMENT", "WORKSPACE/PROJECT", or "WORKSPACE" folder
  • if --symlink, then it is a Comet path to workspace or workspace/project
  • "WORKSPACE/panels" or "WORKSPACE/panels/PANEL-ZIP-FILENAME" to copy panels

where DESTINATION is:

  • WORKSPACE
  • WORKSPACE/PROJECT

Not all combinations are possible:

Destination →
Source ↓
WORKSPACEWORKSPACE/PROJECT
WORKSPACE/*/*Copies all projectsN/A
WORKSPACE/PROJ/*N/ACopies all experiments
WORKSPACE/PROJ/EXPN/ACopies experiment

Asset Types

  • 3d-image
  • 3d-points - deprecated
  • audio
  • confusion-matrix - may contain assets
  • curve
  • dataframe
  • dataframe-profile
  • datagrid
  • embeddings - may reference image asset
  • histogram2d - not used
  • histogram3d - internal only, single histogram, partial logging
  • histogram_combined_3d
  • image
  • llm_data
  • model-element
  • notebook
  • source_code
  • tensorflow-model-graph-text - not used
  • text-sample
  • video

Flags

  • -i IGNORE [IGNORE ...], --ignore IGNORE [IGNORE ...] - Resource(s) (or 'experiments') to ignore
  • --debug - If given, allow debugging
  • --quiet - If given, don't display update info
  • --symlink - Instead of copying, create a link to an experiment in a project
  • --sync - Check to see if experiment name has been created first; if so, skip

For more information, cometx copy --help

cometx log

This command is used to log a resource (metrics, parameters, asset, etc) file to a specific experiment or experiments.

cometx log examples:

cometx log WORKSPACE/PROJECT/EXPERIMENT-KEY FILENAME ... --type=TYPE
cometx log WORKSPACE PANEL-ZIP-FILENAME ... --type=panel
cometx log WORKSPACE PANEL.py ... --type=panel
cometx log WORKSPACE PANEL-URL ... --type=panel
cometx log WORKSPACE/PROJECT --type=other --set "key:value"
cometx log WORKSPACE --type=other --set "key:value"

Where TYPE is one of the following names:

  • all
  • asset
  • audio
  • code
  • image
  • metrics
  • notebook
  • panel
  • tensorflow-file
  • text-sample
  • video
  • other
  • tensorboard-folder-assets

Flags

  • --type TYPE - The type of item to log
  • --set SET - The key:value to log
  • --query QUERY - A Comet Query string, see https://www.comet.com/docs/v2/api-and-sdk/python-sdk/reference/API/#apiquery
  • --debug - If given, allow debugging
  • --use-base-name - If given, using the basename for logging assets

For more information, cometx log --help

cometx delete-assets

To delete experiments assets:

cometx delete-assets WORKSPACE/PROJECT --type=image
cometx delete-assets WORKSPACE/PROJECT/EXPERIMENT --type=all

Type can be valid asset type, including:

  • all
  • asset
  • audio
  • code
  • image
  • notebook
  • text-sample
  • video

Flags

  • --type TYPE - The type of item to log
  • --debug - If given, allow debugging
  • --query QUERY - Only delete experiments that match this Comet query string

For more information, cometx delete-assets --help

cometx config

To enable auto-logging of your notebooks in Jupyter environments:

cometx config --auto-log-notebook yes

To turn auto-logging of notebooks off, use:

cometx config --auto-log-notebook no

If you keep the generated experiment URLs in the notebook, but later edit the notebook, the notebooks will be updated in all of the experiments created in the notebook.

Flags

  • --debug - If given, allow debugging
  • --auto-log-notebook AUTO_LOG_NOTEBOOK - Takes a 1/yes/true, or 0/no/false

For more information, cometx config --help

cometx reproduce

cometx reproduce [-h] [--run] [--executable EXECUTABLE] COMET_PATH OUTPUT_DIR

This command is used to reproduce experiments by copying files to a specified output directory.

Flags

  • --run - Run the reproducible script
  • --executable EXECUTABLE - Run the reproducible script with specified executable

For more information, cometx reproduce --help

cometx update

cometx update [-h] [--debug] COMET_SOURCE COMET_DESTINATION

To update existing experiments.

cometx update SOURCE DESTINATION

where SOURCE is a folder:

  • "WORKSPACE/PROJECT/EXPERIMENT"
  • "WORKSPACE/PROJECT"
  • "WORKSPACE"

where DESTINATION is a Comet:

  • WORKSPACE
  • WORKSPACE/PROJECT

Flags

  • --debug - If given, allow debugging

For more information, cometx update --help

cometx admin

cometx admin [-h] [--host HOST] [--debug] ACTION [YEAR-MONTH]

To perform admin functions

cometx admin chargeback-report

Flags

  • --host HOST - Override the HOST URL
  • --debug - If given, allow debugging

For more information, cometx admin --help

cometx smoke-test

cometx smoke-test [-h] [--exclude [EXCLUDE ...]] [--debug DEBUG] COMET_PATH [include ...]

Perform a smoke test on a Comet installation. Logs results to WORKSPACE/smoke-tests or WORKSPACE/PROJECT.

Examples:

Run all tests:

cometx smoke-test WORKSPACE    # project defaults to smoke-tests
cometx smoke-test WORKSPACE/PROJECT

Run everything except mpm tests:

cometx smoke-test WORKSPACE/PROJECT --exclude mpm

Run just optimizer tests:

cometx smoke-test WORKSPACE/PROJECT optimizer

Run just metric tests:

cometx smoke-test WORKSPACE/PROJECT metric

Items to include or exclude:

  • optimizer
  • mpm
  • panel
  • opik
  • experiment
  • metric
  • image
  • asset
  • dataset-info
  • confusion-matrix
  • embedding

Flags

  • --exclude [EXCLUDE ...] - Items to exclude; any of: asset, confusion-matrix, dataset-info, embedding, experiment, image, metric, mpm, opik, optimizer, panel
  • --debug DEBUG - Show debugging information

For more information, cometx smoke-test --help

Copy/Download Use Cases

In this section we'll explore some common scenarios.

  1. Copy a specific project from one Comet installation to another
  2. Copy all projects in workspace to a new workspace
  3. Copy specific experiments in a project to new experiments

1. Copy a specific project from one comet installation to another

A useful idiom is to set your Comet environment variables on the line of a command. In this manner, you can set the COMET_URL_OVERRIDE and COMET_API_KEY for different installations.

Of course, you don't have to set the environment variables if you are copying experiments on the same Comet installation.

Here is how you one could download the experiments in WORKSPACE/PROJECT from http://comet.a.com:

cometx --api-key=A-KEY download WORKSPACE/PROJECT

The cometx download command downloads all of the Comet experiment data into local files. Note that WORKSPACE/PROJECT refers to a workspace and project on http://comet.a.com.

One could then copy the downloaded experiment data with a similar command:

cometx --api-key=B-KEY copy WORKSPACE/PROJECT NEW-WORKSPACE/NEW-PROJECT

Note that WORKSPACE/PROJECT now refers to a directory, and NEW-WORKSPACE/NEW-PROJECT refers to a workspace and project on http://comet.b.com.

2. Copy all projects in workspace to a new workspace

Similarly, one can copy all of the projects by first downloading them:

cometx --api-key=A-KEY download WORKSPACE

and then copying them:

cometx --api-key=B-KEY copy WORKSPACE NEW-WORKSPACE

3. Copy specific experiments in a project to new experiments

Similarly, one can copy a single experiment first downloading it:

cometx --api-key=A-KEY download WORKSPACE/PROJECT/EXPERIMENT-NAME-OR-ID

and then copying it:

cometx --api-key=B-KEY copy WORKSPACE/PROJECT/EXPERIMENT-NAME-OR-ID NEW-WORKSPACE/NEW-PROJECT

Panels and Dashboards (Python API)

In addition to the command-line tools, cometx includes a Python API for managing code panels and dashboards programmatically. The cometx.API class extends the comet_ml API class with methods to upload panels and to create and update dashboards. The cometx.views module builds dashboard layouts: sections, charts, filters, and experiment table columns.

Note

These features require cometx 3.6.8 or later. cometx.views also requires comet_ml 3.57.0 or later, which adds the get_views() and create_view() methods.

Upload a panel

Use upload_panel_code(workspace, panel_name, code, template_id=None) to upload Python code as a panel in a workspace. It returns a dict that contains the panel's templateId. If you pass a template_id, the existing panel is overwritten in place instead of creating a new panel.

from cometx import API

api = API()

# Create a new panel
result = api.upload_panel_code("my-workspace", "My Panel", "print('hello world')")
panel_id = result["templateId"]

# Update an existing panel's code in place
result = api.upload_panel_code(
    "my-workspace", "My Panel", "print('updated')", template_id=panel_id
)

Use upload_panel_zip(workspace, filename, template_id=None) to upload a panel zip file. The template_id argument works in the same way:

# Create a new panel
result = api.upload_panel_zip("my-workspace", "panel.zip")

# Update an existing panel in place
result = api.upload_panel_zip("my-workspace", "panel.zip", template_id="abc123")

Note

Dashboards that use a panel show the updated code after an in-place update. Refresh the browser page to see the change in the Comet UI.

Create a dashboard

Use create_dashboard(workspace, project_name, template_name, template_id=None, panels=None) to create a new dashboard in a project:

from cometx import API

api = API()

# Create an empty dashboard
dashboard = api.create_dashboard(
    workspace="my-workspace",
    project_name="my-project",
    template_name="My Dashboard",
)

# Create a dashboard with panels
dashboard = api.create_dashboard(
    workspace="my-workspace",
    project_name="my-project",
    template_name="My Dashboard",
    panels=["panel-template-id-1", "panel-template-id-2"],
)

# Create a dashboard cloned from an existing one (copies panel associations)
dashboard = api.create_dashboard(
    workspace="my-workspace",
    project_name="my-project",
    template_name="My Dashboard Copy",
    template_id="existing-dashboard-template-id",
)
ParameterTypeDescription
workspacestrThe workspace name the project belongs to
project_namestrThe project name to create the dashboard in
template_namestrName for the new dashboard
template_idstr (optional)Source dashboard template ID to clone panel associations from
panelslist (optional)List of panel templateId values to include in the new dashboard

Returns a dict that represents the created DashboardTemplate, including its new template_id.

Update a dashboard

Use update_dashboard(workspace, project_name, template_id, template_name=None, panels=None) to rename an existing dashboard, replace its panels, or both:

from cometx import API

api = API()

# Rename a dashboard
api.update_dashboard(
    workspace="my-workspace",
    project_name="my-project",
    template_id="existing-dashboard-id",
    template_name="Renamed Dashboard",
)

# Replace the panels on a dashboard
api.update_dashboard(
    workspace="my-workspace",
    project_name="my-project",
    template_id="existing-dashboard-id",
    panels=["panel-template-id-1", "panel-template-id-2"],
)
ParameterTypeDescription
workspacestrThe workspace name the project belongs to
project_namestrThe project name the dashboard belongs to
template_idstrThe ID of the dashboard to update
template_namestr (optional)New name for the dashboard
panelslist (optional)List of panel templateId values to set on the dashboard

Returns a dict that represents the updated DashboardTemplate.

Typical workflow

from cometx import API

api = API()

# 1. Upload a panel
result = api.upload_panel_code("my-workspace", "Hello World", "print('hello world')")
panel_id = result["templateId"]

# 2. Create a dashboard with the panel
dashboard = api.create_dashboard(
    workspace="my-workspace",
    project_name="my-project",
    template_name="My Dashboard",
    panels=[panel_id],
)
dashboard_id = dashboard["template_id"]

# 3. Later, update the panel code in place
api.upload_panel_code(
    "my-workspace", "Hello World", "print('updated')", template_id=panel_id
)
# Refresh your browser to see the updated panel in the Comet UI

# 4. Add more panels to the dashboard
api.update_dashboard(
    workspace="my-workspace",
    project_name="my-project",
    template_id=dashboard_id,
    panels=[panel_id, "another-panel-id"],
)

Build dashboard layouts with cometx.views

create_dashboard() and update_dashboard() manage the dashboard record: its name, its ID, and the code panels attached to it. They do not set the dashboard layout. The layout (sections, charts, axes, filters, and table columns) is stored in the dashboard's serialized state, and cometx.views builds that state.

To do thisUse
Attach uploaded code panels to a dashboardAPI.create_dashboard() / API.update_dashboard()
Lay out built-in charts, sections, and filterscometx.views + API.create_view()

cometx.views makes no network calls. It returns a comet_ml View object. Pass this object to API.create_view(workspace, project_name, view) for a project dashboard, or to APIExperiment.create_view(view) for the page of a single experiment.

from cometx import API
from cometx.views import Section, build_view, line_panel, scalar_panel

view = build_view(
    "Training Overview",
    sections=[
        Section(
            "At a glance",
            [
                scalar_panel("accuracy", aggregation="max"),
                scalar_panel("loss", aggregation="min"),
            ],
            columns=2,
        ),
        Section(
            "Curves",
            [
                line_panel(["loss", "val_loss"], name="Loss", smoothing=0.6),
                line_panel(["accuracy", "val_accuracy"], name="Accuracy"),
            ],
            columns=2,
        ),
    ],
)

API().create_view("my-workspace", "my-project", view)

The new dashboard appears in the view selector at the top of the project page.

Panel factories

Each factory returns a panel dict with a new chartId. Build a new view object for each target. Do not reuse one view object across projects or experiments.

FactoryProduces
line_panel(y, x="step", smoothing=0, transform_y=None, locked=False, ...)Line chart; y can be a list of metrics to overlay
bar_panel(metric, aggregation="last", plot_type="BAR", group_by_aggregation=None, ...)Bar, box, or violin plot, one bar per experiment
scatter_panel(x, y, z=None, metrics=None, params=None, ...)2D or 3D scatter plot, one point per experiment
parallel_panel(params, metrics=(), target="", ...)Parallel coordinates chart for a hyperparameter sweep
scalar_panel(metric, aggregation="last", precision=3, ...)Single-number tile
image_panel(images=(), step=None, ...)Image grid across experiments
data_panel(file_name, axis=0, join_type="outer", ...)Table from a logged tabular asset
custom_panel(instance_id, instance_name="", ...)An instance of a JavaScript panel
python_panel(template_id, revision_id="", ...)An instance of a Python panel

legend_key(value, label=None, source="log_other") builds one entry of a chart's legend. Use it to color a chart by experiment name, tags, or a hyperparameter.

Note

You can create built-in panels from nothing. custom_panel() and python_panel() refer to a panel instance that already exists, so you cannot invent their IDs. Read the IDs from a panel that is already on a dashboard with get_views(), or from the panel's URL in the Comet UI.

Sections

Section(title, panels=(), columns=3, expanded=True, height=1) is a titled, collapsible group of panels. The dashboard grid is 6 units wide, so columns must divide 6 evenly (1, 2, 3, or 6). Section calculates the grid position of each panel for you. height is the row height in grid units (maximum 3).

from cometx.views import Section, image_panel, data_panel

section = Section("Media & tables", columns=2, height=2, expanded=False)
section.add(image_panel(["prediction-grid.png"]), data_panel("per_class_metrics.csv"))

add() returns the section, so you can chain calls.

Project dashboards

build_view(name, sections, filters=None, table=None, config=None, auto_generate=False, personal=False) assembles sections into a project dashboard View.

ParameterTypeDescription
namestrThe display name of the dashboard
sectionslistSection objects, from top to bottom
filtersstr (optional)Output of query_state(): a saved filter
tablestr (optional)Output of table_state(): experiment table columns and sorting
configdict (optional)Output of global_config(): dashboard-wide x-axis, smoothing, and sampling
auto_generateboolIf True, Comet adds auto-generated panels for newly logged metrics below your sections
personalboolIf True, only you can see the dashboard

Build filters with rule(), all_of(), and any_of():

from cometx.views import (
    all_of, any_of, build_view, global_config, query_state, rule, table_state
)

view = build_view(
    "Successful runs",
    sections=[...],
    # accuracy > 0.9 OR loss < 0.1
    filters=query_state(
        any_of(
            all_of(rule("accuracy", "greater", 0.9)),
            all_of(rule("loss", "less", 0.1)),
        )
    ),
    table=table_state(
        columns=["Name", "experimentTags", "learning_rate", "accuracy", "duration"],
        sort_by="accuracy",
        page_size=50,
    ),
    config=global_config(x_axis="step", smoothing=0.3, sample_size=500),
)

global_config() applies to all panels that do not have locked=True. A locked panel keeps its own axis and smoothing settings.

Experiment views

build_experiment_view(name, sections, x_axis="step", smoothing=0, ...) assembles the same sections into a view for the page of one experiment. Pass the result to APIExperiment.create_view(view):

from comet_ml import API as CometAPI
from cometx.views import Section, build_experiment_view, line_panel

experiment = CometAPI().get_experiments("my-workspace", "my-project")[0]
view = build_experiment_view(
    "Run layout",
    sections=[Section("Curves", [line_panel(["loss", "val_loss"])], columns=2)],
    smoothing=0.4,
)
experiment.create_view(view)

Project dashboards and experiment views store their state in different places: project state is in view.v3, and experiment state is in view.v2["panels"]. For this reason, there are two builders. Sections and panels are the same for both. Do not set experiment_key yourself; create_view() sets it for you.

Warning

APIExperiment.create_view() accepts the result of build_view(), but the view that it creates has no panels. Use build_experiment_view() for experiment views.

Inspect a view

describe_view(view, indent="") returns a readable outline of the sections, panels, and filters of a view. Use it to examine the results of get_views() before you copy or change them:

from cometx import API
from cometx.views import describe_view

for view in API().get_views("my-workspace", "my-project"):
    print(describe_view(view))

Known limitations

  • create_view() always creates a new dashboard. If you call it two times with the same name, you get two dashboards, and there is no delete_view(). To update a dashboard in place, you need its existing template_id. create_view() cannot keep this ID, because it calls as_portable(), which clears it. Use the lower-level upsert instead:

    view.template_id = existing_view.template_id
    api._client.upsert_view(api.get_project(workspace, project)["projectId"], view)
    
  • Panels for metrics that the project never logged are empty. They do not show an error. When you apply one layout to many projects, read the metric names of each project first, and build only the panels that have data.

  • as_portable() makes a view movable between projects, workspaces, or accounts. It removes template_id, project_id, experiment_key, pinned_experiments, and the experimentKey in v2 and v3. It does not change experiment keys inside chart_state, so you must manually fix a panel that is pinned to a specific run.

  • "Unsaved Changes" views are the per-user scratch state of the UI for a project. They are included in the results of get_views(). Skip them when you copy views.

Examples

The examples/ directory of the cometx repository has runnable scripts for all of the features above. See the examples README for a full walkthrough. Scripts 04, 05, and 06 do a dry run by default; pass --apply to write changes.

ScriptDemonstrates
00_seed_demo_project.pyLogs six small runs (metrics, parameters, an image, and a CSV table) so that each panel type in the other scripts has data.
01_inspect_views.pyReads views at the project level (API.get_views) and the experiment level (APIExperiment.get_views).
02_build_project_view.pyBuilds a project dashboard with most panel types, a saved filter, a configured experiment table, and dashboard-wide smoothing and sampling.
03_build_experiment_view.pyBuilds the same sections as an experiment view. --all applies one layout to every run in a project.
04_migrate_views.pyCopies views between projects, workspaces, or accounts (--dst-api-key), with --match filtering.
05_standardize_workspace.pyApplies one standard dashboard to all projects in a workspace. --update updates dashboards in place instead of adding duplicates.
06_harvest_panels.pyFinds custom JavaScript and Python panels in existing views and rebuilds them into a new view.

Running Tests

WARNING: Running the tests will create experiments, models, assets, etc. in your default workspace if not set otherwise.

To run the tests, you can either export all of these items in the environment:

$ export COMET_USER="<USERNAME>"
$ export COMET_WORKSPACE="<WORKSPACE>"
$ export COMET_API_KEY="<API-KEY>"
$ pytest tests

Or, define workspace and api_key in your ~/.comet.config file:

$ export COMET_USER="<USERNAME>"
$ pytest tests
Nov. 10, 2026