Reports¶

The Model Performance page shows the last 7, 30 or 90 days. A report covers whatever window you ask for — half a year, a full year — as a single document you can read offline, attach to a review, or send to someone without a Comet account.

comet-mpm report --workspace WORKSPACE --model MODEL --days 365 --out annual.pdf

Reports are built by the comet-mpm command line tool, which is installed with the MPM SDK.

Install¶

pip install "comet-mpm[report]"          # charts and HTML
pip install "comet-mpm[report-pdf]"      # and PDF
playwright install chromium              # once, for PDF only

PDF is produced by printing the HTML with a headless browser, so the two outputs always match. If you only need HTML, the smaller install is enough.

Set your API key before running anything:

export COMET_API_KEY=<your API key>

Your first report¶

comet-mpm report -w my-workspace -m fraud-detector --out report.html

That covers the last 200 days in weekly buckets and contains:

• the number of predictions
• drift, percentiles and category mix for the first few columns

Output columns come first. Drift on what a model predicts is usually the question people ask before drift on what it was fed.

Every report opens with a cover page naming the model, the period, the interval, when it was generated and against which installation — so a report found six months later still says what it describes.

What a report can contain¶

Charts are the same measurements the MPM UI draws, over a longer window.

Choose columns explicitly with --feature, which accepts input columns (feature_*) and output columns (prediction_*) alike:

comet-mpm report -w WS -m MODEL \
    --feature prediction_probability \
    --feature feature_age \
    --out report.html

Drift sections use EMD by default. The other algorithms — PSI, KL and KS — and when each is the right choice are covered in Drift; set one per section in a spec with drift_algorithm.

Column names carry the prefix the SDK uses — see Dashboard for how age becomes feature_age. Naming a column the model does not have lists the ones it does.

The window and the interval¶

--days sets how much history to cover. --interval sets how it is bucketed:

--interval W-SUN     # weekly (the default)
--interval MS        # monthly
--interval daily     # one point per day

A year of daily points is 365 marks in a chart a few inches wide, so a long report normally wants weekly or monthly buckets.

Each chart is drawn against the report's full window, so a metric that has no data for part of the period shows that as empty space rather than silently rescaling. Charts in one report can therefore be read against each other.

Custom metrics¶

Any metric you can write as custom metric SQL can be a section:

comet-mpm report -w WS -m MODEL \
    --metric-sql "SELECT AVG(prediction_probability) FROM MODEL" \
    --metric-sql "SELECT COUNT(*) FILTER (WHERE prediction_decision = 'approve') * 1.0 / COUNT(*) FROM MODEL" \
    --out report.html

Repeat --metric-sql for more than one. Because the report cannot know what your SQL means, a custom metric is never aggregated automatically — it is shown at the interval it was queried at, and the chart says so.

HTML and PDF¶

The format follows the file extension, or set --format explicitly:

comet-mpm report -w WS -m MODEL --out report.pdf     # PDF
comet-mpm report -w WS -m MODEL --out report.html    # HTML

HTML charts stay interactive — hover for values, zoom, and open the table behind each chart. The file is self-contained, so it works offline.

PDF is portrait A4, one cover page and two charts per page, with the figures printed beneath each chart. This is the one to attach to a review.

Both carry a short summary under every chart — the total, the busiest and quietest buckets, and how much the metric moved over the period — so the numbers do not have to be read off an axis.

Reusing a report¶

Write the report down instead of running it, then run it whenever you like:

comet-mpm report -w WS -m MODEL --days 365 --interval MS --print-spec > quarterly.json
comet-mpm report --spec quarterly.json --out q3.pdf

The spec is readable JSON you can edit — reorder sections, change the interval, add a title. It stores --days rather than fixed dates, so re-running it next quarter reports on that quarter. For a frozen window add an explicit end:

{"days": 90, "end": "2026-06-01T00:00:00+00:00"}

Running a Custom panel as a section¶

A Custom panel and a report want the same chart from the same data, so a report can run a panel script directly:

comet-mpm report -w WS -m MODEL \
    --script my_panel.py \
    --script-params params.json \
    --out report.html

The script runs unmodified. Its st.plotly_chart calls become the section's charts and its print output appears beside them.

The one difference is the controls. A panel asks a person; a report has nobody to ask, so the answers come from --script-params, keyed by the script's filename and then by the control's label:

{
  "my_panel.py": {
    "Feature": "feature_age",
    "Window": 21,
    "Show training range": true
  }
}

Controls you do not set fall back to their own defaults, so a script runs with no parameters at all and produces what the panel would show before anyone touched anything. The values used are printed above the chart, since a report has no dropdowns to display.

Options¶

--chunk-days splits a long window into several requests. Smaller windows are not cheaper — the same rows are read either way — but they avoid a single request large enough to time out, and let you watch progress on a long report.

Limits¶

• Classification curves and gains tables are not yet available as sections.
• Drift, percentiles and distributions cannot be reported at weekly or monthly intervals, for the reason given above. They appear at their native daily interval.
• Alerts do not yet appear in reports; see the Alerts page in the MPM UI.

Learn more¶

• Drift — what a drift score compares, and the four algorithms
• Custom panels — writing the scripts a report can run
• Custom metrics
• Dashboard
• The MPM platform
• Model reference