Ground truth¶

Ground truth — the label — is what actually happened. It is different from every other kind of data in MPM because it arrives late.

Why late arrival matters¶

Input features and predictions are known the moment a prediction is made. Whether a transaction was really fraudulent, or whether a customer really responded, is known days or weeks later.

   t0                                                    t0 + days
    │                                                        │
    ├─ prediction made ──────────────────────────────────────┤
    │  features known                                        │  outcome known
    │  prediction known                                      │  label attached
    ▼                                                        ▼
   ┌──────────────────────────────┐               ┌────────────────┐
   │  feature_*    prediction_*   │  ───────────▶ │    label_*     │
   └──────────────────────────────┘               └────────────────┘
                    matched up by prediction ID

A label is sent as a separate event carrying the same prediction ID, and lands on the row the original prediction created. Three consequences follow:

• Recent time buckets legitimately have less ground truth than older ones.
• Any metric needing a label — accuracy, precision, target drift — is only meaningful for periods where enough ground truth has arrived.
• "How much of my ground truth has arrived?" is itself worth monitoring.

Sending ground truth¶

Ground truth is sent as a set of named values:

mpm.log_label(prediction_id="abc-123", labels={"response": 1})

Each name becomes its own column, prefixed with label_:

   label_response
   ──────────────
        1

You can send several at once, and each becomes a separate column you can chart independently:

mpm.log_label(prediction_id="abc-123", labels={"response": 1, "amount_recovered": 240.0})

Referring to ground truth in a metric¶

Use the full column name, prefix included:

SELECT ACCURACY("label_response", "prediction_class") AS accuracy
FROM MODEL

Monitoring the target column¶

Ground truth can be charted like any other column:

All four drift algorithms — PSI, KL, EMD and KS — work on the target exactly as on any other column. See Drift.

If what you log is a probability rather than a class, the ground truth is also what ROC AUC, KS, Gini and the gains table are measured against — see Classification performance.

Why target drift matters for rare outcomes¶

If you are predicting something rare — fraud, default, response — the base rate is the number everything else depends on.

Suppose the true positive rate in your population moves from 2% to 4%. Nothing about your model has changed, and yet:

• precision will usually rise, because there are genuinely more positives to find;
• recall may fall, if the extra positives look unlike the ones the model learned from;
• accuracy may barely move, because 96% of cases are still negative.

Reading those three numbers without knowing the base rate moved leads to conclusions about your model that are really conclusions about the world:

   accuracy  =  base_rate * true_positive_rate  +  (1 - base_rate) * true_negative_rate
                    ^
                    when this moves, accuracy moves too —
                    even if the model's behaviour is completely unchanged

Drift on the target column is what tells the two apart.

Missing values as an operational check¶

A missing-values chart on the target column reads your label pipeline directly. Because labels arrive late, the expected shape declines toward the present:

   % of predictions with ground truth attached

   100% ┤ ████  ████  ████  ████  ███▌
        │                            ██▌
    50% ┤                              ██▌
        │                                 ▉
      0 ┼──────────────────────────────────────▶
        older                          recent
                                          ^ normal: these labels have not arrived yet

What is not normal is that curve dropping for older periods too, or never recovering. That is the signature of a label pipeline that has stopped delivering — and without this chart it shows up only indirectly, as performance metrics quietly going flat.

Column reference¶