Utility Metrics

Helper components that adapt or combine other metrics. Use them to stitch existing evaluators together without rewriting orchestration logic.

class opik.evaluation.metrics.AggregatedMetric(name: str, metrics: List[BaseMetric], aggregator: Callable[[List[ScoreResult]], ScoreResult], track: bool = True, project_name: str | None = None)

Bases: BaseMetric, ScoreArgumentsValidator

Combine the output of multiple metrics into a single aggregated ScoreResult.

Each metric in metrics is executed with the provided scoring kwargs, then the aggregator callback decides how to merge the individual results. This is handy for building ensembles such as min/max, weighted averages, or custom pass/fail checks without re-implementing the metrics themselves.

Parameters:

  • name – Display name for the aggregated metric result.

  • metrics – Ordered list of metric instances that should be executed.

  • aggregator – Callable receiving the list of ScoreResult objects and returning the final aggregated ScoreResult.

  • track – Whether to automatically track the metric in Opik. Defaults to True.

  • project_name – Optional tracking project used when no parent context exists.

Example

>>> from opik.evaluation.metrics import AggregatedMetric, Contains, RegexMatch
>>> metrics = [Contains(track=False), RegexMatch(pattern=r"\d+", track=False)]
>>> from opik.evaluation.metrics import score_result
>>> def combine(results):
...     score = sum(result.value for result in results) / len(results)
...     return score_result.ScoreResult(
...         name="combined_contains_regex",
...         value=score,
...         reason="Average of contains and regex checks",
...     )
>>> metric = AggregatedMetric(
...     name="combined_contains_regex",
...     metrics=metrics,
...     aggregator=combine,
... )
>>> response = "Order number 12345 confirmed"
>>> result = metric.score(output=response, reference="order")
>>> float(result.value)
1.0
__init__(name: str, metrics: List[BaseMetric], aggregator: Callable[[List[ScoreResult]], ScoreResult], track: bool = True, project_name: str | None = None)
score(**kwargs: Any) → ScoreResult

Public method that can be called independently.

validate_score_arguments(score_kwargs: Dict[str, Any], key_mapping: Dict[str, str | Callable[[Dict[str, Any]], Any]] | None) → None

The subclasses must implement this method to provide actual validation logic.

class opik.evaluation.metrics.RagasMetricWrapper(ragas_metric: ragas_metrics.SingleTurnMetric, track: bool = True, project_name: str | None = None)

Bases: BaseMetric

__init__(ragas_metric: ragas_metrics.SingleTurnMetric, track: bool = True, project_name: str | None = None)
score(**kwargs: Any) → ScoreResult

Public method that can be called independently.