Dialogue Helpfulness Judge

DialogueHelpfulnessJudge inspects the latest assistant reply in the context of preceding turns. It rewards responses that acknowledge the user’s request, use the available context, and offer actionable guidance.

1
from opik.evaluation.metrics import DialogueHelpfulnessJudge
2
3
turns = """USER: My VPN disconnects every 5 minutes.\nASSISTANT: Try reinstalling the client.\nUSER: I already did.\n"""
4
5
metric = DialogueHelpfulnessJudge()
6
score = metric.score(
7
    input=turns,
8
    output="Can you send logs? I'll escalate to network engineering.",
9
)
10
11
print(score.value)
12
print(score.reason)

Inputs

Configuration

Integrate this judge into regression suites to catch regressions after prompt changes or upgrades to your assistant model.