Custom guardrail

class opik.guardrails.guards.custom_guardrail.CustomGuardrail(model_name: str, threshold: float = 0.5)

Bases: Guard

Guard backed by a custom binary classifier that you trained on your own labeled examples.

The model is served by the guardrails backend, which loads it by name from its local adapters directory, so the guardrails server (with the trained model available to it) must be running.

get_validation_configs() → List[Dict[str, Any]]

Get the validation configuration for the custom guardrail.

Returns:

List containing the custom guardrail validation configuration

Creating a custom guardrail

opik.guardrails.create_custom_guardrail(name: str, description: str, examples: List[Dict[str, Any]], base_model: str = 'Qwen/Qwen2.5-1.5B-Instruct', epochs: float = 3.0, overwrite: bool = False, wait: bool = True, poll_interval: float = 10.0, timeout: float = 3600.0, callback: Callable[[Dict[str, Any]], None] | None = None) → Dict[str, Any]

Train a custom binary guardrail on the guardrails server and make it available to the CustomGuardrail guard.

Parameters:

  • name – Name of the model, used later to reference the guardrail.

  • description – Natural-language metric, completing “Determine whether it …”, for example “contains toxic or abusive language”.

  • examples – Labeled examples as {"text": ..., "label": 0 or 1}, where 1 means the metric holds (the guardrail should fail).

  • base_model – Base model to fine-tune the adapter on.

  • epochs – Number of training epochs.

  • overwrite – If True, retrain and replace an existing guardrail with this name. If False (default), a name that already exists is rejected.

  • wait – If True, block until training completes and return the final status.

  • poll_interval – Seconds between status checks while waiting.

  • timeout – Maximum seconds to wait for training to complete.

  • callback – Optional function called once per poll with the current status, which carries a progress dict (percent, epoch, train_loss, latest_eval) while training. Only used when wait is True.

Returns:

The training status. When wait is True this includes the eval metrics.

Raises:

opik.exceptions.GuardrailTrainingError – If training fails or does not complete within timeout.