For AI agents: a documentation index is available at the root level at /llms.txt and /llms-full.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.
In Opik 2.0, prompts, datasets, and experiments are project-scoped. Alert events such as prompt changes and feedback scores are now associated with specific projects.
Alerts allow you to configure automated webhook notifications for important events in your Opik workspace. When specific events occur — such as trace errors, new feedback scores, or prompt changes — Opik sends HTTP POST requests to your configured endpoint with detailed event data.
Opik provides three destination types for alerts:
Slack: Native integration with automatic message formatting for Slack
PagerDuty: Native integration with automatic event formatting for PagerDuty
General: For custom webhooks, no-code automation platforms, or middleware services
Creating an alert
Prerequisites
Access to the Opik Configuration page
A webhook endpoint that can receive HTTP POST requests
(Optional) An HTTPS endpoint with valid SSL certificate for production use
Step-by-step guide
Navigate to Alerts
Go to Configuration → Alerts tab
Click “Create new alert” button
Configure basic settings
Name: Give your alert a descriptive name (e.g., “Production Errors Slack”)
Enable alert: Toggle on to activate the alert immediately
Configure webhook settings
Destination: Select the alert destination type:
General: For custom webhooks, no-code automation platforms, or middleware services
Slack: For native Slack webhook integration (automatically formats messages for Slack)
PagerDuty: For native PagerDuty integration (automatically formats events for PagerDuty)
Endpoint URL: Enter your webhook URL (must start with http:// or https://)
For Slack: Use your Slack Incoming Webhook URL (e.g., https://hooks.slack.com/services/...)
For PagerDuty: Use your PagerDuty Events API v2 integration URL (e.g., https://events.pagerduty.com/v2/enqueue)
For General: Use any HTTP endpoint that can receive POST requests
Advanced webhook settings (optional)
Secret token: Add a secret token to verify webhook authenticity (recommended for General destination)
Custom headers: Add HTTP headers for authentication or routing
Example: X-Custom-Auth: Bearer your-token-here
Add triggers
Click “Add trigger” to select event types
Choose one or more event types from the list
Configure project scope for observability events (optional)
For threshold-based alerts (errors, cost, latency, feedback scores):
Threshold: Set the threshold value that triggers the alert
Operator: Choose comparison operator (>, <) for feedback score alerts
Window: Configure the time window in seconds for metric aggregation
Feedback Score Name: Select which feedback score to monitor (for feedback score alerts only)
Test your configuration
Click “Test connection” to send a sample webhook
Verify your endpoint receives the test payload
Check the response status in the Opik UI
Create the alert
Click “Create alert” to save your configuration
The alert will start monitoring events immediately
Integration examples
Opik supports three main approaches for integrating alerts with external systems:
Native integrations (Slack, PagerDuty): Use built-in formatting for popular services - no middleware required
General webhooks: Send alerts to custom endpoints, no-code platforms, or middleware services
Middleware services (Optional): Add custom logic, routing, or transformations before forwarding to destinations
Slack integration (Native)
Opik provides native Slack integration that automatically formats alert messages for Slack’s Block Kit format.
Generate a webhook URL (e.g., https://hooks.slack.com/services/T00000000/B00000000/XXXX)
Setup steps
In Slack:
Create a Slack app in your workspace
Enable Incoming Webhooks
Add the webhook to your desired channel
Copy the webhook URL
In Opik:
Go to Configuration → Alerts tab
Click “Create new alert”
Give your alert a descriptive name
Select Slack as the destination type
Paste your Slack webhook URL in the Endpoint URL field
Add triggers for the events you want to monitor
Click “Test connection” to verify
Click “Create alert”
Opik will automatically format all alert payloads into Slack-compatible messages with rich formatting, including:
Alert name and event type
Event count and details
Relevant metadata
Links to view full details in Opik
PagerDuty integration (Native)
Opik provides native PagerDuty integration that automatically formats alert events for PagerDuty’s Events API v2.
Prerequisites
A PagerDuty account with permission to create integrations
Access to a service where you want to receive alerts
Setup steps
In PagerDuty:
Navigate to Services → select your service → Integrations tab
Click “Add Integration”
Select “Events API V2”
Give the integration a name (e.g., “Opik Alerts”)
Save the integration and copy the Integration Key
In Opik:
Go to Configuration → Alerts tab
Click “Create new alert”
Give your alert a descriptive name
Select PagerDuty as the destination type
Enter the PagerDuty Events API v2 endpoint: https://events.pagerduty.com/v2/enqueue
In the Routing Key field, enter your PagerDuty Integration Key (this field appears when PagerDuty is selected as the destination)
Add triggers for the events you want to monitor
Click “Test connection” to verify
Click “Create alert”
Opik will automatically format all alert payloads into PagerDuty-compatible events with:
Severity levels based on event type
Detailed event information
Custom fields for filtering and routing
Deduplication keys to prevent duplicate incidents
Custom integration with middleware service (Optional)
For more complex integrations or custom formatting requirements, you can use a middleware service to transform Opik’s payload before sending it to your destination. This approach works with any destination type (General, Slack, or PagerDuty).
When to use middleware
Custom message formatting: Transform payload structure or add custom fields
Multi-destination routing: Send alerts to different endpoints based on event type
Additional processing: Enrich alerts with data from other systems
Legacy systems: Adapt Opik alerts to older webhook formats
Example middleware for Slack with custom formatting
"text": f"*{event_count}* new `{event_type}` events"
23
}
24
},
25
{
26
"type": "section",
27
"text": {
28
"type": "mrkdwn",
29
"text": f"View in Opik: https://www.comet.com/opik"
30
}
31
},
32
{
33
"type": "section",
34
"fields": [
35
{
36
"type": "mrkdwn",
37
"text": f"*Environment:*\nProduction"
38
},
39
{
40
"type": "mrkdwn",
41
"text": f"*Priority:*\nHigh"
42
}
43
]
44
}
45
]
46
}
47
48
@app.route('/opik-to-slack', methods=['POST'])
49
def opik_to_slack():
50
opik_data = request.json
51
slack_payload = transform_to_slack(opik_data)
52
53
# Forward to Slack
54
requests.post(
55
SLACK_WEBHOOK_URL,
56
json=slack_payload
57
)
58
59
return {'status': 'success'}, 200
Setup for middleware approach
Deploy your middleware service to a publicly accessible endpoint
In Opik, create an alert with destination type General
Use your middleware service URL as the Endpoint URL
Configure your middleware to forward to the final destination (Slack, PagerDuty, etc.)
Using no-code automation platforms
No-code automation tools like n8n, Make.com, and IFTTT provide an easy way to connect Opik alerts to other services—without writing or deploying code. These platforms can receive webhooks from Opik, apply filters or conditions, and trigger actions such as sending Slack messages, logging data in Google Sheets, or creating incidents in PagerDuty.
To use them:
Create a new workflow or scenario and add a Webhook trigger node/module
Copy the webhook URL generated by the platform
In Opik, create an alert with destination type General and paste the webhook URL from your automation platform
Secure the connection by validating the Authorization header or including a secret token parameter
Add filters or routing logic to handle different eventType values from Opik (for example, trace:errors or trace:feedback_score)
Chain the desired actions, such as notifications, database updates, or analytics tracking
These tools also provide built-in monitoring, retries, and visual flow editors, making them suitable for both technical and non-technical users who want to automate Opik alert handling securely and efficiently. This approach works well when you need to route alerts to multiple destinations or apply complex business logic.
Custom dashboard integration
Build a custom monitoring dashboard that receives alerts using the General destination type:
1
from fastapi import FastAPI, Request
2
from datetime import datetime
3
4
app = FastAPI()
5
6
# In-memory storage (use a database in production)
7
alert_history = []
8
9
@app.post("/webhook")
10
async def receive_webhook(request: Request):
11
data = await request.json()
12
13
# Store alert
14
alert_history.append({
15
'timestamp': datetime.utcnow(),
16
'event_type': data.get('eventType'),
17
'alert_name': data['payload']['alertName'],
18
'event_count': data['payload']['eventCount'],
19
'data': data
20
})
21
22
# Keep only last 1000 alerts
23
if len(alert_history) > 1000:
24
alert_history.pop(0)
25
26
return {"status": "success"}
27
28
@app.get("/dashboard")
29
async def get_dashboard():
30
# Return aggregated statistics
31
return {
32
'total_alerts': len(alert_history),
33
'by_type': group_by_type(alert_history),
34
'recent_alerts': alert_history[-10:]
35
}
Supported event types
Opik supports ten types of alert events:
Observability events
Trace errors threshold exceeded
Event type: trace:errors
Triggered when: Total trace error count exceeds the specified threshold within a time window
Project scope: Can be configured to specific projects
Configuration: Requires threshold value (error count) and time window (in seconds)
Payload: Metrics alert payload with error count details
Use case: Proactive error monitoring, detect error spikes, prevent system degradation
Trace feedback score threshold exceeded
Event type: trace:feedback_score
Triggered when: Average trace feedback score meets the specified threshold criteria within a time window
Project scope: Can be configured to specific projects
Configuration: Requires feedback score name, threshold value, operator (>, <), and time window
Payload: Metrics alert payload with average feedback score details
Use case: Track model performance, monitor user satisfaction, detect quality degradation
Thread feedback score threshold exceeded
Event type: trace_thread:feedback_score
Triggered when: Average thread feedback score meets the specified threshold criteria within a time window
Project scope: Can be configured to specific projects
Configuration: Requires feedback score name, threshold value, operator (>, <), and time window
Payload: Metrics alert payload with average feedback score details
Opik retries failed webhooks with exponential backoff
Ensure your endpoint returns 2xx status codes on success
Implement idempotency using the webhook id field
Example idempotent handler:
1
processed_webhook_ids = set()
2
3
@app.route('/webhook', methods=['POST'])
4
def handle_webhook():
5
data = request.json
6
webhook_id = data.get('id')
7
8
# Skip if already processed
9
if webhook_id in processed_webhook_ids:
10
return {'status': 'already_processed'}, 200
11
12
# Process webhook
13
process_alert(data)
14
15
# Mark as processed
16
processed_webhook_ids.add(webhook_id)
17
18
return {'status': 'success'}, 200
Events not triggering alerts
Check event type matching:
Verify the alert has a trigger for this event type
For observability events, check project scope configuration
Review project IDs in trigger configuration
Check workspace context:
Ensure events are logged to the correct workspace
Verify the alert is in the same workspace as your events
Check alert evaluation:
View backend logs for alert evaluation messages
Confirm events are being published to the event bus
Check Redis for alert buckets (self-hosted deployments)
SSL certificate errors
If you see SSL certificate errors in logs:
For development/testing:
Use self-signed certificates with proper configuration
Or use HTTP endpoints (not recommended for production)
For production:
Use valid SSL certificates from trusted CAs
Ensure certificate chain is complete
Check certificate expiry dates
Use services like Let’s Encrypt for free SSL
Architecture and internals
Understanding Opik’s alert architecture can help with troubleshooting and optimization.
How alerts work
The Opik Alerts system monitors your workspace for specific events and sends consolidated webhook notifications to your configured endpoints. Here’s the flow:
Event occurs: An event happens in your workspace (e.g., a trace error, prompt creation, guardrail trigger, new feedback score)
Alert evaluation: The system checks if any enabled alerts match this event type and evaluates threshold conditions (for metrics-based alerts like errors, cost, latency, and feedback scores)
Event aggregation: Multiple events are aggregated over a short time window (debouncing)
Webhook delivery: A consolidated HTTP POST request is sent to your webhook URL
Retry handling: Failed requests are automatically retried with exponential backoff
Event debouncing
To prevent overwhelming your webhook endpoint, Opik aggregates multiple events of the same type within a short time window (typically 30-60 seconds) and sends them as a single consolidated webhook. This is particularly useful for high-frequency events like feedback scores.
Event flow
1. Event occurs (e.g., trace error logged)
↓
2. Service publishes AlertEvent to EventBus
↓
3. AlertEventListener receives event
↓
4. AlertEventEvaluationService evaluates against configured alerts
↓
5. Matching events added to AlertBucketService (Redis)
↓
6. AlertJob (runs every 5 seconds) processes ready buckets
↓
7. WebhookPublisher publishes to Redis stream
↓
8. WebhookSubscriber consumes from stream
↓
9. WebhookHttpClient sends HTTP POST request
↓
10. Retries on failure with exponential backoff
Debouncing mechanism
Opik uses Redis-based buckets to aggregate events:
