# Documentation ## Docs - [Home](https://opik.docs.buildwithfern.com/docs/opik.mdx) - [Quickstart](https://opik.docs.buildwithfern.com/docs/opik/quickstart.mdx) - [Roadmap](https://opik.docs.buildwithfern.com/docs/opik/roadmap.mdx) - [FAQ](https://opik.docs.buildwithfern.com/docs/opik/faq.mdx) - [February 10, 2025](https://opik.docs.buildwithfern.com/docs/opik/changelog/2025/2/10.mdx) - [February 3, 2025](https://opik.docs.buildwithfern.com/docs/opik/changelog/2025/2/3.mdx) - [January 27, 2025](https://opik.docs.buildwithfern.com/docs/opik/changelog/2025/1/27.mdx) - [January 20, 2025](https://opik.docs.buildwithfern.com/docs/opik/changelog/2025/1/20.mdx) - [January 13, 2025](https://opik.docs.buildwithfern.com/docs/opik/changelog/2025/1/13.mdx) - [January 6, 2025](https://opik.docs.buildwithfern.com/docs/opik/changelog/2025/1/6.mdx) - [December 30, 2024](https://opik.docs.buildwithfern.com/docs/opik/changelog/2024/12/30.mdx) - [December 23, 2024](https://opik.docs.buildwithfern.com/docs/opik/changelog/2024/12/23.mdx) - [December 16, 2024](https://opik.docs.buildwithfern.com/docs/opik/changelog/2024/12/16.mdx) - [December 9, 2024](https://opik.docs.buildwithfern.com/docs/opik/changelog/2024/12/9.mdx) - [December 2, 2024](https://opik.docs.buildwithfern.com/docs/opik/changelog/2024/12/2.mdx) - [November 25, 2024](https://opik.docs.buildwithfern.com/docs/opik/changelog/2024/11/25.mdx) - [November 18, 2024](https://opik.docs.buildwithfern.com/docs/opik/changelog/2024/11/18.mdx) - [November 11, 2024](https://opik.docs.buildwithfern.com/docs/opik/changelog/2024/11/11.mdx) - [November 4, 2024](https://opik.docs.buildwithfern.com/docs/opik/changelog/2024/11/4.mdx) - [October 21, 2024](https://opik.docs.buildwithfern.com/docs/opik/changelog/2024/10/21.mdx) - [October 18, 2024](https://opik.docs.buildwithfern.com/docs/opik/changelog/2024/10/18.mdx) - [October 14, 2024](https://opik.docs.buildwithfern.com/docs/opik/changelog/2024/10/14.mdx) - [October 7, 2024](https://opik.docs.buildwithfern.com/docs/opik/changelog/2024/10/7.mdx) - [September 30, 2024](https://opik.docs.buildwithfern.com/docs/opik/changelog/2024/9/30.mdx) - [Log traces](https://opik.docs.buildwithfern.com/docs/opik/tracing/log_traces.mdx): Describes how to log LLM calls to the Opik platform using function decorators, integrations or the low level client. - [Log agents](https://opik.docs.buildwithfern.com/docs/opik/tracing/log_agents.mdx): Describes how to track agents using Opik - [Log multimodal traces](https://opik.docs.buildwithfern.com/docs/opik/tracing/log_multimodal_traces.mdx): Describes how to log and view images in traces to the Opik platform - [Log distributed traces](https://opik.docs.buildwithfern.com/docs/opik/tracing/log_distributed_traces.mdx): Describes how to log distributed traces to the Opik platform - [Annotate traces](https://opik.docs.buildwithfern.com/docs/opik/tracing/annotate_traces.mdx): Describes how to annotate traces using the Opik SDK and UI - [Cost tracking](https://opik.docs.buildwithfern.com/docs/opik/tracing/cost_tracking.mdx): Describes how to track and monitor costs for your LLM applications using Opik - [SDK configuration](https://opik.docs.buildwithfern.com/docs/opik/tracing/sdk_configuration.mdx): Describes how to configure the Python SDK - [Export data](https://opik.docs.buildwithfern.com/docs/opik/tracing/export_data.mdx): Describes how to export traces and spans from the Opik platform. - [Overview](https://opik.docs.buildwithfern.com/docs/opik/tracing/integrations/overview.mdx): Describes all the integrations provided by Opik and what each framework can be used for - [OpenAI](https://opik.docs.buildwithfern.com/docs/opik/tracing/integrations/openai.mdx): Describes how to track OpenAI LLM calls using Opik - [OpenRouter](https://opik.docs.buildwithfern.com/docs/opik/tracing/integrations/openrouter.mdx): Describes how to track OpenRouter LLM calls using Opik - [LiteLLM](https://opik.docs.buildwithfern.com/docs/opik/tracing/integrations/litellm.mdx): Describes how to track LiteLLM LLM calls using Opik - [Langchain](https://opik.docs.buildwithfern.com/docs/opik/tracing/integrations/langchain.mdx): Describes how to use Opik with LangChain - [AIsuite](https://opik.docs.buildwithfern.com/docs/opik/tracing/integrations/aisuite.mdx): Describes how to track aisuite LLM calls using Opik - [Anthropic](https://opik.docs.buildwithfern.com/docs/opik/tracing/integrations/anthropic.mdx): Describes how to track Anthropic LLM calls using Opik - [Bedrock](https://opik.docs.buildwithfern.com/docs/opik/tracing/integrations/bedrock.mdx): Describes how to track Bedrock LLM calls using Opik - [Crewai](https://opik.docs.buildwithfern.com/docs/opik/tracing/integrations/crewai.mdx): Describes how to track CrewAI calls using Opik - [Dify](https://opik.docs.buildwithfern.com/docs/opik/tracing/integrations/dify.mdx): Describes how to use Opik with Dify - [DSPY](https://opik.docs.buildwithfern.com/docs/opik/tracing/integrations/dspy.mdx): Describes how to track DSPy calls using Opik - [Gemini](https://opik.docs.buildwithfern.com/docs/opik/tracing/integrations/gemini.mdx): Describes how to track Gemini LLM calls using Opik - [Groq](https://opik.docs.buildwithfern.com/docs/opik/tracing/integrations/groq.mdx): Describes how to track Groq LLM calls using Opik - [Guardrails AI](https://opik.docs.buildwithfern.com/docs/opik/tracing/integrations/guardrails-ai.mdx): Cookbook that showcases Opik's integration with the Guardrails AI Python SDK - [Haystack](https://opik.docs.buildwithfern.com/docs/opik/tracing/integrations/haystack.mdx): Describes how to track Haystack pipeline runs using Opik - [LangGraph](https://opik.docs.buildwithfern.com/docs/opik/tracing/integrations/langgraph.mdx): Describes how to track LangGraph Agent executions using Opik - [Llama Index](https://opik.docs.buildwithfern.com/docs/opik/tracing/integrations/llama_indec.mdx): Describes how to track LlamaIndex pipelines using Opik - [Ollama](https://opik.docs.buildwithfern.com/docs/opik/tracing/integrations/ollama.mdx): Describes how to track Ollama LLM calls using Opik - [Predibase](https://opik.docs.buildwithfern.com/docs/opik/tracing/integrations/predibase.mdx): Describes how to track Predibase LLM calls using Opik - [Ragas](https://opik.docs.buildwithfern.com/docs/opik/tracing/integrations/ragas.mdx): Describes how to log Ragas scores to the Opik platform - [WatsonX](https://opik.docs.buildwithfern.com/docs/opik/tracing/integrations/watsonx.mdx): Describes how to track watsonx LLM calls using Opik - [Overview](https://opik.docs.buildwithfern.com/docs/opik/evaluation/overview.mdx): A high-level overview on how to use Opik's evaluation features including some code snippets - [Concepts](https://opik.docs.buildwithfern.com/docs/opik/evaluation/concepts.mdx): Introduces the concepts behind Opik's evaluation framework - [Evaluate prompts](https://opik.docs.buildwithfern.com/docs/opik/evaluation/evaluate_prompt.mdx): Step by step guide on how to evaluate LLM prompts - [Evaluate your LLM application](https://opik.docs.buildwithfern.com/docs/opik/evaluation/evaluate_your_llm.mdx): Step by step guide on how to evaluate your LLM application - [Update existing experiment](https://opik.docs.buildwithfern.com/docs/opik/evaluation/update_existing_experiment.mdx): Guides you through the process of updating an existing experiment - [Manage datasets](https://opik.docs.buildwithfern.com/docs/opik/evaluation/manage_datasets.mdx): Guides you through the process of creating and managing datasets - [Overview](https://opik.docs.buildwithfern.com/docs/opik/evaluation/metrics/overview.mdx): Describes all the built-in evaluation metrics provided by Opik - [Heuristic metrics](https://opik.docs.buildwithfern.com/docs/opik/evaluation/metrics/heuristic_metrics.mdx): Describes all the built-in heuristic metrics provided by Opik - [Hallucination](https://opik.docs.buildwithfern.com/docs/opik/evaluation/metrics/hallucination.mdx): Describes the Hallucination metric - [G-Eval](https://opik.docs.buildwithfern.com/docs/opik/evaluation/metrics/g_eval.mdx): Describes Opik's built-in G-Eval metric which is is a task agnostic LLM as a Judge metric - [Moderation](https://opik.docs.buildwithfern.com/docs/opik/evaluation/metrics/moderation.mdx): Describes the Moderation metric - [Usefulness](https://opik.docs.buildwithfern.com/docs/opik/evaluation/metrics/usefulness.mdx): Describes the Usefulness metric - [Answer relevance](https://opik.docs.buildwithfern.com/docs/opik/evaluation/metrics/answer_relevance.mdx): Describes the Answer Relevance metric - [Context precision](https://opik.docs.buildwithfern.com/docs/opik/evaluation/metrics/context_precision.mdx): Describes the Context Precision metric - [Context recall](https://opik.docs.buildwithfern.com/docs/opik/evaluation/metrics/context_recall.mdx): Describes the Context Recall metric - [Custom model](https://opik.docs.buildwithfern.com/docs/opik/evaluation/metrics/custom_model.mdx): Describes how to use a custom model for Opik's built-in LLM as a Judge metrics - [Custom metric](https://opik.docs.buildwithfern.com/docs/opik/evaluation/metrics/custom_metric.mdx): Describes how to create your own metric to use with Opik's evaluation framework - [Prompt management](https://opik.docs.buildwithfern.com/docs/opik/prompt_engineering/prompt_management.mdx) - [Managing prompts in code](https://opik.docs.buildwithfern.com/docs/opik/prompt_engineering/managing_prompts_in_code.mdx) - [Playground](https://opik.docs.buildwithfern.com/docs/opik/prompt_engineering/playground.mdx) - [Pytest integration](https://opik.docs.buildwithfern.com/docs/opik/testing/pytest_integration.mdx): Describes how to use Opik with Pytest to write LLM unit tests - [Production monitoring](https://opik.docs.buildwithfern.com/docs/opik/production/production_monitoring.mdx): Describes how to monitor your LLM applications in production using Opik - [Rules](https://opik.docs.buildwithfern.com/docs/opik/production/rules.mdx): Describes how to define scoring rules for production traces - [Gateway](https://opik.docs.buildwithfern.com/docs/opik/production/gateway.mdx): Describes how to use the Opik LLM gateway and how to integrate with the Kong AI Gateway - [Overview](https://opik.docs.buildwithfern.com/docs/opik/self-host/overview.mdx): High-level overview on how to self-host Opik - [Local deployment](https://opik.docs.buildwithfern.com/docs/opik/self-host/local_deployment.mdx): Describes how to run Opik locally using Docker Compose - [Kubernetes deployment](https://opik.docs.buildwithfern.com/docs/opik/self-host/kubernetes.mdx): Describes how to run Opik on a Kubernetes cluster - [Platform Architecture](https://opik.docs.buildwithfern.com/docs/opik/self-host/architecture.mdx): High-level overview on Opik's Platform Architecture - [Anonymous usage statistics](https://opik.docs.buildwithfern.com/docs/opik/self-host/configure/anonymous_usage_statistics.mdx): Describes the usage statistics that are collected by Opik - [Opik Cookbooks](https://opik.docs.buildwithfern.com/docs/opik/cookbook/overview.mdx): An open-source collection of notebooks and guides for using the Opik platform. - [Quickstart notebook](https://opik.docs.buildwithfern.com/docs/opik/cookbook/quickstart_notebook.mdx): Quickstart cookbook that showcases Opik's evaluation, tracing and prompt management functionality. - [OpenAI](https://opik.docs.buildwithfern.com/docs/opik/cookbook/openai.mdx): Cookbook that showcases Opik's integration with the OpenAI Python SDK - [LiteLLM](https://opik.docs.buildwithfern.com/docs/opik/cookbook/litellm.mdx): Cookbook that showcases Opik's integration with the LiteLLM Python SDK - [Langchain](https://opik.docs.buildwithfern.com/docs/opik/cookbook/langchain.mdx): Cookbook that showcases Opik's integration with the LangChain Python SDK - [aisuite](https://opik.docs.buildwithfern.com/docs/opik/cookbook/aisuite.mdx): Cookbook that showcases Opik's integration with the aisuite Python SDK - [Anthropic](https://opik.docs.buildwithfern.com/docs/opik/cookbook/anthropic.mdx): Cookbook that showcases Opik's integration with the Anthropic Python SDK - [Bedrock](https://opik.docs.buildwithfern.com/docs/opik/cookbook/bedrock.mdx): Cookbook that showcases Opik's integration with AWS Bedrock - [Crewai](https://opik.docs.buildwithfern.com/docs/opik/cookbook/crewai.mdx): Cookbook that showcases Opik's integration with the CrewAI - [DSPY](https://opik.docs.buildwithfern.com/docs/opik/cookbook/dspy.mdx): Cookbook that showcases Opik's integration with DSPy - [Gemini](https://opik.docs.buildwithfern.com/docs/opik/cookbook/gemini.mdx): Cookbook that showcases Opik's integration with the Gemini Python SDK - [Groq](https://opik.docs.buildwithfern.com/docs/opik/cookbook/groq.mdx): Cookbook that showcases Opik's integration with Groq - [Guardrails AI](https://opik.docs.buildwithfern.com/docs/opik/cookbook/guardrails-ai.mdx): Cookbook that showcases Opik's integration with the Guardrails AI Python SDK - [Haystack](https://opik.docs.buildwithfern.com/docs/opik/cookbook/haystack.mdx): Cookbook that showcases Opik's integration with Haystack - [LangGraph](https://opik.docs.buildwithfern.com/docs/opik/cookbook/langgraph.mdx): Cookbook that showcases Opik's integration with the LangGraph Python SDK - [Llama Index](https://opik.docs.buildwithfern.com/docs/opik/cookbook/llama-index.mdx): Cookbook that showcases Opik's integration with the LlamaIndex Python SDK - [Ollama](https://opik.docs.buildwithfern.com/docs/opik/cookbook/ollama.mdx): Cookbook that showcases Opik's integration with the Ollama Python SDK - [Predibase](https://opik.docs.buildwithfern.com/docs/opik/cookbook/predibase.mdx): Cookbook that showcases Opik's integration with Predibase - [Ragas](https://opik.docs.buildwithfern.com/docs/opik/cookbook/ragas.mdx): Cookbook that showcases Opik's integration with the Ragas Python SDK - [WatsonX](https://opik.docs.buildwithfern.com/docs/opik/cookbook/watsonx.mdx): Cookbook that showcases Opik's integration with Watsonx through the LiteLLM Python SDK - [Evaluate hallucination metric](https://opik.docs.buildwithfern.com/docs/opik/cookbook/evaluate_hallucination_metric.mdx): Cookbook that evalutes Opik's Hallucination Metric, showcasing both how to use the `evaluation` functionality in the platform as well as the quality of the Hallucination metric included in the SDK. It is a complex example that doesn't always align with how the `evaluate` function works. - [Evaluate moderation metric](https://opik.docs.buildwithfern.com/docs/opik/cookbook/evaluate_moderation_metric.mdx): Cookbook that evalutes Opik's Moderation Metric, showcasing both how to use the `evaluation` functionality in the platform as well as the quality of the Modelration metric included in the SDK. It is a complex example that doesn't always align with how the `evaluate` function works. - [Overview](https://opik.docs.buildwithfern.com/docs/opik/reference/overview.mdx) - [Overview](https://opik.docs.buildwithfern.com/docs/opik/reference/typescript-sdk/overview.mdx) - [Overview](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/overview.mdx) ## API Docs - API Reference > System usage [Get datasets information for BI events](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/system-usage/get-dataset-bi-info.mdx) - API Reference > System usage [Get experiments information for BI events](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/system-usage/get-experiment-bi-info.mdx) - API Reference > System usage [Get traces information for BI events](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/system-usage/get-traces-bi-info.mdx) - API Reference > System usage [Get traces count on previous day for all available workspaces](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/system-usage/get-traces-count-for-workspaces.mdx) - API Reference > Check [Check user access to workspace](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/check/access.mdx) - API Reference > Automation rule evaluators [Find project Evaluators](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/automation-rule-evaluators/find-evaluators.mdx) - API Reference > Automation rule evaluators [Create automation rule evaluator](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/automation-rule-evaluators/create-automation-rule-evaluator.mdx) - API Reference > Automation rule evaluators [Delete automation rule evaluators](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/automation-rule-evaluators/delete-automation-rule-evaluator-batch.mdx) - API Reference > Automation rule evaluators [Get automation rule evaluator by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/automation-rule-evaluators/get-evaluator-by-id.mdx) - API Reference > Automation rule evaluators [update Automation Rule Evaluator by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/automation-rule-evaluators/update-automation-rule-evaluator.mdx) - API Reference > Automation rule evaluators [Get automation rule evaluator logs by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/automation-rule-evaluators/get-evaluator-logs-by-id.mdx) - API Reference > Automation rule evaluators [Find project Evaluators Deprecated](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/automation-rule-evaluators/find-evaluators-deprecated.mdx) - API Reference > Automation rule evaluators [Create automation rule evaluator Deprecated](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/automation-rule-evaluators/create-automation-rule-evaluator-deprecated.mdx) - API Reference > Automation rule evaluators [Delete automation rule evaluators Deprecated](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/automation-rule-evaluators/delete-automation-rule-evaluator-batch-deprecated.mdx) - API Reference > Automation rule evaluators [Get automation rule evaluator by id Deprecated](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/automation-rule-evaluators/get-evaluator-by-id-deprecated.mdx) - API Reference > Automation rule evaluators [update Automation Rule Evaluator by id Deprecated](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/automation-rule-evaluators/update-automation-rule-evaluator-deprecated.mdx) - API Reference > Automation rule evaluators [Get automation rule evaluator logs by id Deprecated](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/automation-rule-evaluators/get-evaluator-logs-by-id-deprecated.mdx) - API Reference > Chat Completions [Create chat completions](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/chat-completions/create-chat-completions.mdx) - API Reference > Datasets [Find datasets](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/datasets/find-datasets.mdx) - API Reference > Datasets [Create dataset](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/datasets/create-dataset.mdx) - API Reference > Datasets [Create/update dataset items](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/datasets/create-or-update-dataset-items.mdx) - API Reference > Datasets [Get dataset by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/datasets/get-dataset-by-id.mdx) - API Reference > Datasets [Update dataset by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/datasets/update-dataset.mdx) - API Reference > Datasets [Delete dataset by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/datasets/delete-dataset.mdx) - API Reference > Datasets [Delete dataset by name](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/datasets/delete-dataset-by-name.mdx) - API Reference > Datasets [Delete dataset items](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/datasets/delete-dataset-items.mdx) - API Reference > Datasets [Delete datasets](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/datasets/delete-datasets-batch.mdx) - API Reference > Datasets [Find dataset items with experiment items](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/datasets/find-dataset-items-with-experiment-items.mdx) - API Reference > Datasets [Get dataset by name](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/datasets/get-dataset-by-identifier.mdx) - API Reference > Datasets [Get dataset item by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/datasets/get-dataset-item-by-id.mdx) - API Reference > Datasets [Get dataset items](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/datasets/get-dataset-items.mdx) - API Reference > Datasets [Get dataset items output columns](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/datasets/get-dataset-items-output-columns.mdx) - API Reference > Datasets [Stream dataset items](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/datasets/stream-dataset-items.mdx) - API Reference > Experiments [Find experiments](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/experiments/find-experiments.mdx) - API Reference > Experiments [Create experiment](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/experiments/create-experiment.mdx) - API Reference > Experiments [Create experiment items](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/experiments/create-experiment-items.mdx) - API Reference > Experiments [Delete experiment items](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/experiments/delete-experiment-items.mdx) - API Reference > Experiments [Delete experiments by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/experiments/delete-experiments-by-id.mdx) - API Reference > Experiments [Find Feedback Score names](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/experiments/find-feedback-score-names.mdx) - API Reference > Experiments [Get experiment by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/experiments/get-experiment-by-id.mdx) - API Reference > Experiments [Get experiment by name](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/experiments/get-experiment-by-name.mdx) - API Reference > Experiments [Get experiment item by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/experiments/get-experiment-item-by-id.mdx) - API Reference > Experiments [Stream experiment items](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/experiments/stream-experiment-items.mdx) - API Reference > Feedback-definitions [Find Feedback definitions](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/feedback-definitions/find-feedback-definitions.mdx) - API Reference > Feedback-definitions [Create feedback definition](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/feedback-definitions/create-feedback-definition.mdx) - API Reference > Feedback-definitions [Get feedback definition by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/feedback-definitions/get-feedback-definition-by-id.mdx) - API Reference > Feedback-definitions [Update feedback definition by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/feedback-definitions/update-feedback-definition.mdx) - API Reference > Feedback-definitions [Delete feedback definition by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/feedback-definitions/delete-feedback-definition-by-id.mdx) - API Reference > Feedback-definitions [Delete feedback definitions](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/feedback-definitions/delete-feedback-definitions-batch.mdx) - API Reference > LlmProviderKey [Delete LLM Provider's ApiKeys](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/llm-provider-key/delete-llm-provider-api-keys-batch.mdx) - API Reference > LlmProviderKey [Find LLM Provider's ApiKeys](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/llm-provider-key/find-llm-provider-keys.mdx) - API Reference > LlmProviderKey [Store LLM Provider's ApiKey](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/llm-provider-key/store-llm-provider-api-key.mdx) - API Reference > LlmProviderKey [Get LLM Provider's ApiKey by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/llm-provider-key/get-llm-provider-api-key-by-id.mdx) - API Reference > LlmProviderKey [Update LLM Provider's ApiKey](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/llm-provider-key/update-llm-provider-api-key.mdx) - API Reference > Projects [Find projects](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/projects/find-projects.mdx) - API Reference > Projects [Create project](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/projects/create-project.mdx) - API Reference > Projects [Get project by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/projects/get-project-by-id.mdx) - API Reference > Projects [Delete project by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/projects/delete-project-by-id.mdx) - API Reference > Projects [Update project by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/projects/update-project.mdx) - API Reference > Projects [Delete projects](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/projects/delete-projects-batch.mdx) - API Reference > Projects [Find Feedback Score names By Project Ids](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/projects/find-feedback-score-names-by-project-ids.mdx) - API Reference > Projects [Get Project Metrics](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/projects/get-project-metrics.mdx) - API Reference > Projects [Get Project Stats](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/projects/get-project-stats.mdx) - API Reference > Projects [Retrieve project](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/projects/retrieve-project.mdx) - API Reference > Prompts [Get prompts](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/prompts/get-prompts.mdx) - API Reference > Prompts [Create prompt](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/prompts/create-prompt.mdx) - API Reference > Prompts [Create prompt version](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/prompts/create-prompt-version.mdx) - API Reference > Prompts [Get prompt by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/prompts/get-prompt-by-id.mdx) - API Reference > Prompts [Update prompt](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/prompts/update-prompt.mdx) - API Reference > Prompts [Delete prompt](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/prompts/delete-prompt.mdx) - API Reference > Prompts [Delete prompts](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/prompts/delete-prompts-batch.mdx) - API Reference > Prompts [Get prompt version by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/prompts/get-prompt-version-by-id.mdx) - API Reference > Prompts [Get prompt versions](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/prompts/get-prompt-versions.mdx) - API Reference > Prompts [Retrieve prompt version](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/prompts/retrieve-prompt-version.mdx) - API Reference > Spans [Add span comment](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/spans/add-span-comment.mdx) - API Reference > Spans [Add span feedback score](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/spans/add-span-feedback-score.mdx) - API Reference > Spans [Get spans by project_name or project_id and optionally by trace_id and/or type](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/spans/get-spans-by-project.mdx) - API Reference > Spans [Create span](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/spans/create-span.mdx) - API Reference > Spans [Create spans](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/spans/create-spans.mdx) - API Reference > Spans [Get span by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/spans/get-span-by-id.mdx) - API Reference > Spans [Delete span by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/spans/delete-span-by-id.mdx) - API Reference > Spans [Update span by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/spans/update-span.mdx) - API Reference > Spans [Delete span comments](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/spans/delete-span-comments.mdx) - API Reference > Spans [Delete span feedback score](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/spans/delete-span-feedback-score.mdx) - API Reference > Spans [Find Feedback Score names](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/spans/find-feedback-score-names-1.mdx) - API Reference > Spans [Get span comment](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/spans/get-span-comment.mdx) - API Reference > Spans [Get span stats](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/spans/get-span-stats.mdx) - API Reference > Spans [Batch feedback scoring for spans](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/spans/score-batch-of-spans.mdx) - API Reference > Spans [Search spans](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/spans/search-spans.mdx) - API Reference > Spans [Update span comment by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/spans/update-span-comment.mdx) - API Reference > Traces [Add trace comment](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/traces/add-trace-comment.mdx) - API Reference > Traces [Add trace feedback score](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/traces/add-trace-feedback-score.mdx) - API Reference > Traces [Get traces by project_name or project_id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/traces/get-traces-by-project.mdx) - API Reference > Traces [Create trace](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/traces/create-trace.mdx) - API Reference > Traces [Create traces](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/traces/create-traces.mdx) - API Reference > Traces [Get trace by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/traces/get-trace-by-id.mdx) - API Reference > Traces [Delete trace by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/traces/delete-trace-by-id.mdx) - API Reference > Traces [Update trace by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/traces/update-trace.mdx) - API Reference > Traces [Delete trace comments](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/traces/delete-trace-comments.mdx) - API Reference > Traces [Delete trace feedback score](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/traces/delete-trace-feedback-score.mdx) - API Reference > Traces [Delete traces](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/traces/delete-traces.mdx) - API Reference > Traces [Find Feedback Score names](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/traces/find-feedback-score-names-2.mdx) - API Reference > Traces [Get trace stats](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/traces/get-trace-stats.mdx) - API Reference > Traces [Get trace comment](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/traces/get-trace-comment.mdx) - API Reference > Traces [Batch feedback scoring for traces](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/traces/score-batch-of-traces.mdx) - API Reference > Traces [Update trace comment by id](https://opik.docs.buildwithfern.com/docs/opik/reference/rest-api/traces/update-trace-comment.mdx)