{"id":13663,"date":"2025-05-23T08:12:15","date_gmt":"2025-05-23T16:12:15","guid":{"rendered":"https:\/\/live-cometml.pantheonsite.io\/?p=13663"},"modified":"2025-06-05T11:47:20","modified_gmt":"2025-06-05T11:47:20","slug":"opik-guardrails-release","status":"publish","type":"post","link":"https:\/\/www.comet.com\/site\/blog\/opik-guardrails-release\/","title":{"rendered":"Announcing Opik\u2019s Guardrails Beta: Moderate LLM Applications in Real-Time"},"content":{"rendered":"\n

We\u2019ve spent the past year building tools that make LLM applications more transparent, measurable, and accountable. Since launching Opik, our open source evaluation platform, we\u2019ve shipped dozens of features focused on monitoring, evaluating, and optimizing LLM applications. This includes heuristic and LLM-as-a-Judge metrics<\/a>, realtime tracing and monitoring<\/a>, prompt management capabilities<\/a>, development playgrounds<\/a>, automatic prompt optimization<\/a>, and so much more.<\/p>\n\n\n\n

Today, we\u2019re excited to take a new step in our mission: a public beta of our Guardrails<\/strong> project\u2014a lightweight system designed to help developers build safer, more robust LLM applications.<\/p>\n\n\n\n

Why Guardrails?<\/h2>\n\n\n\n

As language models continue to unlock new capabilities, they also raise new questions about content safety, privacy, and responsible deployment. For most LLM applications, it is valuable to be able to diagnose problematic behavior post-hoc. For some LLM applications, however, it is *essential* to prevent problematic behavior in the first place.<\/p>\n\n\n\n

Towards that end, we\u2019ve heard from many users that what they need most in their workflow is a way to catch \u201cbad behavior\u201d early\u2014whether it\u2019s off-topic or inappropriate generation, the accidental leak of sensitive data, or a so-called \u201cjailbreak\u201d of their application. This is where Guardrails come in.<\/p>\n\n\n\n

If you\u2019re unfamiliar, \u201cGuardrails\u201d refer to a broad category of tools for validating the output of an LLM application and intervening in cases where the output is unacceptable. The actual way that these tools go about monitoring and intervening varies quite a bit depending on the tool in question (more on that in a moment), but at a high-level, this is the general behavior.<\/p>\n\n\n\n

To put it simply: so far, Opik has been focused on measuring<\/em> behavior. With Opik Guardrails, we\u2019re starting to build tools to help shape<\/em> it<\/p>\n\n\n\n

What We\u2019re Launching<\/h2>\n\n\n\n

This initial release is a public beta<\/strong>. While we\u2019re committed to building a Guardrails product, we\u2019re treating this an initial release as an experimental project, sharing things early so we can learn in the open and gather feedback from real-world use.<\/p>\n\n\n\n

The beta consists of a simple REST API<\/strong><\/code> with two backend-hosted endpoints:<\/p>\n\n\n\n

    \n
  • Topic Moderation:<\/strong> Uses a fine-tuned BART model to evaluate whether an input falls within a specified topical boundary. Useful for domain-specific assistants or content-limited interfaces.<\/li>\n\n\n\n
  • PII Filtering:<\/strong> Built on Presidio, this endpoint identifies and redacts personally identifiable information (PII) from LLM input or output text.<\/li>\n<\/ul>\n\n\n\n

    Below, you can see a simple snippet that shows how you might interact with either of these endpoints: