{"id":11236,"date":"2024-09-16T07:42:29","date_gmt":"2024-09-16T15:42:29","guid":{"rendered":"https:\/\/live-cometml.pantheonsite.io\/?p=11236"},"modified":"2026-04-08T16:45:28","modified_gmt":"2026-04-08T16:45:28","slug":"announcing-opik","status":"publish","type":"post","link":"https:\/\/www.comet.com\/site\/blog\/announcing-opik\/","title":{"rendered":"Meet Opik: Your New Tool to Evaluate, Test, and Monitor LLM Applications"},"content":{"rendered":"\n<p><span style=\"font-weight: 400;\">Today, we\u2019re thrilled to introduce Opik \u2013 an open-source, end-to-end LLM development platform that provides the <a href=\"https:\/\/www.comet.com\/site\/blog\/llm-observability-tools\/\">LLM observability tools<\/a> you need to confidently evaluate, test, and monitor your LLM applications across development and production.<\/span><\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"2560\" height=\"1440\" src=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/09\/comet-opik-llm-eval-announcement.jpg\" alt=\"\" class=\"wp-image-13407\" srcset=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/09\/comet-opik-llm-eval-announcement.jpg 2560w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/09\/comet-opik-llm-eval-announcement-300x169.jpg 300w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/09\/comet-opik-llm-eval-announcement-1024x576.jpg 1024w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/09\/comet-opik-llm-eval-announcement-768x432.jpg 768w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/09\/comet-opik-llm-eval-announcement-1536x864.jpg 1536w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/09\/comet-opik-llm-eval-announcement-2048x1152.jpg 2048w\" sizes=\"auto, (max-width: 2560px) 100vw, 2560px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<p>&nbsp;<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-end-to-end-llm-evaluation-in-one-platform\">End-to-End LLM Evaluation in One Platform<\/h2>\n\n\n\n<p><span style=\"font-weight: 400;\">Opik was created to address the unique challenges of LLM-based development:<\/span><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>See what\u2019s happening within your LLM pipeline:<\/strong> Opik automatically traces your entire pipeline, allowing you to step through and debug each component of your application, even in complex <a href=\"https:\/\/www.comet.com\/site\/blog\/retrieval-augmented-generation\/\">retrieval augmented generation<\/a> (RAG) or multi-agent architectures.<\/li>\n\n\n\n<li><strong>Deploy LLM applications you can trust:<\/strong> Opik has out-of-the-box support for complex LLM-based evaluations, as well as real-time monitoring, allowing you to detect hallucinations, unintended behaviors, and performance degradations immediately.<\/li>\n\n\n\n<li><strong>Simplify unit testing:<\/strong> Opik integrates with Pytest to support \u201cmodel unit tests,\u201d allowing you to compose robust evaluation pipelines out of reusable components. You can implement <a href=\"https:\/\/www.comet.com\/site\/blog\/llm-as-a-judge\/\">LLM-as-a-judge<\/a> metrics in a single line of Python, and reuse them across all of your applications.<\/li>\n\n\n\n<li><strong>Continuously improve your LLM application:<\/strong> Use Opik to collect, annotate and score your production data through simple Python methods or through the Opik UI.<\/li>\n<\/ul>\n\n\n\n<p>&nbsp;<\/p>\n\n\n\n<p><iframe loading=\"lazy\" title=\"YouTube video player\" src=\"https:\/\/www.youtube.com\/embed\/B4oboG62lyA?si=sSov_7xP468dBQTd\" width=\"820\" height=\"463\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-why-comet-developer-first-ai-amp-ml-expertise\"><span style=\"font-weight: 400;\">Why Comet? Developer-First AI &amp; ML Expertise<\/span><\/h2>\n\n\n\n<p>We\u2019ve always been closely connected to AI developers, working to streamline how teams build and productionize machine learning models. Our MLOps tools like <a href=\"https:\/\/www.comet.com\/site\/products\/ml-experiment-tracking\/\">Experiment Management<\/a> and <a href=\"https:\/\/www.comet.com\/site\/products\/model-production-monitoring\/\">Model Production Monitoring<\/a> help close the loop in the model development workflow. But our commitment to the community goes beyond our products \u2013 we\u2019ve also been passionate about supporting the open-source community by open-sourcing portions of our platform such as <a href=\"https:\/\/github.com\/comet-ml\/kangas\">Kangas<\/a>, our tool for ML analysis and visualization.<\/p>\n\n\n\n<p><span style=\"font-weight: 400;\">Opik is a natural next step for us. We hope it enables Comet users and the AI community at large to get more out of LLMs. We named it after Ernst Opik, an Estonian astronomer who was at the forefront of the study of comets and solar system dynamics. With so many of today\u2019s great discoveries happening at the forefront of AI, we hope you\u2019ll find inspiration in Opik\u2019s story.&nbsp;<\/span><\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-getting-started-with-opik\"><span style=\"font-weight: 400;\">Getting Started with Opik<\/span><\/h2>\n\n\n\n<p><span style=\"font-weight: 400;\">Install<\/span> <span style=\"font-weight: 400;\">Opik with just a few lines of code \u2013 whether you\u2019re self-hosting or using Comet\u2019s cloud platform, Opik is built to fit seamlessly into your existing stack. Opik is compatible with any LLM you like, and supports direct integrations with OpenAI, LangChain, LlamaIndex, Predibase, Ragas, promptfoo, LiteLLM, and Pinecone out of the box.&nbsp;<\/span><\/p>\n\n\n\n<p><b>Opik\u2019s full <a href=\"https:\/\/www.comet.com\/site\/blog\/llm-evaluation-guide\/\">LLM evaluation<\/a> feature set is free to use<\/b><span style=\"font-weight: 400;\">, with a highly scalable and industry compliant version available for enterprise teams. <\/span><a href=\"\/signup?from=llm\"><span style=\"font-weight: 400;\">Sign up for free<\/span><\/a><span style=\"font-weight: 400;\">, check out the <\/span><a href=\"https:\/\/www.comet.com\/docs\/v2\/guides\/llm_evaluation\/opik\/\"><span style=\"font-weight: 400;\">documentation<\/span><\/a><span style=\"font-weight: 400;\">, and log your first LLM trace today. <\/span><\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-join-the-opik-community-nbsp\"><span style=\"font-weight: 400;\">Join the Opik Community&nbsp;<\/span><\/h3>\n\n\n\n<p><span style=\"font-weight: 400;\">As an open-source project, we\u2019re eager to see how the community shapes Opik\u2019s future. We invite you to contribute bug reports, feature requests, or improvements to the documentation \u2013 or join our discussions on <\/span><a href=\"https:\/\/join.slack.com\/t\/cometml\/shared_invite\/zt-2qjjuykua-cqIlMWUPCjGXYnIdMqhRLg\"><span style=\"font-weight: 400;\">Slack<\/span><\/a><span style=\"font-weight: 400;\"> and <\/span><a href=\"https:\/\/github.com\/comet-ml\/opik\"><span style=\"font-weight: 400;\">Github<\/span><\/a><span style=\"font-weight: 400;\">to leave feedback and help shape the roadmap.\u00a0<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Today, we\u2019re thrilled to introduce Opik \u2013 an open-source, end-to-end LLM development platform that provides the LLM observability tools you need to confidently evaluate, test, and monitor your LLM applications across development and production. &nbsp; End-to-End LLM Evaluation in One Platform Opik was created to address the unique challenges of LLM-based development: &nbsp; Why Comet? [&hellip;]<\/p>\n","protected":false},"author":141,"featured_media":13407,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"customer_name":"","customer_description":"","customer_industry":"","customer_technologies":"","customer_logo":"","footnotes":""},"categories":[8,65,9],"tags":[],"coauthors":[107,129],"class_list":["post-11236","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-comet-community-hub","category-llmops","category-product"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v25.9 (Yoast SEO v25.9) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Introducing Open Source LLM Evaluation from Comet<\/title>\n<meta name=\"description\" content=\"Discover Opik, the new open-source LLM observability platform from Comet.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.comet.com\/site\/blog\/announcing-opik\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Meet Opik: Your New Tool to Evaluate, Test, and Monitor LLM Applications\" \/>\n<meta property=\"og:description\" content=\"Discover Opik, the new open-source LLM observability platform from Comet.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.comet.com\/site\/blog\/announcing-opik\/\" \/>\n<meta property=\"og:site_name\" content=\"Comet\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cometdotml\" \/>\n<meta property=\"article:published_time\" content=\"2024-09-16T15:42:29+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-04-08T16:45:28+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/09\/comet-opik-llm-eval-announcement-1024x576.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t<meta property=\"og:image:height\" content=\"576\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Gideon Mendels, Jacques Verre\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:image\" content=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/09\/comet-opik-announcement-2-scaled-1.jpg\" \/>\n<meta name=\"twitter:creator\" content=\"@Cometml\" \/>\n<meta name=\"twitter:site\" content=\"@Cometml\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Gideon Mendels, Jacques Verre\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Introducing Open Source LLM Evaluation from Comet","description":"Discover Opik, the new open-source LLM observability platform from Comet.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.comet.com\/site\/blog\/announcing-opik\/","og_locale":"en_US","og_type":"article","og_title":"Meet Opik: Your New Tool to Evaluate, Test, and Monitor LLM Applications","og_description":"Discover Opik, the new open-source LLM observability platform from Comet.","og_url":"https:\/\/www.comet.com\/site\/blog\/announcing-opik\/","og_site_name":"Comet","article_publisher":"https:\/\/www.facebook.com\/cometdotml","article_published_time":"2024-09-16T15:42:29+00:00","article_modified_time":"2026-04-08T16:45:28+00:00","og_image":[{"width":1024,"height":576,"url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/09\/comet-opik-llm-eval-announcement-1024x576.jpg","type":"image\/jpeg"}],"author":"Gideon Mendels, Jacques Verre","twitter_card":"summary_large_image","twitter_image":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/09\/comet-opik-announcement-2-scaled-1.jpg","twitter_creator":"@Cometml","twitter_site":"@Cometml","twitter_misc":{"Written by":"Gideon Mendels, Jacques Verre","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.comet.com\/site\/blog\/announcing-opik\/#article","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/blog\/announcing-opik\/"},"author":{"name":"James Shakespeare","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/051334c3ff961a01c6a0b459cfe3ba20"},"headline":"Meet Opik: Your New Tool to Evaluate, Test, and Monitor LLM Applications","datePublished":"2024-09-16T15:42:29+00:00","dateModified":"2026-04-08T16:45:28+00:00","mainEntityOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/announcing-opik\/"},"wordCount":502,"publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/announcing-opik\/#primaryimage"},"thumbnailUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/09\/comet-opik-llm-eval-announcement.jpg","articleSection":["Comet Community Hub","LLMOps","Product"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.comet.com\/site\/blog\/announcing-opik\/","url":"https:\/\/www.comet.com\/site\/blog\/announcing-opik\/","name":"Introducing Open Source LLM Evaluation from Comet","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/announcing-opik\/#primaryimage"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/announcing-opik\/#primaryimage"},"thumbnailUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/09\/comet-opik-llm-eval-announcement.jpg","datePublished":"2024-09-16T15:42:29+00:00","dateModified":"2026-04-08T16:45:28+00:00","description":"Discover Opik, the new open-source LLM observability platform from Comet.","breadcrumb":{"@id":"https:\/\/www.comet.com\/site\/blog\/announcing-opik\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.comet.com\/site\/blog\/announcing-opik\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/blog\/announcing-opik\/#primaryimage","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/09\/comet-opik-llm-eval-announcement.jpg","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/09\/comet-opik-llm-eval-announcement.jpg","width":2560,"height":1440},{"@type":"BreadcrumbList","@id":"https:\/\/www.comet.com\/site\/blog\/announcing-opik\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.comet.com\/site\/"},{"@type":"ListItem","position":2,"name":"Meet Opik: Your New Tool to Evaluate, Test, and Monitor LLM Applications"}]},{"@type":"WebSite","@id":"https:\/\/www.comet.com\/site\/#website","url":"https:\/\/www.comet.com\/site\/","name":"Comet","description":"Build Better Models Faster","publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.comet.com\/site\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.comet.com\/site\/#organization","name":"Comet ML, Inc.","alternateName":"Comet","url":"https:\/\/www.comet.com\/site\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","width":310,"height":310,"caption":"Comet ML, Inc."},"image":{"@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/cometdotml","https:\/\/x.com\/Cometml","https:\/\/www.youtube.com\/channel\/UCmN63HKvfXSCS-UwVwmK8Hw"]},{"@type":"Person","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/051334c3ff961a01c6a0b459cfe3ba20","name":"James Shakespeare","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/image\/dbeda0ea8667437687cc7412418c248a","url":"https:\/\/secure.gravatar.com\/avatar\/f0ca0a670f12fb9f3e6b0e083aae1341ce18f5da45b092e303088dfb51cb7f7d?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/f0ca0a670f12fb9f3e6b0e083aae1341ce18f5da45b092e303088dfb51cb7f7d?s=96&d=mm&r=g","caption":"James Shakespeare"},"url":"https:\/\/www.comet.com\/site\/blog\/author\/jshakes\/"}]}},"_links":{"self":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/11236","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/users\/141"}],"replies":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/comments?post=11236"}],"version-history":[{"count":2,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/11236\/revisions"}],"predecessor-version":[{"id":19485,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/11236\/revisions\/19485"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media\/13407"}],"wp:attachment":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media?parent=11236"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/categories?post=11236"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/tags?post=11236"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/coauthors?post=11236"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}