{"id":12093,"date":"2024-12-02T08:09:47","date_gmt":"2024-12-02T16:09:47","guid":{"rendered":"https:\/\/live-cometml.pantheonsite.io\/?post_type=press_release&#038;p=12093"},"modified":"2025-05-15T14:10:57","modified_gmt":"2025-05-15T14:10:57","slug":"comet-launches-opik","status":"publish","type":"press_release","link":"https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/comet-launches-opik\/","title":{"rendered":"Comet Launches Opik, an Open Source LLM Evaluation Platform"},"content":{"rendered":"\n<p><strong>NEW YORK&nbsp;\u2013 September 17, 2024 \u2013&nbsp;<\/strong><a href=\"https:\/\/www.comet.com\/site\/\" target=\"_blank\" rel=\"noopener\" data-saferedirecturl=\"https:\/\/www.google.com\/url?q=https:\/\/www.comet.com\/site\/&amp;source=gmail&amp;ust=1733241071173000&amp;usg=AOvVaw1y8oB61JWRFSLe-aOvXJ5C\">Comet<\/a>, a leading end-to-end model evaluation platform, today announced a vanguard large language model (LLM) evaluation product: <a href=\"https:\/\/www.comet.com\/site\/products\/opik\/\">Opik<\/a>. The platform is a true open-source project, with the full suite of tools included free in the source code.<\/p>\n\n\n\n<p>While building LLM-based applications is increasingly prevalent, it remains a challenging task for developers, due to a low tolerance for failure in many use cases. A bridge between software engineering and data science, Opik enables developers to evaluate, test and ship LLM applications with various observability tools designed to improve language model interactions across the development life cycle.<\/p>\n\n\n\n<p>The platform\u2019s three core components contribute to optimizing and benchmarking LLM applications with ease:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><p dir=\"ltr\" role=\"presentation\"><strong>Observability:<\/strong> Gain visibility with the ability to record, sort, search and understand each step an LLM application takes to generate a response. Manually annotate, view and compare LLM responses. Tracing capabilities are possible during development and in production.<\/p><\/li>\n\n\n\n<li><p dir=\"ltr\" role=\"presentation\"><strong>Model unit testing:<\/strong> A convenient SDK library allows developers to choose and run metrics, as well as consult built-in LLM judges for complex issues like hallucination detection, factuality and moderation. This automates evaluation and eliminates the need to manually review LLM responses, lending to better scalability.<\/p><\/li>\n\n\n\n<li><p dir=\"ltr\" role=\"presentation\"><strong>Scoring:<\/strong> Store test cases as datasets, run evaluation experiments and compare the results. Score individual LLM outputs and aggregate performance across application versions.<\/p><\/li>\n<\/ul>\n\n\n\n<p>Fully open source, individual users can download the code from GitHub and run it locally. Opik is also compatible with any LLM, and it comes with a direct OpenAI integration out of the box, allowing developers to work with significant efficiency.<\/p>\n\n\n\n<p>\u201cComet has been contributing to machine learning open source for seven years and will continue to do so,\u201d said Comet Co-founder and CEO Gideon Mendels. \u201cWhile we previously open-sourced smaller components of our platform with our ML analysis and visualization tool, Kangas, Opik will allow any developer to evaluate their AI applications and models.\u201d<\/p>\n\n\n\n<p>A highly scalable and industry-compliant version is also available to enterprise teams, which offers additional benefits, such as implementation flexibility, team collaboration and user management for enhanced safety and security.<\/p>\n\n\n\n<p>Opik is a pertinent extension of Comet\u2019s mission to help data scientists, engineers and team leaders accelerate and optimize artificial intelligence. Comet\u2019s tools\u2013focused on experiment management, model management and production monitoring\u2013address fundamental pain points and reduce friction in the AI workflow.<\/p>\n\n\n\n<p>Alongside the company\u2019s dedication to open-source contributions, hundreds of organizations utilize Comet\u2019s platform, including Netflix, Uber, Cisco, Ancestry, Etsy and Zappos. Founded in 2017, Comet is headquartered in New York City, with a remote team spanning 14 countries. Comet has secured $70 million in funding to empower practitioners and teams to achieve business value with AI.<\/p>\n\n\n\n<p>To learn more about Comet and Opik, visit&nbsp;<a href=\"http:\/\/www.comet.com\/\" target=\"_blank\" rel=\"noopener\" data-saferedirecturl=\"https:\/\/www.google.com\/url?q=http:\/\/www.comet.com&amp;source=gmail&amp;ust=1733241071173000&amp;usg=AOvVaw1oLZp9bsZCzCikIbUoqsiq\">www.comet.com<\/a>.<\/p>\n\n\n\n<p><strong>About Comet<\/strong><\/p>\n\n\n\n<p>Comet provides an end-to-end model evaluation platform for AI developers, with best in class LLM evaluation, experiment tracking, and production monitoring. Comet\u2019s platform is trusted by over 150 enterprise customers including Netflix, Cepsa, Etsy, Uber and Zappos. Individuals and academic teams use Comet\u2019s platform to advance research in their fields of study. Founded in 2017, Comet is headquartered in New York, NY with a remote workforce in 14 countries on four continents. Comet is free to individuals and academic teams. Startup, team, and enterprise licensing is also available. To learn more, visit&nbsp;<a href=\"http:\/\/www.comet.com\/\" target=\"_blank\" rel=\"noopener\" data-saferedirecturl=\"https:\/\/www.google.com\/url?q=http:\/\/www.comet.com\/&amp;source=gmail&amp;ust=1733241071173000&amp;usg=AOvVaw3I4TVhjtALgTHmeuofixS3\">www.comet.com<\/a>.<\/p>\n\n\n\n<p><strong>Editorial Contact:<\/strong><br>Claire Pe\u00f1a<br>VP of Marketing<br>clairep@comet.com<\/p>\n","protected":false},"excerpt":{"rendered":"<p>NEW YORK&nbsp;\u2013 September 17, 2024 \u2013&nbsp;Comet, a leading end-to-end model evaluation platform, today announced a vanguard large language model (LLM) evaluation product: Opik. The platform is a true open-source project, with the full suite of tools included free in the source code. While building LLM-based applications is increasingly prevalent, it remains a challenging task for [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":11238,"template":"","meta":{"customer_name":"","customer_description":"","customer_industry":"","customer_technologies":"","customer_logo":"","subtitle":"End-to-end Platform Enables Developers To Evaluate, Test and Ship LLM Applications With a Suite of Observability Tools ","footnotes":""},"coauthors":[126],"class_list":["post-12093","press_release","type-press_release","status-publish","has-post-thumbnail","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v25.9 (Yoast SEO v25.9) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Comet Launches Opik, an Open Source LLM Evaluation Platform<\/title>\n<meta name=\"description\" content=\"End-to-end Platform Enables Developers To Evaluate, Test and Ship LLM Applications With a Suite of Observability Tools\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/comet-launches-opik\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Comet Launches Opik, an Open Source LLM Evaluation Platform\" \/>\n<meta property=\"og:description\" content=\"End-to-end Platform Enables Developers To Evaluate, Test and Ship LLM Applications With a Suite of Observability Tools\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/comet-launches-opik\/\" \/>\n<meta property=\"og:site_name\" content=\"Comet\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cometdotml\" \/>\n<meta property=\"article:modified_time\" content=\"2025-05-15T14:10:57+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/09\/comet-opik-announcement-2-scaled-1.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"2560\" \/>\n\t<meta property=\"og:image:height\" content=\"1440\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@Cometml\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"3 minutes\" \/>\n\t<meta name=\"twitter:label2\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data2\" content=\"engineering@atre.net\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Comet Launches Opik, an Open Source LLM Evaluation Platform","description":"End-to-end Platform Enables Developers To Evaluate, Test and Ship LLM Applications With a Suite of Observability Tools","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/comet-launches-opik\/","og_locale":"en_US","og_type":"article","og_title":"Comet Launches Opik, an Open Source LLM Evaluation Platform","og_description":"End-to-end Platform Enables Developers To Evaluate, Test and Ship LLM Applications With a Suite of Observability Tools","og_url":"https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/comet-launches-opik\/","og_site_name":"Comet","article_publisher":"https:\/\/www.facebook.com\/cometdotml","article_modified_time":"2025-05-15T14:10:57+00:00","og_image":[{"width":2560,"height":1440,"url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/09\/comet-opik-announcement-2-scaled-1.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_site":"@Cometml","twitter_misc":{"Est. reading time":"3 minutes","Written by":"engineering@atre.net"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/comet-launches-opik\/","url":"https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/comet-launches-opik\/","name":"Comet Launches Opik, an Open Source LLM Evaluation Platform","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/comet-launches-opik\/#primaryimage"},"image":{"@id":"https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/comet-launches-opik\/#primaryimage"},"thumbnailUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/09\/comet-opik-announcement-2-scaled-1.jpg","datePublished":"2024-12-02T16:09:47+00:00","dateModified":"2025-05-15T14:10:57+00:00","description":"End-to-end Platform Enables Developers To Evaluate, Test and Ship LLM Applications With a Suite of Observability Tools","breadcrumb":{"@id":"https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/comet-launches-opik\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/comet-launches-opik\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/comet-launches-opik\/#primaryimage","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/09\/comet-opik-announcement-2-scaled-1.jpg","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/09\/comet-opik-announcement-2-scaled-1.jpg","width":2560,"height":1440,"caption":"open source llm evaluation"},{"@type":"BreadcrumbList","@id":"https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/comet-launches-opik\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.comet.com\/site\/"},{"@type":"ListItem","position":2,"name":"Press Releases","item":"https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/"},{"@type":"ListItem","position":3,"name":"Comet Launches Opik, an Open Source LLM Evaluation Platform"}]},{"@type":"WebSite","@id":"https:\/\/www.comet.com\/site\/#website","url":"https:\/\/www.comet.com\/site\/","name":"Comet","description":"Build Better Models Faster","publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.comet.com\/site\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.comet.com\/site\/#organization","name":"Comet ML, Inc.","alternateName":"Comet","url":"https:\/\/www.comet.com\/site\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","width":310,"height":310,"caption":"Comet ML, Inc."},"image":{"@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/cometdotml","https:\/\/x.com\/Cometml","https:\/\/www.youtube.com\/channel\/UCmN63HKvfXSCS-UwVwmK8Hw"]}]}},"_links":{"self":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/press_release\/12093","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/press_release"}],"about":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/types\/press_release"}],"author":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/users\/1"}],"version-history":[{"count":1,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/press_release\/12093\/revisions"}],"predecessor-version":[{"id":15932,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/press_release\/12093\/revisions\/15932"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media\/11238"}],"wp:attachment":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media?parent=12093"}],"wp:term":[{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/coauthors?post=12093"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}