{"id":15330,"date":"2024-12-18T13:28:42","date_gmt":"2024-12-18T21:28:42","guid":{"rendered":"https:\/\/live-cometml.pantheonsite.io\/?p=12278"},"modified":"2025-04-24T17:02:31","modified_gmt":"2025-04-24T17:02:31","slug":"new-features-december-2024","status":"publish","type":"post","link":"https:\/\/www.comet.com\/site\/blog\/new-features-december-2024\/","title":{"rendered":"Comet Product Releases December 2024"},"content":{"rendered":"\n<p>Each layer of visibility into your training and debugging workflows builds confidence that your models will work reliably in production. With a focus on details that help you iterate and ship complex systems at scale, we\u2019ve added new logging capabilities and more to the Comet platform. Log in today to discover a set of new features designed to help you understand how your ML and GenAI systems are performing on the whole, and drill down to analyze specific issues.<\/p>\n\n\n\n<p>In case you missed the big news out of AWS re:Invent, Comet is now available as one of the very first third-party apps fully integrated within Amazon SageMaker. You can now log into your SageMaker account and provision a self-contained and managed version of Comet securely inside SageMaker. <a href=\"https:\/\/www.comet.com\/site\/sagemaker\/\">Learn more<\/a>.<\/p>\n\n\n\n<p>Meanwhile, as teams race to build accurate and efficient systems on top of LLMs, we\u2019ve added 4 powerful new features to <a href=\"https:\/\/www.comet.com\/site\/products\/opik\/\">Opik<\/a>, Comet\u2019s end-to-end <a href=\"https:\/\/www.comet.com\/site\/products\/opik\/\">LLM evaluation<\/a> platform. Opik\u2019s new prompt library makes it easy to store and version your prompt templates \u2014 and manage them in production. Opik now supports multimodal logging, allowing you to track not just the text input and output of your LLM, but images too. New features within our monitoring dashboard include a trace metrics tab featuring feedback scores, token usage, and number of traces over time, as well as a cost tracking feature for greater visibility into your LLM usage.<\/p>\n\n\n\n<p>Read on for details on how these enhancements streamline your LLM development workflows!<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Comet is Now Available Natively within AWS<\/h2>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img decoding=\"async\" src=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/12\/sagemaker-x-comet-1-1-1024x385.png\" alt=\"Amazon Sagemaker and Comet partnership \" class=\"wp-image-12280\"\/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<p>Comet is now one of the first fully managed 3rd-party solutions available natively in Amazon SageMaker! This partnership lets you launch a private Comet instance directly within your SageMaker console \u2013 with infrastructure, billing, and security fully managed by AWS.<\/p>\n\n\n\n<p>With the native Comet + SageMaker integration, developers get:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Automated LLM evaluation with Opik (including Bedrock support)<\/li>\n\n\n\n<li>Best-in-class experiment tracking &amp; visualization, fully integrated with SageMaker<\/li>\n<\/ul>\n\n\n\n<p>Read more about the announcement <a href=\"https:\/\/aws.amazon.com\/blogs\/machine-learning\/building-generative-ai-and-ml-solutions-faster-with-ai-apps-from-aws-partners-using-amazon-sagemaker\">here<\/a>!<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Opik: Multi-Modal Logging<\/h2>\n\n\n\n<p>Opik now supports <a href=\"https:\/\/www.comet.com\/docs\/opik\/tracing\/log_multimodal_traces\">multimodal traces<\/a>, so you can log images to the platform and better analyze the relationship between text and images in your LLM applications. Log images using OpenAI\u2019s SDK, or manually using automatic detection within the Opik UI.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img decoding=\"async\" src=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/12\/image_trace-ca3b944272ca35a48560999b6c9d8ba4-1024x576.png\" alt=\"Multi Modal Logging example\" class=\"wp-image-12281\"\/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Opik: Prompt Library<\/h2>\n\n\n\n<p>It can be easy to lose track of which prompt templates you\u2019ve used in the past, but Opik\u2019s new <a href=\"https:\/\/www.comet.com\/docs\/opik\/library\/prompt_management\/\">prompt library<\/a> makes versioning a breeze. Our SDK prompt object lets you version prompts stored in code or in files, making it simple to download the most recent prompt in production without deploying new code.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img decoding=\"async\" src=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/12\/Screenshot-2024-11-07-at-17.05.08-1-1024x550.png\" alt=\"Opik Prompt Library\" class=\"wp-image-12283\"\/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Opik: LLM Trace Metrics<\/h2>\n\n\n\n<p>In Opik\u2019s Projects section, you\u2019ll see a new tab that plots <a href=\"https:\/\/www.comet.com\/docs\/opik\/tracing\/production_monitoring\/?utm_source=newsletter&amp;utm_medium=email&amp;utm_campaign=12-24-product&amp;mkt_tok=OTEyLUpKUC00NDUAAAGXWuSed4Z99f-H1QQCQgwVu1j3qZjYTjvhRwg6udkoRwSUbgCUfGfiLnEfGDorVG_mR5qAAZa6uDjTpnFHDCjF4M2D6nwn-Y6rv2ZWkUYv0g\">trace metrics over time<\/a>, including average feedback scores, token usage, and number of traces, helping you more effectively monitor your LLMs in production.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img decoding=\"async\" src=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/12\/www.comet_.com_opik_opik-cookbooks_projects_0192cf60-46fe-7fb5-b65e-0e34ed416621_traces_filters5B5DtypemetricsDocumentation-screenshot-1024x576.png\" alt=\"\" class=\"wp-image-12282\"\/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Opik: LLM Cost Tracking<\/h2>\n\n\n\n<p>Monitor and optimize your LLM spend with the addition of <a href=\"https:\/\/www.comet.com\/docs\/opik\/tracing\/cost_tracking\">cost tracking<\/a>. Opik measures token usage across all traces, so you can identify cost patterns and anomalies at any level, from individual spans to project-wide analytics. All costs are estimated in USD and automatically calculated for text OpenAI models.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img decoding=\"async\" src=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/12\/unnamed-1-1024x530.png\" alt=\"LLM Cost Tracking\" class=\"wp-image-12284\"\/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">New Integration: Opik + AWS Bedrock<\/h2>\n\n\n\n<p>We\u2019re pleased to announce our <a href=\"https:\/\/www.comet.com\/docs\/opik\/tracing\/integrations\/bedrock\">integration with AWS Bedrock<\/a>. This new feature enables teams to monitor their LLM models deployed on AWS from leading AI providers \u2014 such as AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon. This feature is fully available now with Opik SDK v1.0.5.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">How to Upgrade<\/h2>\n\n\n\n<p>If you are using www.comet.com, no action is required! You have access to these features and can use them immediately.<\/p>\n\n\n\n<p>If you are using a self-hosted version of Comet, before you upgrade please verify which version of Comet you are running.<br>\nComet Versions December 2024<br>\nFrontend version: 5.90.5<br>\nBackend version: 3.48.238<br>\nOptimizer version: 2.1.4<br>\nSDK version: 3.47.4<\/p>\n\n\n\n<p>To check your frontend version: Click your username in the top-right corner of the Comet application and check the Version at the bottom of the pop-up.<\/p>\n\n\n\n<p>To check your backend version: Visit . &lt;your_comet_url&gt;.com\/api\/isAlive\/ver<\/p>\n\n\n\n<p>To check your SDK version: In your training environment run the command pip freeze | grep comet_ml<\/p>\n\n\n\n<p>If you would like to upgrade, please contact your Technical Account Manager or Deployment Engineer on the Comet team. You can always reach out directly to Claire Longo (clairel@comet.com) with general questions.<\/p>\n\n\n\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Each layer of visibility into your training and debugging workflows builds confidence that your models will work reliably in production. With a focus on details that help you iterate and ship complex systems at scale, we\u2019ve added new logging capabilities and more to the Comet platform. Log in today to discover a set of new [&hellip;]<\/p>\n","protected":false},"author":140,"featured_media":12324,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"customer_name":"","customer_description":"","customer_industry":"","customer_technologies":"","customer_logo":"","footnotes":""},"categories":[9],"tags":[],"coauthors":[127],"class_list":["post-15330","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-product"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v25.9 (Yoast SEO v25.9) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Release Notes: New LLM Eval Features December 2024<\/title>\n<meta name=\"description\" content=\"Explore 4 new releases within Opik, Comet&#039;s open-source LLM evaluation tool, and the details of Comet&#039;s new availability inside SageMaker.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.comet.com\/site\/blog\/new-features-december-2024\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Comet Product Releases December 2024\" \/>\n<meta property=\"og:description\" content=\"Explore 4 new releases within Opik, Comet&#039;s open-source LLM evaluation tool, and the details of Comet&#039;s new availability inside SageMaker.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.comet.com\/site\/blog\/new-features-december-2024\/\" \/>\n<meta property=\"og:site_name\" content=\"Comet\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cometdotml\" \/>\n<meta property=\"article:published_time\" content=\"2024-12-18T21:28:42+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-04-24T17:02:31+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/12\/unnamed-1.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1920\" \/>\n\t<meta property=\"og:image:height\" content=\"994\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Caroline Borders\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@Cometml\" \/>\n<meta name=\"twitter:site\" content=\"@Cometml\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Caroline Borders\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Release Notes: New LLM Eval Features December 2024","description":"Explore 4 new releases within Opik, Comet's open-source LLM evaluation tool, and the details of Comet's new availability inside SageMaker.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.comet.com\/site\/blog\/new-features-december-2024\/","og_locale":"en_US","og_type":"article","og_title":"Comet Product Releases December 2024","og_description":"Explore 4 new releases within Opik, Comet's open-source LLM evaluation tool, and the details of Comet's new availability inside SageMaker.","og_url":"https:\/\/www.comet.com\/site\/blog\/new-features-december-2024\/","og_site_name":"Comet","article_publisher":"https:\/\/www.facebook.com\/cometdotml","article_published_time":"2024-12-18T21:28:42+00:00","article_modified_time":"2025-04-24T17:02:31+00:00","og_image":[{"width":1920,"height":994,"url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/12\/unnamed-1.png","type":"image\/png"}],"author":"Caroline Borders","twitter_card":"summary_large_image","twitter_creator":"@Cometml","twitter_site":"@Cometml","twitter_misc":{"Written by":"Caroline Borders","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.comet.com\/site\/blog\/new-features-december-2024\/#article","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/blog\/new-features-december-2024\/"},"author":{"name":"Caroline Borders","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/8500e2f020e85676c245e00af46bae3c"},"headline":"Comet Product Releases December 2024","datePublished":"2024-12-18T21:28:42+00:00","dateModified":"2025-04-24T17:02:31+00:00","mainEntityOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/new-features-december-2024\/"},"wordCount":732,"publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/new-features-december-2024\/#primaryimage"},"thumbnailUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/12\/unnamed-1.png","articleSection":["Product"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.comet.com\/site\/blog\/new-features-december-2024\/","url":"https:\/\/www.comet.com\/site\/blog\/new-features-december-2024\/","name":"Release Notes: New LLM Eval Features December 2024","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/new-features-december-2024\/#primaryimage"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/new-features-december-2024\/#primaryimage"},"thumbnailUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/12\/unnamed-1.png","datePublished":"2024-12-18T21:28:42+00:00","dateModified":"2025-04-24T17:02:31+00:00","description":"Explore 4 new releases within Opik, Comet's open-source LLM evaluation tool, and the details of Comet's new availability inside SageMaker.","breadcrumb":{"@id":"https:\/\/www.comet.com\/site\/blog\/new-features-december-2024\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.comet.com\/site\/blog\/new-features-december-2024\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/blog\/new-features-december-2024\/#primaryimage","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/12\/unnamed-1.png","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/12\/unnamed-1.png","width":1920,"height":994,"caption":"LLM Cost Tracking"},{"@type":"BreadcrumbList","@id":"https:\/\/www.comet.com\/site\/blog\/new-features-december-2024\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.comet.com\/site\/"},{"@type":"ListItem","position":2,"name":"Comet Product Releases December 2024"}]},{"@type":"WebSite","@id":"https:\/\/www.comet.com\/site\/#website","url":"https:\/\/www.comet.com\/site\/","name":"Comet","description":"Build Better Models Faster","publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.comet.com\/site\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.comet.com\/site\/#organization","name":"Comet ML, Inc.","alternateName":"Comet","url":"https:\/\/www.comet.com\/site\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","width":310,"height":310,"caption":"Comet ML, Inc."},"image":{"@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/cometdotml","https:\/\/x.com\/Cometml","https:\/\/www.youtube.com\/channel\/UCmN63HKvfXSCS-UwVwmK8Hw"]},{"@type":"Person","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/8500e2f020e85676c245e00af46bae3c","name":"Caroline Borders","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/image\/77bfb2d62bc772cc39672e46e3e8059f","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/12\/cropped-1672334331755-2-96x96.jpeg","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2024\/12\/cropped-1672334331755-2-96x96.jpeg","caption":"Caroline Borders"},"url":"https:\/\/www.comet.com\/site\/blog\/author\/carolineb\/"}]}},"_links":{"self":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/15330","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/users\/140"}],"replies":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/comments?post=15330"}],"version-history":[{"count":1,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/15330\/revisions"}],"predecessor-version":[{"id":15350,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/15330\/revisions\/15350"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media\/12324"}],"wp:attachment":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media?parent=15330"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/categories?post=15330"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/tags?post=15330"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/coauthors?post=15330"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}