{"id":6291,"date":"2023-06-14T21:37:48","date_gmt":"2023-06-15T05:37:48","guid":{"rendered":"https:\/\/live-cometml.pantheonsite.io\/?p=6291"},"modified":"2025-04-24T17:15:26","modified_gmt":"2025-04-24T17:15:26","slug":"comet-snowflake-a-powerful-combination-for-better-reproducibility-and-visibility-into-your-ml-workflow","status":"publish","type":"post","link":"https:\/\/www.comet.com\/site\/blog\/comet-snowflake-a-powerful-combination-for-better-reproducibility-and-visibility-into-your-ml-workflow\/","title":{"rendered":"Comet + Snowflake: A Powerful Combination for Better Reproducibility and Visibility Into Your ML Workflow"},"content":{"rendered":"\n<p><span style=\"font-weight: 400;\">Companies around the world use Snowflake to securely store, manage and process their data at scale. Teams and organizations who want to use their data in Snowflake to train machine learning (ML) models, use <\/span><a href=\"https:\/\/www.snowflake.com\/en\/data-cloud\/snowpark\/\"><span style=\"font-weight: 400;\">Snowpark<\/span><\/a><span style=\"font-weight: 400;\"> and its integrated repository of Python ML libraries and frameworks.&nbsp;<\/span><\/p>\n\n\n\n<p><span style=\"font-weight: 400;\">Comet is an MLOps platform that allows ML teams to reproduce, debug, manage, and monitor their models with our Experiment Management, Artifacts, Model Registry, and Model Production Monitoring products.<\/span><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span style=\"font-weight: 400;\">The Need for Dataset Lineage and Versioning in ML<\/span><\/h2>\n\n\n\n<p><span style=\"font-weight: 400;\">\u201cGarbage in, garbage out\u201d is a common saying in the machine learning world. If you give your machine learning model bad data to train on, chances are it won\u2019t perform to your expectations. During the debugging phase of the ML Lifecycle, it&#8217;s important for practitioners to not only view their model\u2019s metrics, but also log which exact dataset version was used for training. Dataset Version 1.0.2&nbsp; has more recent data which might help the model generalize better. But how would you know that\u2019s the case if you have no way of seeing the lineage from a training run to a specific dataset version?<\/span><\/p>\n\n\n\n<p>&nbsp;<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span style=\"font-weight: 400;\">Upload Snowpark DataFrames as Comet Artifacts<\/span><\/h2>\n\n\n\n<p><span style=\"font-weight: 400;\">Comet\u2019s Integration with Snowflake makes it seamless to upload a Snowpark DataFrame as a Comet Artifact.&nbsp;<\/span><\/p>\n\n\n\n<p><script src=\"https:\/\/gist.github.com\/sherpan\/3112017a89aaf17d9c465ed145b7725c.js\"><\/script><\/p>\n\n\n\n<p><span style=\"font-weight: 400;\">Then within Comet&#8217;s Artifact UI, users can find the<\/span><\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><span style=\"font-weight: 400;\">Dataset version\u00a0<\/span><\/li>\n\n\n\n<li><span style=\"font-weight: 400;\">SQL Query used to create the DataFrame\u00a0<\/span><\/li>\n\n\n\n<li><span style=\"font-weight: 400;\">Sample data from the DataFrame<\/span><\/li>\n\n\n\n<li><span style=\"font-weight: 400;\">Link back to the Snowflake UI where the data is stored\u00a0<\/span><\/li>\n\n\n\n<li><span style=\"font-weight: 400;\">Lineage to see which experiments are using this dataset<\/span><\/li>\n<\/ol>\n\n\n\n<h2 class=\"wp-block-heading\"><figure><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-6293\" src=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/06\/sf_artifact.gif\" alt=\"\" width=\"1463\" height=\"773\"><\/figure><\/h2>\n\n\n\n<h2 class=\"wp-block-heading\"><span style=\"font-weight: 400;\">Connect Snowpark Artifacts to an Experiment<\/span><\/h2>\n\n\n\n<p><span style=\"font-weight: 400;\">Comet tracks all the relevant information needed to reproduce and debug model training runs. With Comet\u2019s SDK, developers can log metrics, hyper-parameters, code, assets, and artifacts for an experiment and visualize it within Comet. <\/span><\/p>\n\n\n\n<p><script src=\"https:\/\/gist.github.com\/sherpan\/49b23451bae80c6332c7004f835c0166.js\"><\/script><\/p>\n\n\n\n<p><span style=\"font-weight: 400;\">By <\/span><b>linking<\/b><span style=\"font-weight: 400;\"> an artifact to an experiment, it\u2019s now possible to fully debug your model. In the graphic below, see how in just a couple of clicks in the Comet UI, practitioners can see the output metrics, code, mode graph and dataset version for a training run using Snowflake!<\/span><\/p>\n\n\n\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"1463\" height=\"773\" src=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/06\/sf_single_exp.gif\" alt=\"snowflake single experiment\" class=\"wp-image-6299\"\/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span style=\"font-weight: 400;\">Try for free&nbsp;<\/span><\/h2>\n\n\n\n<p><span style=\"font-weight: 400;\">Comet is an extremely easy tool to integrate with your current machine learning workflows. <\/span><a href=\"http:\/\/comet.com\/signup\"><span style=\"font-weight: 400;\">Sign-up for a free account today<\/span><\/a><span style=\"font-weight: 400;\"> and see how easy it makes debugging and reproducing machine learning models!<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Companies around the world use Snowflake to securely store, manage and process their data at scale. Teams and organizations who want to use their data in Snowflake to train machine learning (ML) models, use Snowpark and its integrated repository of Python ML libraries and frameworks.&nbsp; Comet is an MLOps platform that allows ML teams to [&hellip;]<\/p>\n","protected":false},"author":21,"featured_media":6320,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"customer_name":"","customer_description":"","customer_industry":"","customer_technologies":"","customer_logo":"","footnotes":""},"categories":[8,5],"tags":[],"coauthors":[134],"class_list":["post-6291","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-comet-community-hub","category-partners-integrations"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v25.9 (Yoast SEO v25.9) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Comet + Snowflake: A Powerful Combination for Better Reproducibility and Visibility Into Your ML Workflow - Comet<\/title>\n<meta name=\"description\" content=\"Comet is now fully integrated with Snowflake and Snowpark so you can reproduce, debug, manage, and monitor your models faster.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.comet.com\/site\/blog\/comet-snowflake-a-powerful-combination-for-better-reproducibility-and-visibility-into-your-ml-workflow\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Comet + Snowflake: A Powerful Combination for Better Reproducibility and Visibility Into Your ML Workflow\" \/>\n<meta property=\"og:description\" content=\"Comet is now fully integrated with Snowflake and Snowpark so you can reproduce, debug, manage, and monitor your models faster.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.comet.com\/site\/blog\/comet-snowflake-a-powerful-combination-for-better-reproducibility-and-visibility-into-your-ml-workflow\/\" \/>\n<meta property=\"og:site_name\" content=\"Comet\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cometdotml\" \/>\n<meta property=\"article:published_time\" content=\"2023-06-15T05:37:48+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-04-24T17:15:26+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/06\/V02.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"675\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Siddharth Mehta\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@Cometml\" \/>\n<meta name=\"twitter:site\" content=\"@Cometml\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Siddharth Mehta\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Comet + Snowflake: A Powerful Combination for Better Reproducibility and Visibility Into Your ML Workflow - Comet","description":"Comet is now fully integrated with Snowflake and Snowpark so you can reproduce, debug, manage, and monitor your models faster.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.comet.com\/site\/blog\/comet-snowflake-a-powerful-combination-for-better-reproducibility-and-visibility-into-your-ml-workflow\/","og_locale":"en_US","og_type":"article","og_title":"Comet + Snowflake: A Powerful Combination for Better Reproducibility and Visibility Into Your ML Workflow","og_description":"Comet is now fully integrated with Snowflake and Snowpark so you can reproduce, debug, manage, and monitor your models faster.","og_url":"https:\/\/www.comet.com\/site\/blog\/comet-snowflake-a-powerful-combination-for-better-reproducibility-and-visibility-into-your-ml-workflow\/","og_site_name":"Comet","article_publisher":"https:\/\/www.facebook.com\/cometdotml","article_published_time":"2023-06-15T05:37:48+00:00","article_modified_time":"2025-04-24T17:15:26+00:00","og_image":[{"width":1200,"height":675,"url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/06\/V02.png","type":"image\/png"}],"author":"Siddharth Mehta","twitter_card":"summary_large_image","twitter_creator":"@Cometml","twitter_site":"@Cometml","twitter_misc":{"Written by":"Siddharth Mehta","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.comet.com\/site\/blog\/comet-snowflake-a-powerful-combination-for-better-reproducibility-and-visibility-into-your-ml-workflow\/#article","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/blog\/comet-snowflake-a-powerful-combination-for-better-reproducibility-and-visibility-into-your-ml-workflow\/"},"author":{"name":"Siddharth Mehta","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/652eb7d782d18f295922f50ea3b9e54c"},"headline":"Comet + Snowflake: A Powerful Combination for Better Reproducibility and Visibility Into Your ML Workflow","datePublished":"2023-06-15T05:37:48+00:00","dateModified":"2025-04-24T17:15:26+00:00","mainEntityOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/comet-snowflake-a-powerful-combination-for-better-reproducibility-and-visibility-into-your-ml-workflow\/"},"wordCount":401,"publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/comet-snowflake-a-powerful-combination-for-better-reproducibility-and-visibility-into-your-ml-workflow\/#primaryimage"},"thumbnailUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/06\/V02.png","articleSection":["Comet Community Hub","Partners &amp; Integrations"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.comet.com\/site\/blog\/comet-snowflake-a-powerful-combination-for-better-reproducibility-and-visibility-into-your-ml-workflow\/","url":"https:\/\/www.comet.com\/site\/blog\/comet-snowflake-a-powerful-combination-for-better-reproducibility-and-visibility-into-your-ml-workflow\/","name":"Comet + Snowflake: A Powerful Combination for Better Reproducibility and Visibility Into Your ML Workflow - Comet","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/comet-snowflake-a-powerful-combination-for-better-reproducibility-and-visibility-into-your-ml-workflow\/#primaryimage"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/comet-snowflake-a-powerful-combination-for-better-reproducibility-and-visibility-into-your-ml-workflow\/#primaryimage"},"thumbnailUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/06\/V02.png","datePublished":"2023-06-15T05:37:48+00:00","dateModified":"2025-04-24T17:15:26+00:00","description":"Comet is now fully integrated with Snowflake and Snowpark so you can reproduce, debug, manage, and monitor your models faster.","breadcrumb":{"@id":"https:\/\/www.comet.com\/site\/blog\/comet-snowflake-a-powerful-combination-for-better-reproducibility-and-visibility-into-your-ml-workflow\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.comet.com\/site\/blog\/comet-snowflake-a-powerful-combination-for-better-reproducibility-and-visibility-into-your-ml-workflow\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/blog\/comet-snowflake-a-powerful-combination-for-better-reproducibility-and-visibility-into-your-ml-workflow\/#primaryimage","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/06\/V02.png","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/06\/V02.png","width":1200,"height":675,"caption":"Comet + Snowflake text with a combination of both logos"},{"@type":"BreadcrumbList","@id":"https:\/\/www.comet.com\/site\/blog\/comet-snowflake-a-powerful-combination-for-better-reproducibility-and-visibility-into-your-ml-workflow\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.comet.com\/site\/"},{"@type":"ListItem","position":2,"name":"Comet + Snowflake: A Powerful Combination for Better Reproducibility and Visibility Into Your ML Workflow"}]},{"@type":"WebSite","@id":"https:\/\/www.comet.com\/site\/#website","url":"https:\/\/www.comet.com\/site\/","name":"Comet","description":"Build Better Models Faster","publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.comet.com\/site\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.comet.com\/site\/#organization","name":"Comet ML, Inc.","alternateName":"Comet","url":"https:\/\/www.comet.com\/site\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","width":310,"height":310,"caption":"Comet ML, Inc."},"image":{"@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/cometdotml","https:\/\/x.com\/Cometml","https:\/\/www.youtube.com\/channel\/UCmN63HKvfXSCS-UwVwmK8Hw"]},{"@type":"Person","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/652eb7d782d18f295922f50ea3b9e54c","name":"Siddharth Mehta","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/image\/940c7280faea9e1b8b086c2ed7ec01db","url":"https:\/\/secure.gravatar.com\/avatar\/27a672e997fa7a66796e4be0503e0efeec6bd34daae185bb6de163227a5a0739?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/27a672e997fa7a66796e4be0503e0efeec6bd34daae185bb6de163227a5a0739?s=96&d=mm&r=g","caption":"Siddharth Mehta"},"description":"ML Growth Engineer @ Comet. Interested in Computer Vision, Robotics, and Reinforcement Learning","sameAs":["https:\/\/www.comet.com\/"],"url":"https:\/\/www.comet.com\/site\/blog\/author\/siddharthmcomet-com\/"}]}},"_links":{"self":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/6291","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/users\/21"}],"replies":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/comments?post=6291"}],"version-history":[{"count":1,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/6291\/revisions"}],"predecessor-version":[{"id":15616,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/6291\/revisions\/15616"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media\/6320"}],"wp:attachment":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media?parent=6291"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/categories?post=6291"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/tags?post=6291"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/coauthors?post=6291"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}