{"id":7846,"date":"2023-10-06T12:48:19","date_gmt":"2023-10-06T20:48:19","guid":{"rendered":"https:\/\/live-cometml.pantheonsite.io\/?p=7846"},"modified":"2025-04-24T17:05:55","modified_gmt":"2025-04-24T17:05:55","slug":"enhancing-mlflow-with-comet","status":"publish","type":"post","link":"https:\/\/www.comet.com\/site\/blog\/enhancing-mlflow-with-comet\/","title":{"rendered":"Enhancing MLflow with Comet"},"content":{"rendered":"\n<div class=\"fi fj fk fl fm\">\n<div class=\"ab ca\">\n<div class=\"ch bg eu ev ew ex\">\n<figure class=\"mj mk ml mm mn mo mg mh paragraph-image\">\n<div class=\"mp mq ec mr bg ms\" tabindex=\"0\" role=\"button\">\n<figure><img loading=\"lazy\" decoding=\"async\" class=\"bg mt mu c\" role=\"presentation\" src=\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*vQjA9xyHMTNHZUJNVILmTg.jpeg\" alt=\"\" width=\"700\" height=\"1050\"><\/figure><div class=\"mg mh mi\"><picture><\/picture><\/div>\n<\/div><figcaption class=\"mv mw mx mg mh my mz be b bf z dw\" data-selectable-paragraph=\"\">Victor Iaqu, Unsplash.<\/figcaption><\/figure>\n<p id=\"0edd\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">As a data professional, switching machine learning tracking tools happens. It ends up being time consuming work rewriting the code into the new tool. Data scientists are no exception: we often have to rework previous code that was used in experiment tracking.<\/p>\n<p id=\"bad3\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">This was the problem I faced in 2021, when I switched from MLflow workflows to <a class=\"af nv\" href=\"https:\/\/www.comet.com\/site\/\" target=\"_blank\" rel=\"noopener ugc nofollow\">Comet<\/a>. I was worried that I would have to translate the code and face the process of retesting and retraining. Fortunately, this wasn\u2019t the case.<\/p>\n<p id=\"dbe9\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">One of the less known features of Comet is that it can be used with MLflow. This means that experiment tracking can be used with your existing code.<\/p>\n<p id=\"ee7b\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">This article is directed towards people with familiarity with experiment tracking, as well as the Comet and MLflow user interface.<\/p>\n<p id=\"3c5c\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">In this article, I will discuss:<\/p>\n<ul class=\"\">\n<li id=\"33ce\" class=\"na nb fp be b gn nc nd ne gq nf ng nh ni oq nk nl nm or no np nq os ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\">Differences between MLflow and Comet.<\/li>\n<li id=\"f044\" class=\"na nb fp be b gn ow nd ne gq ox ng nh ni oy nk nl nm oz no np nq pa ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\">The advantages of using Comet to enhance your existing MLflow experiments.<\/li>\n<li id=\"d2c9\" class=\"na nb fp be b gn ow nd ne gq ox ng nh ni oy nk nl nm oz no np nq pa ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\">How to set up your existing MLflow experiment workflow and integrate it with Comet<\/li>\n<li id=\"68e8\" class=\"na nb fp be b gn ow nd ne gq ox ng nh ni oy nk nl nm oz no np nq pa ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\">Limitations of MLflow experiment workflow.<\/li>\n<\/ul>\n<p id=\"2b24\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">Let\u2019s get started.<\/p>\n<h1 id=\"c3d3\" class=\"pb pc fp be pd pe pf gp pg ph pi gs pj pk pl pm pn po pp pq pr ps pt pu pv pw bj\" data-selectable-paragraph=\"\">MLflow vs. Comet<\/h1>\n<figure class=\"mj mk ml mm mn mo mg mh paragraph-image\">\n<div class=\"mp mq ec mr bg ms\" tabindex=\"0\" role=\"button\">\n<figure><img loading=\"lazy\" decoding=\"async\" class=\"bg mt mu c\" role=\"presentation\" src=\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*hyAmQsLATMzy43iva4a14g.jpeg\" alt=\"\" width=\"700\" height=\"243\"><\/figure><div class=\"mg mh px\"><picture><source srcset=\"https:\/\/miro.medium.com\/v2\/resize:fit:640\/format:webp\/1*hyAmQsLATMzy43iva4a14g.jpeg 640w, https:\/\/miro.medium.com\/v2\/resize:fit:720\/format:webp\/1*hyAmQsLATMzy43iva4a14g.jpeg 720w, https:\/\/miro.medium.com\/v2\/resize:fit:750\/format:webp\/1*hyAmQsLATMzy43iva4a14g.jpeg 750w, https:\/\/miro.medium.com\/v2\/resize:fit:786\/format:webp\/1*hyAmQsLATMzy43iva4a14g.jpeg 786w, https:\/\/miro.medium.com\/v2\/resize:fit:828\/format:webp\/1*hyAmQsLATMzy43iva4a14g.jpeg 828w, https:\/\/miro.medium.com\/v2\/resize:fit:1100\/format:webp\/1*hyAmQsLATMzy43iva4a14g.jpeg 1100w, https:\/\/miro.medium.com\/v2\/resize:fit:1400\/format:webp\/1*hyAmQsLATMzy43iva4a14g.jpeg 1400w\" type=\"image\/webp\" sizes=\"(min-resolution: 4dppx) and (max-width: 700px) 50vw, (-webkit-min-device-pixel-ratio: 4) and (max-width: 700px) 50vw, (min-resolution: 3dppx) and (max-width: 700px) 67vw, (-webkit-min-device-pixel-ratio: 3) and (max-width: 700px) 65vw, (min-resolution: 2.5dppx) and (max-width: 700px) 80vw, (-webkit-min-device-pixel-ratio: 2.5) and (max-width: 700px) 80vw, (min-resolution: 2dppx) and (max-width: 700px) 100vw, (-webkit-min-device-pixel-ratio: 2) and (max-width: 700px) 100vw, 700px\"><source srcset=\"https:\/\/miro.medium.com\/v2\/resize:fit:640\/1*hyAmQsLATMzy43iva4a14g.jpeg 640w, https:\/\/miro.medium.com\/v2\/resize:fit:720\/1*hyAmQsLATMzy43iva4a14g.jpeg 720w, https:\/\/miro.medium.com\/v2\/resize:fit:750\/1*hyAmQsLATMzy43iva4a14g.jpeg 750w, https:\/\/miro.medium.com\/v2\/resize:fit:786\/1*hyAmQsLATMzy43iva4a14g.jpeg 786w, https:\/\/miro.medium.com\/v2\/resize:fit:828\/1*hyAmQsLATMzy43iva4a14g.jpeg 828w, https:\/\/miro.medium.com\/v2\/resize:fit:1100\/1*hyAmQsLATMzy43iva4a14g.jpeg 1100w, https:\/\/miro.medium.com\/v2\/resize:fit:1400\/1*hyAmQsLATMzy43iva4a14g.jpeg 1400w\" sizes=\"(min-resolution: 4dppx) and (max-width: 700px) 50vw, (-webkit-min-device-pixel-ratio: 4) and (max-width: 700px) 50vw, (min-resolution: 3dppx) and (max-width: 700px) 67vw, (-webkit-min-device-pixel-ratio: 3) and (max-width: 700px) 65vw, (min-resolution: 2.5dppx) and (max-width: 700px) 80vw, (-webkit-min-device-pixel-ratio: 2.5) and (max-width: 700px) 80vw, (min-resolution: 2dppx) and (max-width: 700px) 100vw, (-webkit-min-device-pixel-ratio: 2) and (max-width: 700px) 100vw, 700px\" data-testid=\"og\"><\/picture><\/div>\n<\/div>\n<figcaption class=\"mv mw mx mg mh my mz be b bf z dw\" data-selectable-paragraph=\"\">ML flow and Comet Logos, Author\u2019s Creation<\/figcaption>\n<\/figure>\n<p id=\"08d5\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">Before I get into how to use Comet with MLflow, it\u2019s important to compare the two.<\/p>\n<h2 id=\"6ef2\" class=\"py pc fp be pd pz qa qb pg qc qd qe pj ni qf qg qh nm qi qj qk nq ql qm qn qo bj\" data-selectable-paragraph=\"\">What is MLflow?<\/h2>\n<p id=\"fa84\" class=\"pw-post-body-paragraph na nb fp be b gn qp nd ne gq qq ng nh ni qr nk nl nm qs no np nq qt ns nt nu fi bj\" data-selectable-paragraph=\"\">MLflow is an open source platform to manage the different stages of machine learning operations: experimentation, reproducibility, deployment, and a central model registry. For MLflow, this consists of:<\/p>\n<ul class=\"\">\n<li id=\"1a1a\" class=\"na nb fp be b gn nc nd ne gq nf ng nh ni oq nk nl nm or no np nq os ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\">Experiment Tracking: artifact, metrics, and parameter tracking<\/li>\n<li id=\"b08d\" class=\"na nb fp be b gn ow nd ne gq ox ng nh ni oy nk nl nm oz no np nq pa ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\">Reproducible Experiment Runs<\/li>\n<li id=\"3646\" class=\"na nb fp be b gn ow nd ne gq ox ng nh ni oy nk nl nm oz no np nq pa ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\">Machine Learning Model Stores<\/li>\n<li id=\"a51f\" class=\"na nb fp be b gn ow nd ne gq ox ng nh ni oy nk nl nm oz no np nq pa ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\">Model Registry<\/li>\n<\/ul>\n<p id=\"666a\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">It is also available on cloud platforms such as Databricks in Azure or AWS, allowing data scientists and machine learning engineers to reproduce their work in the cloud and in Jupyter notebook.<\/p>\n<p id=\"134e\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">It allows sending models to your deployment tools of choice. You can record runs, organize them into experiments, and log additional data using the MLflow tracking API and UI.<\/p>\n<p id=\"6ecf\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">MLflow also allows you to access artifacts and models from experiment runs their UI on either your system or cloud provider.<\/p>\n<p id=\"5121\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">So how does Comet compare?<\/p>\n<h2 id=\"b639\" class=\"py pc fp be pd pz qa qb pg qc qd qe pj ni qf qg qh nm qi qj qk nq ql qm qn qo bj\" data-selectable-paragraph=\"\">Comet\u2019s Advantages<\/h2>\n<p id=\"b2d6\" class=\"pw-post-body-paragraph na nb fp be b gn qp nd ne gq qq ng nh ni qr nk nl nm qs no np nq qt ns nt nu fi bj\" data-selectable-paragraph=\"\">Comet has several advantages over MLflow:<\/p>\n<ul class=\"\">\n<li id=\"4f2f\" class=\"na nb fp be b gn nc nd ne gq nf ng nh ni oq nk nl nm or no np nq os ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\"><a class=\"af nv\" href=\"https:\/\/www.comet.com\/site\/workspaces-interactive-reports\/\" target=\"_blank\" rel=\"noopener ugc nofollow\"><strong class=\"be qu\">Automatic Logging<\/strong><\/a><strong class=\"be qu\">.<\/strong> Track and version Jupyter notebooks and machine learning Python notebooks, along with their outputs. Regardless of whether you submit your experiment, everything will be safely versioned and ready to be explored.<\/li>\n<li id=\"995e\" class=\"na nb fp be b gn ow nd ne gq ox ng nh ni oy nk nl nm oz no np nq pa ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\"><strong class=\"be qu\">Hardware Metrics.<\/strong> Comet automatically monitors your hardware metrics. You can monitor CPU, GPU, memory consumption live while you train your models. With MLflow, you need to wait until a run is completed. If you have existing MLflow runs, and use Comet with it, you\u2019ll be able to see resource usages in real time.<\/li>\n<li id=\"0a29\" class=\"na nb fp be b gn ow nd ne gq ox ng nh ni oy nk nl nm oz no np nq pa ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\"><strong class=\"be qu\">Easier Visualization.<\/strong> Visualizations are easily accessible and customizable within Comet via <a class=\"af nv\" href=\"https:\/\/www.comet.com\/site\/panels\/\" target=\"_blank\" rel=\"noopener ugc nofollow\">Panels<\/a>. You can graph the changes easily using drag and drop<\/li>\n<li id=\"7b6c\" class=\"na nb fp be b gn ow nd ne gq ox ng nh ni oy nk nl nm oz no np nq pa ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\"><a class=\"af nv\" href=\"https:\/\/www.comet.com\/site\/data-scientists\/\" target=\"_blank\" rel=\"noopener ugc nofollow\"><strong class=\"be qu\">Better Experiment Organization<\/strong><\/a><strong class=\"be qu\">.<\/strong> You can see everything that was logged and can conveniently organize your experiments, filtering them by tag, parameter, etc.<\/li>\n<li id=\"459b\" class=\"na nb fp be b gn ow nd ne gq ox ng nh ni oy nk nl nm oz no np nq pa ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\"><strong class=\"be qu\">Online Storage.<\/strong> Comet allows you to save experiments by backing it up on Comet, allowing you to easily share experiments between team members. MLflow stores and tracks experiments locally, limiting user collaboration.<\/li>\n<\/ul>\n<p id=\"2c68\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">If you\u2019re making the switch over to Comet and have MLflow experiments, you are able to take advantage of these features of Comet.<\/p>\n<h1 id=\"0a0d\" class=\"pb pc fp be pd pe pf gp pg ph pi gs pj pk pl pm pn po pp pq pr ps pt pu pv pw bj\" data-selectable-paragraph=\"\">Setting Up<\/h1>\n<p id=\"239b\" class=\"pw-post-body-paragraph na nb fp be b gn qp nd ne gq qq ng nh ni qr nk nl nm qs no np nq qt ns nt nu fi bj\" data-selectable-paragraph=\"\">To get Comet to run with MLflow, you will need to install an additional extension to Comet: comet for mlflow.<\/p>\n<pre>!pip install comet-for-mlflow<\/pre>\n<p id=\"aec8\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">A Comet API Key must be configured prior to the run to be able to log experiment data from MLflow into a live experiment. More on this can be found on the <a class=\"af nv\" href=\"https:\/\/www.comet.com\/docs\/rest-api\/getting-started\/\" target=\"_blank\" rel=\"noopener ugc nofollow\">Comet documentation<\/a>.<\/p>\n<p id=\"3a99\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">It is also useful to change the experiment workspace, since Comet will place all experiments in the default folder. To set the workspace simply change the workspace:<\/p>\n<pre>comet_ml.login(project_name=\"enhancing-ml-flow\")<\/pre>\n<p id=\"b3dd\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">This will also allow you to check if you have your API key is valid or automatically set for the environment.<\/p>\n<p id=\"3cbb\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">Comet will automatically run and save the following in MLflow:<\/p>\n<ul class=\"\">\n<li id=\"77c0\" class=\"na nb fp be b gn nc nd ne gq nf ng nh ni oq nk nl nm or no np nq os ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\">Metrics<\/li>\n<li id=\"4547\" class=\"na nb fp be b gn ow nd ne gq ox ng nh ni oy nk nl nm oz no np nq pa ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\">Hyperparameters<\/li>\n<li id=\"986e\" class=\"na nb fp be b gn ow nd ne gq ox ng nh ni oy nk nl nm oz no np nq pa ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\">Models<\/li>\n<li id=\"76f6\" class=\"na nb fp be b gn ow nd ne gq ox ng nh ni oy nk nl nm oz no np nq pa ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\">Assets<\/li>\n<li id=\"c6b1\" class=\"na nb fp be b gn ow nd ne gq ox ng nh ni oy nk nl nm oz no np nq pa ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\">Source code<\/li>\n<li id=\"f174\" class=\"na nb fp be b gn ow nd ne gq ox ng nh ni oy nk nl nm oz no np nq pa ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\">Git repo and patch info<\/li>\n<li id=\"32aa\" class=\"na nb fp be b gn ow nd ne gq ox ng nh ni oy nk nl nm oz no np nq pa ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\">System Metrics<\/li>\n<li id=\"041a\" class=\"na nb fp be b gn ow nd ne gq ox ng nh ni oy nk nl nm oz no np nq pa ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\">CPU and GPU usage<\/li>\n<li id=\"1732\" class=\"na nb fp be b gn ow nd ne gq ox ng nh ni oy nk nl nm oz no np nq pa ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\">Python packages<\/li>\n<li id=\"e95f\" class=\"na nb fp be b gn ow nd ne gq ox ng nh ni oy nk nl nm oz no np nq pa ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\">Command-line arguments<\/li>\n<li id=\"b5de\" class=\"na nb fp be b gn ow nd ne gq ox ng nh ni oy nk nl nm oz no np nq pa ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\">Standard output<\/li>\n<li id=\"8215\" class=\"na nb fp be b gn ow nd ne gq ox ng nh ni oy nk nl nm oz no np nq pa ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\">Installed OS packages<\/li>\n<\/ul>\n<p id=\"9e38\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">Comet for MLflow is designed to work with existing MLflow pipelines, with relatively few modifications. Your metrics, parameters, artifacts, and models will be logged and you will be able to use some of the features of Comet\u2019s experiment tracking.<\/p>\n<h1 id=\"cd9f\" class=\"pb pc fp be pd pe pf gp pg ph pi gs pj pk pl pm pn po pp pq pr ps pt pu pv pw bj\" data-selectable-paragraph=\"\">Running Comet with an MLflow Experiment<\/h1>\n<figure class=\"mj mk ml mm mn mo mg mh paragraph-image\">\n<div class=\"mp mq ec mr bg ms\" tabindex=\"0\" role=\"button\">\n<figure><img loading=\"lazy\" decoding=\"async\" class=\"bg mt mu c\" role=\"presentation\" src=\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*JWsihtEGkqHKSIRwBCVyIw.png\" alt=\"\" width=\"700\" height=\"400\"><\/figure><div class=\"mg mh qy\"><picture><source srcset=\"https:\/\/miro.medium.com\/v2\/resize:fit:640\/format:webp\/1*JWsihtEGkqHKSIRwBCVyIw.png 640w, https:\/\/miro.medium.com\/v2\/resize:fit:720\/format:webp\/1*JWsihtEGkqHKSIRwBCVyIw.png 720w, https:\/\/miro.medium.com\/v2\/resize:fit:750\/format:webp\/1*JWsihtEGkqHKSIRwBCVyIw.png 750w, https:\/\/miro.medium.com\/v2\/resize:fit:786\/format:webp\/1*JWsihtEGkqHKSIRwBCVyIw.png 786w, https:\/\/miro.medium.com\/v2\/resize:fit:828\/format:webp\/1*JWsihtEGkqHKSIRwBCVyIw.png 828w, https:\/\/miro.medium.com\/v2\/resize:fit:1100\/format:webp\/1*JWsihtEGkqHKSIRwBCVyIw.png 1100w, https:\/\/miro.medium.com\/v2\/resize:fit:1400\/format:webp\/1*JWsihtEGkqHKSIRwBCVyIw.png 1400w\" type=\"image\/webp\" sizes=\"(min-resolution: 4dppx) and (max-width: 700px) 50vw, (-webkit-min-device-pixel-ratio: 4) and (max-width: 700px) 50vw, (min-resolution: 3dppx) and (max-width: 700px) 67vw, (-webkit-min-device-pixel-ratio: 3) and (max-width: 700px) 65vw, (min-resolution: 2.5dppx) and (max-width: 700px) 80vw, (-webkit-min-device-pixel-ratio: 2.5) and (max-width: 700px) 80vw, (min-resolution: 2dppx) and (max-width: 700px) 100vw, (-webkit-min-device-pixel-ratio: 2) and (max-width: 700px) 100vw, 700px\"><source srcset=\"https:\/\/miro.medium.com\/v2\/resize:fit:640\/1*JWsihtEGkqHKSIRwBCVyIw.png 640w, https:\/\/miro.medium.com\/v2\/resize:fit:720\/1*JWsihtEGkqHKSIRwBCVyIw.png 720w, https:\/\/miro.medium.com\/v2\/resize:fit:750\/1*JWsihtEGkqHKSIRwBCVyIw.png 750w, https:\/\/miro.medium.com\/v2\/resize:fit:786\/1*JWsihtEGkqHKSIRwBCVyIw.png 786w, https:\/\/miro.medium.com\/v2\/resize:fit:828\/1*JWsihtEGkqHKSIRwBCVyIw.png 828w, https:\/\/miro.medium.com\/v2\/resize:fit:1100\/1*JWsihtEGkqHKSIRwBCVyIw.png 1100w, https:\/\/miro.medium.com\/v2\/resize:fit:1400\/1*JWsihtEGkqHKSIRwBCVyIw.png 1400w\" sizes=\"(min-resolution: 4dppx) and (max-width: 700px) 50vw, (-webkit-min-device-pixel-ratio: 4) and (max-width: 700px) 50vw, (min-resolution: 3dppx) and (max-width: 700px) 67vw, (-webkit-min-device-pixel-ratio: 3) and (max-width: 700px) 65vw, (min-resolution: 2.5dppx) and (max-width: 700px) 80vw, (-webkit-min-device-pixel-ratio: 2.5) and (max-width: 700px) 80vw, (min-resolution: 2dppx) and (max-width: 700px) 100vw, (-webkit-min-device-pixel-ratio: 2) and (max-width: 700px) 100vw, 700px\" data-testid=\"og\"><\/picture><\/div>\n<\/div>\n<figcaption class=\"mv mw mx mg mh my mz be b bf z dw\" data-selectable-paragraph=\"\">Getting Started, Author\u2019s Picture<\/figcaption>\n<\/figure>\n<p id=\"2165\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">This experiment will be run from start to finish with the following Jupyter notebook using the UCI Wine dataset:<\/p>\n<p id=\"4a09\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\"><a class=\"af nv\" href=\"https:\/\/archive.ics.uci.edu\/ml\/datasets\/wine+quality\" target=\"_blank\" rel=\"noopener ugc nofollow\">UCI Machine Learning Repository: Wine Quality Data Set<\/a><\/p>\n<p id=\"254a\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">For clarity, we will be using snippets from that notebook to demonstrate how to enhance your existing MLflow experiments with Comet.<\/p>\n<p id=\"4909\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">Please note:<\/p>\n<p id=\"1878\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\"><em class=\"qz\">Comet_ML library and experiment must be run before MLflow and sklearn. They cannot be run in the same cell if you are using Jupyter notebook.<\/em><\/p>\n<h2 id=\"72ab\" class=\"py pc fp be pd pz qa qb pg qc qd qe pj ni qf qg qh nm qi qj qk nq ql qm qn qo bj\" data-selectable-paragraph=\"\">Starting and Ending an MLflow with Comet Experiment<\/h2>\n<p id=\"bd88\" class=\"pw-post-body-paragraph na nb fp be b gn qp nd ne gq qq ng nh ni qr nk nl nm qs no np nq qt ns nt nu fi bj\" data-selectable-paragraph=\"\">Running an MLflow experiment with Comet is different than a standard Comet experiment. In a normal Comet experiment, you would call the <em class=\"qz\">Experiment<\/em> method first, then define the parameters. This is seen below:<\/p>\n<pre class=\"mj mk ml mm mn ra rb rc rd ax re bj\"><span id=\"780c\" class=\"py pc fp rb b ia rf rg l iq rh\" data-selectable-paragraph=\"\">experiment = Experiment(workspace=\"mattblasa\",\n\tproject_name=\"comet_mlflow\",\n\tlog_code = True)<\/span><\/pre>\n<p id=\"4eae\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">However, in an experiment that uses existing MLflow coding, calling the <em class=\"qz\">Experiment() <\/em>method would result in two different experiments being created. Instead, you can use your existing MLflow experiment to run Comet:<\/p>\n<pre>#Comet Experiment begins here\nmlflow.start_run()\nmlflow.set_experiment(\"mlflow_wine\") #use this to set the run name.\n'''\nYour Code\n'''\n#Comet Experiment ends here\nmlflow.end_run()<\/pre>\n<p id=\"0914\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">Comet will automatically run an existing MLflow experiment starting with the <em class=\"qz\">mlflow.start_run() <\/em>method, and end with the <em class=\"qz\">mflow.end_run()<\/em> method. All code in between the two will be logged.<\/p>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<div class=\"fi fj fk fl fm\">\n<div class=\"ab ca\">\n<div class=\"ch bg eu ev ew ex\">\n<blockquote class=\"rq\"><p id=\"eeae\" class=\"rr rs fp be rt ru rv rw rx ry rz nu dw\" data-selectable-paragraph=\"\">Want to try Comet today? It\u2019s free! <a class=\"af nv\" href=\"https:\/\/www.comet.com\/site\/academics\/?utm_source=pricingpage\" target=\"_blank\" rel=\"noopener ugc nofollow\">Sign up for an individual or start-up account and see how Comet can help you make better models, faster<\/a>.<\/p><\/blockquote>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<div class=\"fi fj fk fl fm\">\n<div class=\"ab ca\">\n<div class=\"ch bg eu ev ew ex\">\n<h2 id=\"3e07\" class=\"py pc fp be pd pz qa qb pg qc qd qe pj ni qf qg qh nm qi qj qk nq ql qm qn qo bj\" data-selectable-paragraph=\"\">Logging Graphs<\/h2>\n<p id=\"cb0f\" class=\"pw-post-body-paragraph na nb fp be b gn qp nd ne gq qq ng nh ni qr nk nl nm qs no np nq qt ns nt nu fi bj\" data-selectable-paragraph=\"\">You can also log your current graphs that are created in an MLflow run. Unlike a normal Comet run, you will not be able to log the figure directly to the run. You will need to do two steps:<\/p>\n<ul class=\"\">\n<li id=\"0cfa\" class=\"na nb fp be b gn nc nd ne gq nf ng nh ni oq nk nl nm or no np nq os ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\">Save using Matplotlib\u2019s savefig method. When Comet for MLflow runs this method, it will automatically log it to the image store.<\/li>\n<li id=\"d67f\" class=\"na nb fp be b gn ow nd ne gq ox ng nh ni oy nk nl nm oz no np nq pa ns nt nu ot ou ov bj\" data-selectable-paragraph=\"\">Log artifact. It\u2019s a good practice to save the picture in both your local machine flow and in an online Comet experiment. To do this, use the mlflow.log_artifact with the saved figure from the previous step.<\/li>\n<\/ul>\n<p id=\"2a11\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">These will be saved in the graphics tab of a current experiment.<\/p>\n<p id=\"66c4\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">An example of this code is in the snippet below:<\/p>\n<pre>##create figure\nplt.bar(data['quality'], data['fixed acidity'])\nplt.xlabel('Quality')\nplt.ylabel('Fixed Acidity')\nplt.title('Quality')\nplt.savefig(\"quality.png\") #saves figure to comet image store\nplt.show()\n\nmlflow.log_artifact(\"quality.png\") #saves figure to artifact store in comet<\/pre>\n<h2 id=\"60a4\" class=\"py pc fp be pd pz qa qb pg qc qd qe pj ni qf qg qh nm qi qj qk nq ql qm qn qo bj\" data-selectable-paragraph=\"\">Logging Metrics and Parameters.<\/h2>\n<p id=\"d827\" class=\"pw-post-body-paragraph na nb fp be b gn qp nd ne gq qq ng nh ni qr nk nl nm qs no np nq qt ns nt nu fi bj\" data-selectable-paragraph=\"\">Comet will also log parameters used in hyperparameters, mlflow.log_param, and mlflow.log_metric. This will log into the hyperparamters and the model metrics to a Comet experiment run. These can be accessed from the metrics column in the experiment tab.<\/p>\n<figure class=\"mj mk ml mm mn mo mg mh paragraph-image\">\n<div class=\"mp mq ec mr bg ms\" tabindex=\"0\" role=\"button\">\n<figure><img loading=\"lazy\" decoding=\"async\" class=\"bg mt mu c\" role=\"presentation\" src=\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*VPgB04OVxRQ4u1h2ikj6tg.png\" alt=\"\" width=\"700\" height=\"314\"><\/figure><div class=\"mg mh sa\"><picture><source srcset=\"https:\/\/miro.medium.com\/v2\/resize:fit:640\/format:webp\/1*VPgB04OVxRQ4u1h2ikj6tg.png 640w, https:\/\/miro.medium.com\/v2\/resize:fit:720\/format:webp\/1*VPgB04OVxRQ4u1h2ikj6tg.png 720w, https:\/\/miro.medium.com\/v2\/resize:fit:750\/format:webp\/1*VPgB04OVxRQ4u1h2ikj6tg.png 750w, https:\/\/miro.medium.com\/v2\/resize:fit:786\/format:webp\/1*VPgB04OVxRQ4u1h2ikj6tg.png 786w, https:\/\/miro.medium.com\/v2\/resize:fit:828\/format:webp\/1*VPgB04OVxRQ4u1h2ikj6tg.png 828w, https:\/\/miro.medium.com\/v2\/resize:fit:1100\/format:webp\/1*VPgB04OVxRQ4u1h2ikj6tg.png 1100w, https:\/\/miro.medium.com\/v2\/resize:fit:1400\/format:webp\/1*VPgB04OVxRQ4u1h2ikj6tg.png 1400w\" type=\"image\/webp\" sizes=\"(min-resolution: 4dppx) and (max-width: 700px) 50vw, (-webkit-min-device-pixel-ratio: 4) and (max-width: 700px) 50vw, (min-resolution: 3dppx) and (max-width: 700px) 67vw, (-webkit-min-device-pixel-ratio: 3) and (max-width: 700px) 65vw, (min-resolution: 2.5dppx) and (max-width: 700px) 80vw, (-webkit-min-device-pixel-ratio: 2.5) and (max-width: 700px) 80vw, (min-resolution: 2dppx) and (max-width: 700px) 100vw, (-webkit-min-device-pixel-ratio: 2) and (max-width: 700px) 100vw, 700px\"><source srcset=\"https:\/\/miro.medium.com\/v2\/resize:fit:640\/1*VPgB04OVxRQ4u1h2ikj6tg.png 640w, https:\/\/miro.medium.com\/v2\/resize:fit:720\/1*VPgB04OVxRQ4u1h2ikj6tg.png 720w, https:\/\/miro.medium.com\/v2\/resize:fit:750\/1*VPgB04OVxRQ4u1h2ikj6tg.png 750w, https:\/\/miro.medium.com\/v2\/resize:fit:786\/1*VPgB04OVxRQ4u1h2ikj6tg.png 786w, https:\/\/miro.medium.com\/v2\/resize:fit:828\/1*VPgB04OVxRQ4u1h2ikj6tg.png 828w, https:\/\/miro.medium.com\/v2\/resize:fit:1100\/1*VPgB04OVxRQ4u1h2ikj6tg.png 1100w, https:\/\/miro.medium.com\/v2\/resize:fit:1400\/1*VPgB04OVxRQ4u1h2ikj6tg.png 1400w\" sizes=\"(min-resolution: 4dppx) and (max-width: 700px) 50vw, (-webkit-min-device-pixel-ratio: 4) and (max-width: 700px) 50vw, (min-resolution: 3dppx) and (max-width: 700px) 67vw, (-webkit-min-device-pixel-ratio: 3) and (max-width: 700px) 65vw, (min-resolution: 2.5dppx) and (max-width: 700px) 80vw, (-webkit-min-device-pixel-ratio: 2.5) and (max-width: 700px) 80vw, (min-resolution: 2dppx) and (max-width: 700px) 100vw, (-webkit-min-device-pixel-ratio: 2) and (max-width: 700px) 100vw, 700px\" data-testid=\"og\"><\/picture><\/div>\n<\/div>\n<figcaption class=\"mv mw mx mg mh my mz be b bf z dw\" data-selectable-paragraph=\"\">Comet Metrics Tab, Author\u2019s Screenshot<\/figcaption>\n<\/figure>\n<p id=\"e2ce\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">Example of MLflow metrics and parameters is below:<\/p>\n<pre>predicted_qualities = lr.predict(test_x)\n(rmse, mae, r2) = eval_metrics(test_y, predicted_qualities)\n\nprint(\"Elasticnet model (alpha=%f, l1_ratio=%f):\" % (alpha, l1_ratio))\nprint(\"  RMSE: %s\" % rmse)\nprint(\"  MAE: %s\" % mae)\nprint(\"  R2: %s\" % r2)\n\n#Log hyperparameters\nmlflow.log_param(\"alpha\", alpha)\nmlflow.log_param(\"l1_ratio\", l1_ratio)\n\n#Log metrics\nmlflow.log_metric(\"rmse\", rmse)\nmlflow.log_metric(\"r2\", r2)\nmlflow.log_metric(\"mae\", mae)<\/pre>\n<h2 id=\"2788\" class=\"py pc fp be pd pz qa qb pg qc qd qe pj ni qf qg qh nm qi qj qk nq ql qm qn qo bj\" data-selectable-paragraph=\"\">Logging Model<\/h2>\n<p id=\"5f36\" class=\"pw-post-body-paragraph na nb fp be b gn qp nd ne gq qq ng nh ni qr nk nl nm qs no np nq qt ns nt nu fi bj\" data-selectable-paragraph=\"\">To save your MLflow model it\u2019s best to save it on your computer. Use the <em class=\"qz\">mlflow.sklearn.log_model <\/em>method to save it to your local drive. This allows you to revert to an older previous model version if you need to.<\/p>\n<pre>#save model to artifact store in local drive\nmlflow.sklearn.log_model(lr, \"model\")\n\n#This will output the local folder name for your run meta data, artifacts, and data model\nprint(\"Model saved in run %s\" % mlflow.active_run().info.run_uuid)<\/pre>\n<p id=\"41a1\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">All models saved by this method are saved as a pickle file (Example: model.pkl) within the run name (the uuid). These will also be logged into the Assets and <a class=\"af nv\" href=\"https:\/\/www.comet.com\/site\/artifacts\/\" target=\"_blank\" rel=\"noopener ugc nofollow\">Artifact<\/a> tabs in Comet.<\/p>\n<p id=\"0820\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">If you want to upload the model created in MLflow to Comet, you will need to create a separate experiment, and upload it that way. Here is an example:<\/p>\n<pre>from comet_ml import Experiment\nexperiment = Experiment(workspace=\"mattblasa\", project_name=\"comet_mlflow\", log_code = True)\nexperiment.log_model(\"Test Model\", \"model.pkl\")\nexperiment.add_tag('Register Model')\nexperiment.end()<\/pre>\n<h2 id=\"802c\" class=\"py pc fp be pd pz qa qb pg qc qd qe pj ni qf qg qh nm qi qj qk nq ql qm qn qo bj\" data-selectable-paragraph=\"\">Finding Your MLflow Experiment Artifacts<\/h2>\n<figure class=\"mj mk ml mm mn mo mg mh paragraph-image\">\n<figure><img loading=\"lazy\" decoding=\"async\" class=\"bg mt mu c\" role=\"presentation\" src=\"https:\/\/miro.medium.com\/v2\/resize:fit:484\/1*Kdb23Jucz0ERr6y-y9O2XQ.png\" alt=\"\" width=\"484\" height=\"637\"><\/figure><div class=\"mg mh sb\"><picture><source srcset=\"https:\/\/miro.medium.com\/v2\/resize:fit:640\/format:webp\/1*Kdb23Jucz0ERr6y-y9O2XQ.png 640w, https:\/\/miro.medium.com\/v2\/resize:fit:720\/format:webp\/1*Kdb23Jucz0ERr6y-y9O2XQ.png 720w, https:\/\/miro.medium.com\/v2\/resize:fit:750\/format:webp\/1*Kdb23Jucz0ERr6y-y9O2XQ.png 750w, https:\/\/miro.medium.com\/v2\/resize:fit:786\/format:webp\/1*Kdb23Jucz0ERr6y-y9O2XQ.png 786w, https:\/\/miro.medium.com\/v2\/resize:fit:828\/format:webp\/1*Kdb23Jucz0ERr6y-y9O2XQ.png 828w, https:\/\/miro.medium.com\/v2\/resize:fit:1100\/format:webp\/1*Kdb23Jucz0ERr6y-y9O2XQ.png 1100w, https:\/\/miro.medium.com\/v2\/resize:fit:968\/format:webp\/1*Kdb23Jucz0ERr6y-y9O2XQ.png 968w\" type=\"image\/webp\" sizes=\"(min-resolution: 4dppx) and (max-width: 700px) 50vw, (-webkit-min-device-pixel-ratio: 4) and (max-width: 700px) 50vw, (min-resolution: 3dppx) and (max-width: 700px) 67vw, (-webkit-min-device-pixel-ratio: 3) and (max-width: 700px) 65vw, (min-resolution: 2.5dppx) and (max-width: 700px) 80vw, (-webkit-min-device-pixel-ratio: 2.5) and (max-width: 700px) 80vw, (min-resolution: 2dppx) and (max-width: 700px) 100vw, (-webkit-min-device-pixel-ratio: 2) and (max-width: 700px) 100vw, 484px\"><source srcset=\"https:\/\/miro.medium.com\/v2\/resize:fit:640\/1*Kdb23Jucz0ERr6y-y9O2XQ.png 640w, https:\/\/miro.medium.com\/v2\/resize:fit:720\/1*Kdb23Jucz0ERr6y-y9O2XQ.png 720w, https:\/\/miro.medium.com\/v2\/resize:fit:750\/1*Kdb23Jucz0ERr6y-y9O2XQ.png 750w, https:\/\/miro.medium.com\/v2\/resize:fit:786\/1*Kdb23Jucz0ERr6y-y9O2XQ.png 786w, https:\/\/miro.medium.com\/v2\/resize:fit:828\/1*Kdb23Jucz0ERr6y-y9O2XQ.png 828w, https:\/\/miro.medium.com\/v2\/resize:fit:1100\/1*Kdb23Jucz0ERr6y-y9O2XQ.png 1100w, https:\/\/miro.medium.com\/v2\/resize:fit:968\/1*Kdb23Jucz0ERr6y-y9O2XQ.png 968w\" sizes=\"(min-resolution: 4dppx) and (max-width: 700px) 50vw, (-webkit-min-device-pixel-ratio: 4) and (max-width: 700px) 50vw, (min-resolution: 3dppx) and (max-width: 700px) 67vw, (-webkit-min-device-pixel-ratio: 3) and (max-width: 700px) 65vw, (min-resolution: 2.5dppx) and (max-width: 700px) 80vw, (-webkit-min-device-pixel-ratio: 2.5) and (max-width: 700px) 80vw, (min-resolution: 2dppx) and (max-width: 700px) 100vw, (-webkit-min-device-pixel-ratio: 2) and (max-width: 700px) 100vw, 484px\" data-testid=\"og\"><\/picture><\/div>\n<figcaption class=\"mv mw mx mg mh my mz be b bf z dw\" data-selectable-paragraph=\"\">Asset and Artifacts Tab in an Experiment<\/figcaption>\n<\/figure>\n<p id=\"9b64\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">Once you have ran the experiment, go to the Assets and Artifacts tab in the dropdown menu. All images, machine learning models, csv, and any other Artifacts will be added to this tab. Please note that you cannot register a ML model directly to Comet\u2019s model store nor to the the Comet\u2019s Artifact store. These tasks will have to be done manually.<\/p>\n<h1 id=\"1aa8\" class=\"pb pc fp be pd pe pf gp pg ph pi gs pj pk pl pm pn po pp pq pr ps pt pu pv pw bj\" data-selectable-paragraph=\"\">Limitations<\/h1>\n<p id=\"cd42\" class=\"pw-post-body-paragraph na nb fp be b gn qp nd ne gq qq ng nh ni qr nk nl nm qs no np nq qt ns nt nu fi bj\" data-selectable-paragraph=\"\">Using Comet for MLflow runs has limitations that are not present in a normal Comet experiment run. Running an existing MLflow experiment can be inefficient since it limit using some of Comet\u2019s best features.<\/p>\n<h2 id=\"5710\" class=\"py pc fp be pd pz qa qb pg qc qd qe pj ni qf qg qh nm qi qj qk nq ql qm qn qo bj\" data-selectable-paragraph=\"\">Nested Flow Runs are Not Supported<\/h2>\n<p id=\"db5f\" class=\"pw-post-body-paragraph na nb fp be b gn qp nd ne gq qq ng nh ni qr nk nl nm qs no np nq qt ns nt nu fi bj\" data-selectable-paragraph=\"\">Comet does not support MLflow nested runs. This means that you cannot use the <em class=\"qz\">with<\/em> statement to run the MLflow code, it must be outside of the with statement. MLflow runs that are enhanced using Comet need to be de-nested.<\/p>\n<p id=\"6354\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">A nested run example is below:<\/p>\n<pre>with mlflow.start_run():\n  '''\n\tML code Here\n\t'''\n        # Model registry does not work with file store\n        if tracking_url_type_store != \"file\":\n            # Register the model\n            # There are other ways to use the Model Registry, which depends on the use case,\n            # please refer to the doc for more information:\n            # &lt;https:\/\/mlflow.org\/docs\/latest\/model-registry.html#api-workflow&gt;\n            mlflow.sklearn.log_model(lr, \"model\", registered_model_name=\"ElasticnetWineModel\")\n        else:\n            mlflow.sklearn.log_model(lr, \"model\")<\/pre>\n<p id=\"e817\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">A de-nested MLflow required to run Comet is below:<\/p>\n<pre>mlflow.start_run()\n'''\nML code here\n'''\n# Model registry does not work with file store\nif tracking_url_type_store != \"file\":\n    # Register the model\n    # There are other ways to use the Model Registry, which depends on the use case,\n    # please refer to the doc for more information:\n    # &lt;https:\/\/mlflow.org\/docs\/latest\/model-registry.html#api-workflow&gt;\n    mlflow.sklearn.log_model(lr, \"model\", registered_model_name=\"ElasticnetWineModel\")\nelse:\n    mlflow.sklearn.log_model(lr, \"model\")\nmlflow.end_run()<\/pre>\n<p id=\"e20c\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">This can make exception handling and resource management difficult, since the <code class=\"cw sc sd se rb b\"><strong class=\"be qu\">with<\/strong><\/code> statement usually encapsulates common preparation and cleanup tasks. It ensures that encapsulated tasks don\u2019t block other processes if an exception is raised, but terminates properly.<\/p>\n<p id=\"7e16\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">This can make MLflow runs enhanced with Comet difficult to use. It may require significantly refactoring the code within a normal Comet experiment flow.<\/p>\n<h2 id=\"df92\" class=\"py pc fp be pd pz qa qb pg qc qd qe pj ni qf qg qh nm qi qj qk nq ql qm qn qo bj\" data-selectable-paragraph=\"\">Manual Versioning of ML Models<\/h2>\n<p id=\"4675\" class=\"pw-post-body-paragraph na nb fp be b gn qp nd ne gq qq ng nh ni qr nk nl nm qs no np nq qt ns nt nu fi bj\" data-selectable-paragraph=\"\"><a class=\"af nv\" href=\"https:\/\/www.comet.com\/site\/using-comet-model-registry\/\" target=\"_blank\" rel=\"noopener ugc nofollow\">Comet\u2019s model registry <\/a>allows you to log, register, version, and deploy machine learning models. This is very important for being able to reproduce and iterate on ML models. In a normal Comet run, models that are uploaded via the model logging method can be registered.<\/p>\n<p id=\"dbfe\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">MLflow experiments enhanced with Comet cannot send machine learning models to this model registry. It will save it as a model in the local drive and will be saved as an Artifact on Comet. Since it is an Artifact, you will not be able to version it.<\/p>\n<p id=\"574d\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">The only way to send a model created by MLflow is to create a second experiment using a normal Comet workflow to upload the model. This can slow down development and deployment time significantly but is a workaround.<\/p>\n<h2 id=\"758f\" class=\"py pc fp be pd pz qa qb pg qc qd qe pj ni qf qg qh nm qi qj qk nq ql qm qn qo bj\" data-selectable-paragraph=\"\">Artifacts Cannot Be Pushed to the Artifact Store<\/h2>\n<p id=\"653f\" class=\"pw-post-body-paragraph na nb fp be b gn qp nd ne gq qq ng nh ni qr nk nl nm qs no np nq qt ns nt nu fi bj\" data-selectable-paragraph=\"\">Artifacts from MLflow experiment run with Comet cannot be saved to the Comet\u2019s Artifact store. Comet\u2019s Artifact store allows you to keep track of versioning of assets: ML model, a flat file, pictures, audio files, etc., allowing the users to use them or retrieve them at any stage of the ML pipeline. It provides a location online with the ability to version Artifacts used.<\/p>\n<p id=\"79db\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">MLflow experiments enhanced with Comet are not included in this feature. While they are uploaded in the Artifacts and Assets area each time an experiment is run, they cannot be saved to the Artifact store.<\/p>\n<p id=\"876f\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">Like the model registry store, the only way to send your Artifacts created in MLflow experiment is to create a second experiment using a normal Comet workflow and uploaded them individually.<\/p>\n<h1 id=\"2785\" class=\"pb pc fp be pd pe pf gp pg ph pi gs pj pk pl pm pn po pp pq pr ps pt pu pv pw bj\" data-selectable-paragraph=\"\">Conclusion<\/h1>\n<p id=\"4c07\" class=\"pw-post-body-paragraph na nb fp be b gn qp nd ne gq qq ng nh ni qr nk nl nm qs no np nq qt ns nt nu fi bj\" data-selectable-paragraph=\"\">MLflow is a great entry level tool for users getting into experiment tracking, but can be limiting for teams. Using Comet for MLflow is a great way to promote teams to work together to version, comment, and view performance of machine learning models. It\u2019s also a great way to maintain existing MLflow experiments.<\/p>\n<p id=\"fb15\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">Comet for MLflow is useful for this transition from MLflow to Comet. However, it is not replacement for the full functionality of Comet, which includes a greater range of features.<\/p>\n<p id=\"294b\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">I hope this article will give you the knowledge and motivation to use Comet for your existing MLflow experiments, and to migrate to Comet for your future experiments.<\/p>\n<p id=\"1e3a\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">Feel free to play with and fork the source code of this article here:<\/p>\n<p id=\"c1ba\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\"><span style=\"text-decoration: underline;\"><strong><a class=\"af nv\" href=\"https:\/\/github.com\/mattblasa\/Enhancing-ML-Flow-with-Comet\" target=\"_blank\" rel=\"noopener ugc nofollow\">Github Repo \u2014 Enhancing MLflow with Comet<\/a><\/strong><\/span><\/p>\n<p id=\"f7aa\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\"><span style=\"text-decoration: underline;\"><strong><a class=\"af nv\" href=\"https:\/\/www.comet.com\/mattblasa\/enhancing-ml-flow\/view\/new\/panels\" target=\"_blank\" rel=\"noopener ugc nofollow\">Comet Experiment Workspace<\/a><\/strong><\/span><\/p>\n<p id=\"95ff\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">If you enjoyed this article, you might also enjoy my other articles at Heartbeat:<\/p>\n<p data-selectable-paragraph=\"\"><a href=\"https:\/\/heartbeat.comet.ml\/integrating-comet-and-azure-databricks-4ec97703a2fe\"><span style=\"text-decoration: underline;\"><strong>Integrating Comet and Azure Databricks<\/strong><\/span><\/a><\/p>\n<p id=\"e478\" class=\"pw-post-body-paragraph na nb fp be b gn nc nd ne gq nf ng nh ni nj nk nl nm nn no np nq nr ns nt nu fi bj\" data-selectable-paragraph=\"\">Thank you for reading! Connect with me on <a class=\"af nv\" href=\"https:\/\/www.linkedin.com\/in\/mblasa\/\" target=\"_blank\" rel=\"noopener ugc nofollow\">LinkedIn<\/a> for more on data science topics.<\/p>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Victor Iaqu, Unsplash. As a data professional, switching machine learning tracking tools happens. It ends up being time consuming work rewriting the code into the new tool. Data scientists are no exception: we often have to rework previous code that was used in experiment tracking. This was the problem I faced in 2021, when I [&hellip;]<\/p>\n","protected":false},"author":102,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"customer_name":"","customer_description":"","customer_industry":"","customer_technologies":"","customer_logo":"","footnotes":""},"categories":[9,7],"tags":[],"coauthors":[200],"class_list":["post-7846","post","type-post","status-publish","format-standard","hentry","category-product","category-tutorials"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v25.9 (Yoast SEO v25.9) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Enhancing MLflow with Comet - Comet<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.comet.com\/site\/blog\/enhancing-mlflow-with-comet\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Enhancing MLflow with Comet\" \/>\n<meta property=\"og:description\" content=\"Victor Iaqu, Unsplash. As a data professional, switching machine learning tracking tools happens. It ends up being time consuming work rewriting the code into the new tool. Data scientists are no exception: we often have to rework previous code that was used in experiment tracking. This was the problem I faced in 2021, when I [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.comet.com\/site\/blog\/enhancing-mlflow-with-comet\/\" \/>\n<meta property=\"og:site_name\" content=\"Comet\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cometdotml\" \/>\n<meta property=\"article:published_time\" content=\"2023-10-06T20:48:19+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-04-24T17:05:55+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*vQjA9xyHMTNHZUJNVILmTg.jpeg\" \/>\n<meta name=\"author\" content=\"Matt Blasa\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@Cometml\" \/>\n<meta name=\"twitter:site\" content=\"@Cometml\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Matt Blasa\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"11 minutes\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Enhancing MLflow with Comet - Comet","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.comet.com\/site\/blog\/enhancing-mlflow-with-comet\/","og_locale":"en_US","og_type":"article","og_title":"Enhancing MLflow with Comet","og_description":"Victor Iaqu, Unsplash. As a data professional, switching machine learning tracking tools happens. It ends up being time consuming work rewriting the code into the new tool. Data scientists are no exception: we often have to rework previous code that was used in experiment tracking. This was the problem I faced in 2021, when I [&hellip;]","og_url":"https:\/\/www.comet.com\/site\/blog\/enhancing-mlflow-with-comet\/","og_site_name":"Comet","article_publisher":"https:\/\/www.facebook.com\/cometdotml","article_published_time":"2023-10-06T20:48:19+00:00","article_modified_time":"2025-04-24T17:05:55+00:00","og_image":[{"url":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*vQjA9xyHMTNHZUJNVILmTg.jpeg","type":"","width":"","height":""}],"author":"Matt Blasa","twitter_card":"summary_large_image","twitter_creator":"@Cometml","twitter_site":"@Cometml","twitter_misc":{"Written by":"Matt Blasa","Est. reading time":"11 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.comet.com\/site\/blog\/enhancing-mlflow-with-comet\/#article","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/blog\/enhancing-mlflow-with-comet\/"},"author":{"name":"Matt Blasa","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/b689e5bf064facf03364f36108b53a7f"},"headline":"Enhancing MLflow with Comet","datePublished":"2023-10-06T20:48:19+00:00","dateModified":"2025-04-24T17:05:55+00:00","mainEntityOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/enhancing-mlflow-with-comet\/"},"wordCount":1911,"publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/enhancing-mlflow-with-comet\/#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*vQjA9xyHMTNHZUJNVILmTg.jpeg","articleSection":["Product","Tutorials"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.comet.com\/site\/blog\/enhancing-mlflow-with-comet\/","url":"https:\/\/www.comet.com\/site\/blog\/enhancing-mlflow-with-comet\/","name":"Enhancing MLflow with Comet - Comet","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/enhancing-mlflow-with-comet\/#primaryimage"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/enhancing-mlflow-with-comet\/#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*vQjA9xyHMTNHZUJNVILmTg.jpeg","datePublished":"2023-10-06T20:48:19+00:00","dateModified":"2025-04-24T17:05:55+00:00","breadcrumb":{"@id":"https:\/\/www.comet.com\/site\/blog\/enhancing-mlflow-with-comet\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.comet.com\/site\/blog\/enhancing-mlflow-with-comet\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/blog\/enhancing-mlflow-with-comet\/#primaryimage","url":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*vQjA9xyHMTNHZUJNVILmTg.jpeg","contentUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*vQjA9xyHMTNHZUJNVILmTg.jpeg"},{"@type":"BreadcrumbList","@id":"https:\/\/www.comet.com\/site\/blog\/enhancing-mlflow-with-comet\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.comet.com\/site\/"},{"@type":"ListItem","position":2,"name":"Enhancing MLflow with Comet"}]},{"@type":"WebSite","@id":"https:\/\/www.comet.com\/site\/#website","url":"https:\/\/www.comet.com\/site\/","name":"Comet","description":"Build Better Models Faster","publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.comet.com\/site\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.comet.com\/site\/#organization","name":"Comet ML, Inc.","alternateName":"Comet","url":"https:\/\/www.comet.com\/site\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","width":310,"height":310,"caption":"Comet ML, Inc."},"image":{"@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/cometdotml","https:\/\/x.com\/Cometml","https:\/\/www.youtube.com\/channel\/UCmN63HKvfXSCS-UwVwmK8Hw"]},{"@type":"Person","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/b689e5bf064facf03364f36108b53a7f","name":"Matt Blasa","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/image\/ef901c28c6d95d1994d584481e592d11","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/10\/1691951148078-96x96.jpg","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/10\/1691951148078-96x96.jpg","caption":"Matt Blasa"},"url":"https:\/\/www.comet.com\/site\/blog\/author\/blasa-matthewyahoo-com\/"}]}},"_links":{"self":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/7846","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/users\/102"}],"replies":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/comments?post=7846"}],"version-history":[{"count":1,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/7846\/revisions"}],"predecessor-version":[{"id":15514,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/7846\/revisions\/15514"}],"wp:attachment":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media?parent=7846"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/categories?post=7846"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/tags?post=7846"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/coauthors?post=7846"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}