{"id":2422,"date":"2021-08-05T13:30:46","date_gmt":"2021-08-05T21:30:46","guid":{"rendered":"https:\/\/live-cometml.pantheonsite.io\/blog\/debugging-your-machine-learning-models-with-comet-artifacts\/"},"modified":"2021-08-05T13:30:46","modified_gmt":"2021-08-05T21:30:46","slug":"debugging-your-machine-learning-models-with-comet-artifacts","status":"publish","type":"post","link":"https:\/\/www.comet.com\/site\/blog\/debugging-your-machine-learning-models-with-comet-artifacts\/","title":{"rendered":"Debugging Your Machine Learning Models with Comet Artifacts"},"content":{"rendered":"\n

In this post, we\u2019ll introduce Comet Artifacts<\/a>, a new tool that provides Machine Learning teams with a convenient way to log, version, browse, and access data from all parts of their experimentation pipelines.<\/p>\n\n\n\n

This post is divided into four sections:\u00a0<\/p>\n\n\n\n

    \n
  1. Challenge – We present the issues that PetCam, a fictional company, is having while trying to build a Pet Detection model.<\/li>\n
  2. An Introduction to Artifacts as a Solution to PetCam’s problems<\/li>\n
  3. Step-By-Step Guide: How PetCam can use Artifacts to debug their model<\/li>\n
  4. Summary Discussion<\/li>\n<\/ol>\n\n\n\n

    You can try out the code for this example in this Colab Notebook<\/a>.<\/h3>\n\n\n\n

    1. The Challenge<\/h2>\n\n\n\n

    PetCam is looking to build an app that identifies pets in photos. They build a\u00a0 classification model and tune a bunch of different hyperparameters, but are unable to meet their accuracy requirements.\u00a0<\/em><\/p>\n\n\n\n

    To determine the root cause of this performance issue, PetCam creates a derivative dataset of the images that were difficult for all existing model versions.\u00a0<\/em><\/p>\n\n\n\n

    How can PetCam iterate on both models and the derived datasets created by the models, in a way that\u2019s reproducible and managed without losing track of their work?<\/em><\/p>\n\n\n\n

    In this scenario, we may want to track and reproduce the lineage of experiments, models, and datasets that are created iteratively over time.<\/p>\n\n\n\n

    2. Artifacts as a Solution<\/h2>\n\n\n\n

    We\u2019ve built Comet Artifacts to help Machine Learning teams solve the challenges like the one outlined above\u2014iterating on datasets and tracking pipelines in which data generated from one experiment is fed into another experiment.\u00a0<\/p>\n\n\n\n

    This need to track the datasets used or created by an experiment or model arises frequently. In many cases, ML teams must:<\/p>\n\n\n\n