{"id":8076,"date":"2023-11-02T09:42:52","date_gmt":"2023-11-02T17:42:52","guid":{"rendered":"https:\/\/live-cometml.pantheonsite.io\/?p=8076"},"modified":"2025-04-24T17:04:53","modified_gmt":"2025-04-24T17:04:53","slug":"monitoring-a-convolutional-neural-network-cnn-in-comet","status":"publish","type":"post","link":"https:\/\/www.comet.com\/site\/blog\/monitoring-a-convolutional-neural-network-cnn-in-comet\/","title":{"rendered":"Monitoring A Convolutional Neural Network (CNN) in Comet"},"content":{"rendered":"\n\n\n\n\n
\n
\n
\n
\n
\n
\"\"<\/figure>
<\/picture><\/div>\n<\/div>
Photo from nmedia on Shutterstock.com<\/figcaption><\/figure>\n

Introduction<\/h2>\n

Image classification is a task that involves training a neural network to recognize and classify items in images. A dataset of labeled images is used to train the network, with each image given a particular class or label. Thousands or even millions of photos make up the normal size of the dataset needed to train the model. Before being fed into the network, the photos are pre-processed and shrunk to the same size.<\/p>\n

A convolutional neural network (CNN) is primarily used for image classification. Convolutional, pooling, and fully linked layers are some of the layers that make up a CNN. The pooling layers are used to shrink the spatial dimensions of the image while preserving the key features, whereas the convolutional layers are in charge of identifying patterns and features in the image. The image is subsequently classified using the convolutional and pooling layers\u2019 retrieved features by the fully connected layers.<\/p>\n

Once trained, the model can be used to categorize new, unseen photos. The network processes the image, and the result is a probability distribution across all potential classifications. The anticipated label for the image is the class with the highest likelihood.<\/p>\n

Numerous applications, including object identification, image search, and image annotation, can benefit from image categorization. For instance, picture classification can be used in self-driving automobiles to recognize and classify items like vehicles, people, and traffic signs so that the vehicle can travel safely. Image classification can be used in search engines to index and retrieve photos based on their content. Additionally, picture categorization can be used in photo management software to automatically identify photographs with pertinent labels, making it simpler to find and organize them.<\/p>\n

Comet<\/h1>\n

Comet is a machine-learning experimentation tool that assists you in keeping track of your machine-learning experiments. Another important feature of Comet is that it allows users to track or monitor the performance of a model by utilizing the various tools accessible on the Comet platform. You can learn more about Comet here<\/a>.<\/p>\n

In this tutorial, we will learn how to keep track of our image classification model in Comet. After building our model, we will log it onto the Comet experimentation website or platform to follow and track the progress of our experiment. So let\u2019s get started with it.<\/p>\n

Prerequisites<\/h1>\n

In order to focus on the model monitoring function in Comet, we will be using a code tutorial from CodeBasics<\/a> to build, train, and test our models. You can find the original code in this repo here<\/a>, or follow along with my modified version here<\/a>.<\/p>\n

Using one of the following lines at the command prompt, you can install the Comet library on your machine if it isn\u2019t already present. Be aware that pip is probably what you should use if you\u2019re installing packages directly into a Colab notebook or another environment that makes use of virtual machines.<\/p>\n

pip install comet_ml<\/span><\/pre>\n
\u2014 or \u2014<\/p>\n
conda install -c comet_ml<\/span><\/pre>\n
About The Dataset.<\/h2>\n
The CIFAR-10 dataset consists of 6,000 images per class in 10 classes totaling 60,000 32×32 color images. 10,000 test photos and 50,000 training images are available.<\/p>\n
Five training batches and one test batch, each with 10,000 photos, are created from the dataset. Exactly 1,000 randomly chosen photos from each class make up the test batch. The remaining images are distributed across the training batches in random order, but certain training batches can have a disproportionate number of images from a particular class. The training batches are made up of 5,000 photographs from each class together. You can get the dataset here<\/a>.<\/p>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n
\n
\n
\n
Want to see the evolution of AI-generated art projects? Visit our public project to see time-lapses, experiment evolutions, and more<\/a>!<\/p><\/blockquote>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n
\n
\n
\n
Understanding the goal<\/strong><\/h2>\n
    \n
  1. Build a neural network to recognize different types of images assigned to it.<\/li>\n
  2. Use both the CNN and ANN and see the network with the most accuracy.<\/li>\n
  3. Detect layers and patterns in the dataset.<\/li>\n
  4. Monitor ML model performance after training in the Comet dashboard, including its accuracy and loss.<\/li>\n
  5. Monitor hardware performance.<\/li>\n<\/ol>\n
    Exploring the model:<\/h2>\n
    Let\u2019s import and Install the needed libraries<\/p>\n
    import<\/span> tensorflow as<\/span> tf\nfrom<\/span> tensorflow.keras import<\/span> datasets,layers,models\nimport<\/span> matplotlib as<\/span> plt\nimport<\/span> pandas as<\/span> pd\nimport<\/span> numpy as<\/span> np<\/span><\/pre>\n
    The dataset is subsequently loaded, and the test and training datasets are loaded simultaneously.<\/p>\n
    (X_train,y_train),(X_test,y_test)= datasets.cifar10.load_data()<\/span><\/pre>\n
    Next, we examine the training dataset\u2019s form. The dataset\u2019s form also reveals that we have 50,000 training samples, a 32 by 32 image, and that the third digit stands for the RGB channel.<\/p>\n
    X_train.shape<\/span><\/pre>\n
    We check the test samples after examining the train sample shapes.<\/p>\n
    X_test.shape <\/span><\/pre>\n
    We also define our classes:<\/p>\n
    classes = [\"airplane\"<\/span>,\"automobile\"<\/span>,\"bird\"<\/span>,\"cat\"<\/span>,\"deer\"<\/span>,\"dog\"<\/span>,\"frog\"<\/span>,\"horse\"<\/span>,\"ship\"<\/span>,\"truck\"<\/span>]<\/span><\/pre>\n
    Then, we create the following function:<\/p>\n
    def<\/span> plot_sample<\/span>(X, y, index<\/span>):\n    plt.figure(figsize = (15<\/span>,2<\/span>))\n    plt.imshow(X[index])\n    plt.xlabel(classes[y[index]])<\/span><\/pre>\n
    We can test the function as follows:<\/p>\n
    plot_sample(X_train, y_train, 0<\/span>)<\/span><\/pre>\n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/figure>\n
    We can try another sample image:<\/p>\n
    plot_sample(X_train, y_train, 0<\/span>)<\/span><\/pre>\n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/figure>\n
    Normalize the pictures to a value between 0 and 1. The image has three channels (R, G, and B), each of which can have a value between 0 and 255. So we must divide it by 255 to normalize it in the 0\u20131 range.<\/p>\n
    X_train = X_train \/ 255.0<\/span>\nX_test = X_test \/ 255.0<\/span><\/span><\/pre>\n
    Model Building<\/h2>\n
    We then construct an artificial neural network first, followed by a convolutional neural network, and we can test the effectiveness of both models as well as compare their benefits and drawbacks.<\/p>\n
    ann = models.Sequential([\n        layers.Flatten(input_shape=(32<\/span>,32<\/span>,3<\/span>)),\n        layers.Dense(3000<\/span>, activation='relu'<\/span>),\n        layers.Dense(1000<\/span>, activation='relu'<\/span>),\n        layers.Dense(10<\/span>, activation='softmax'<\/span>)\n    ])\n\nann.compile<\/span>(optimizer='SGD'<\/span>,\n              loss='sparse_categorical_crossentropy'<\/span>,\n              metrics=['accuracy'<\/span>])\n\nann.fit(X_train, y_train, epochs=5<\/span>)<\/span><\/pre>\n
    \n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/div>\n<\/figure>\n
    We can therefore see from the code above that there are four layers. The first layer accepts images with a 32 by 32 pixel dimension, while the second and third levels are deep layers with 3000 and 1000 neurons, respectively. The fourth layer then includes 10 categories to reflect the 10 sample images that are included in our dataset. After executing the code, we can see that the ANN\u2019s accuracy on training samples is quite low\u2014about 10%.<\/p>\n
    We then try it on test samples to see if the results get better.<\/p>\n
    ann.evaluate(X_test,y_test)<\/span><\/pre>\n
    We can see from the review above that ANN continues to perform poorly. The last step is to print a classification report that displays the recall and precision for each sample image.<\/p>\n
    from<\/span> sklearn.metrics import<\/span> confusion_matrix , classification_report\nimport<\/span> numpy as<\/span> np\ny_pred = ann.predict(X_test)\ny_pred_classes = [np.argmax(element) for<\/span> element in<\/span> y_pred]\n\nprint<\/span>(\"Classification Report: \\n\"<\/span>, classification_report(y_test, y_pred_classes))<\/span><\/pre>\n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/figure>\n
    The precision and recall scores for each of the sample photos can then be examined (0\u20139). A sample image with index 1 has, for instance, the highest precision scores when compared to the others, as can be seen by looking at it.<\/p>\n
    The next step is to utilize build and train a CNN:<\/p>\n
    model = models.Sequential([\n    layers.Conv2D(filters=32<\/span>, kernel_size=(3<\/span>, 3<\/span>), activation='relu'<\/span>, input_shape=(32<\/span>, 32<\/span>, 3<\/span>)),\n    layers.MaxPooling2D((2<\/span>, 2<\/span>)),\n\n    layers.Conv2D(filters=64<\/span>, kernel_size=(3<\/span>, 3<\/span>), activation='relu'<\/span>),\n    layers.MaxPooling2D((2<\/span>, 2<\/span>)),\n\n    layers.Flatten(),\n    layers.Dense(64<\/span>, activation='relu'<\/span>),\n    layers.Dense(10<\/span>, activation='softmax'<\/span>)\n])\n\nmodel.compile<\/span>(optimizer='adam'<\/span>,\n              loss='sparse_categorical_crossentropy'<\/span>,\n              metrics=['accuracy'<\/span>])<\/span><\/pre>\n
    We then run our loss and metrics tests using the Adam optimizer, which is known for its versatility.<\/p>\n
    model.fit(X_train, y_train, epochs=10<\/span>, verbose=1<\/span>,batch_size=128<\/span>)<\/span><\/pre>\n
    We train the model for 10 epochs and get 70% accuracy, which is significantly higher than ANN.<\/p>\n
    \n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/div>\n<\/figure>\n
    To further our model building, we also try to evaluate the test dataset to see if we can get a high accuracy score.<\/p>\n
    model.evaluate(X_test,y_test)<\/span><\/pre>\n
    \n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/div>\n<\/figure>\n
    The accuracy we have is 65%, which is above average, as we can then see.
    \nSo now that our CNN model is prepared, all that is left to do is plot a sample image and wait for the model to identify it.<\/p>\n
    plot_sample(X_test, y_test,6<\/span>)<\/span><\/pre>\n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/figure>\n
    We can see that this is obviously an automobile; let\u2019s see if our model can identify it as such.<\/p>\n
    classes[y_classes[6<\/span>]]<\/span><\/pre>\n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/figure>\n
    Monitoring In Comet<\/h2>\n
    In this section, I\u2019ll be revealing the step-by-step process of logging our experiments in Comet.<\/p>\n
    The first step involves Installing comet which we have done at the beginning<\/p>\n
    !pip3 install comet_ml<\/span><\/pre>\n
    The next step is to import experiments from Comet to enable us to log on to the Comet platform.<\/p>\n
    from<\/span> comet_ml import<\/span> experiment<\/span><\/pre>\n
    We then move forward by creating an experiment and also setting the auto-logging features and other important features to true.<\/p>\n
    # Create an experiment<\/span>\nexperiment = comet_ml.Experiment(\n    project_name=\"image classification\"<\/span>,\n    workspace=\"<olujerry>\"<\/span>,\n     api_key=\"API_KEY\"<\/span>,\n    auto_metric_logging=True<\/span>,\n    auto_param_logging=True<\/span>,\n    auto_histogram_weight_logging=True<\/span>,\n    auto_histogram_gradient_logging=True<\/span>,\n    auto_histogram_activation_logging=True<\/span>,\n    log_code=True<\/span>\n)<\/span><\/pre>\n
    The next step involves logging important features that we want to monitor on the Comet platform.<\/p>\n
    Using the experiment, we record some parameters using log_parameters(params)<\/code>:<\/p>\n
    batch_size = 128<\/span>\nepochs = 10<\/span>\nEMBEDDING_SIZE = 50<\/span>\nlearning_rate = 0.001<\/span>\n\nparams={\n    \"batch_size\"<\/span>:batch_size,\n    \"epochs\"<\/span>:epochs,\n    \"embed_size\"<\/span>:EMBEDDING_SIZE,\n    \"optimizer\"<\/span>:\"Adam\"<\/span>,\n    \"learning_rate\"<\/span>:learning_rate,\n    \"loss\"<\/span>:\"sparse_categorical_crossentropy\"<\/span>,\n    }\n\nexperiment.log_parameters(params)<\/span><\/pre>\n
    We then log the accuracy and loss of our model using the experiment.log_metric<\/code> method:<\/p>\n
    loss, accuracy = model.evaluate(X_test,y_test)\nprint<\/span>(\"Loss: \"<\/span>, loss)\nprint<\/span>(\"Accuracy: \"<\/span>, accuracy)\n\nexperiment.log_metric(\"Loss\"<\/span>, loss, step=None<\/span>, include_context=True<\/span>)\nexperiment.log_metric(\"Accuracy\"<\/span>, accuracy, step=None<\/span>, include_context=True<\/span>)<\/span><\/pre>\n
    Finally, before viewing our logged model on the Comet platform we log the saved model and then end the experiment afterward, as we are using an interactive notebook.<\/p>\n
    experiment.log_model(model, 'model'<\/span>)\nexperiment.end()<\/span><\/pre>\n
    The next step involves checking out the performance of our model on the Comet platform.<\/p>\n
    \n
    \n