{"id":8073,"date":"2023-11-02T09:28:35","date_gmt":"2023-11-02T17:28:35","guid":{"rendered":"https:\/\/live-cometml.pantheonsite.io\/?p=8073"},"modified":"2025-04-24T17:04:55","modified_gmt":"2025-04-24T17:04:55","slug":"monitoring-your-cv-model-a-beginners-guide-using-kangas-and-comet","status":"publish","type":"post","link":"https:\/\/www.comet.com\/site\/blog\/monitoring-your-cv-model-a-beginners-guide-using-kangas-and-comet\/","title":{"rendered":"Monitoring Your CV Model: A Beginner\u2019s Guide Using Kangas and Comet"},"content":{"rendered":"\n
\n
\n
\n
\n
\n
\"\"<\/figure>
<\/picture><\/div>\n<\/div>\n<\/figure>\n

Computer Vision (CV) is a rapidly growing field that is transforming the way we interact with technology. As more and more companies adopt computer vision models for various applications, it has become increasingly important to monitor the performance of these models over time to ensure they continue to deliver accurate results.<\/p>\n

In this article, we\u2019ll introduce you to the basics of monitoring your CV models using Kangas and Comet, two powerful tools that make it easy to track and analyze the performance of your models. Whether you\u2019re new to computer vision or have some experience under your belt, this beginner\u2019s guide will help you gain a better understanding of the importance of monitoring your models and how to get started with these powerful monitoring tools.<\/p>\n

We\u2019ll cover everything from setting up your environment to tracking key metrics and analyzing the results. By the end of this article, you\u2019ll have a solid understanding of how to monitor your CV models using Kangas and Comet, and be well on your way to delivering more accurate and reliable results.<\/p>\n

About Comet<\/strong><\/h2>\n

Comet is a powerful tool for monitoring and analyzing machine learning experiments. It allows users to track their experiments in real time, visualize the results, and share their findings with others. With Comet, users can quickly identify issues with their models, optimize performance, and collaborate with team members. It\u2019s particularly useful for CV models, where accuracy and performance are critical to success.<\/p>\n

About Kangas<\/strong><\/h2>\n

Kangas is a robust platform for monitoring and analyzing structured and unstructured data. By using Kangas and Comet together, users can get a complete picture of their models\u2019 performance, identify issues quickly, and collaborate with team members to optimize results.<\/p>\n

How to Use Kangas and Comet Together<\/h1>\n

Using Kangas and Comet together provides users with a complete picture of their models\u2019 performance. By tracking key metrics and visualizing the results, users can quickly identify issues that may impact accuracy and optimize performance over time.<\/p>\n

To get started, you\u2019ll need to set up your environment. Then, you can start tracking metrics and visualizing the results using the intuitive user interfaces of both tools. This makes it easy to identify trends and patterns that may impact performance, and collaborate with team members to optimize results.<\/p>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n

\n
\n
\n

The most important thing to keep in mind when building and deploying your model? Understanding your end-goal. Read our interview with ML experts from Stanford, Google, and HuggingFace to learn more<\/a>.<\/p><\/blockquote>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n

\n
\n
\n

About The Dataset<\/strong><\/h2>\n

Today, we\u2019re using the bean dataset, which is a collection of pictures of beans that were captured outdoors with smartphones. It has three classes: two for diseases and one for healthy bean plants. In this blog, we\u2019ll attempt to differentiate between healthy and unhealthy bean leaves. You can get the data here<\/a>.<\/p>\n

Getting Started<\/strong><\/h2>\n

The first step is to install all the needed libraries.<\/p>\n

!pip3 install comet_ml --q\nfrom<\/span> comet_ml import<\/span> Experiment\n\ncomet_ml.login(api_key= \"YOUR-COMET-API-KEY\"<\/span>,\n              project_name= \"my-project\"<\/span>)\nexperiment = comet_ml.Experiment()\n\n!pip install kangas torch datasets transformers scikit-learn --q\nimport<\/span> kangas as<\/span> kg\nimport<\/span> tensorflow as<\/span> tf\nimport<\/span> matplotlib.pyplot as<\/span> plt\nimport<\/span> numpy as<\/span> np\nfrom<\/span> tqdm import<\/span> tqdm\nfrom<\/span> tensorflow import<\/span> keras\nfrom<\/span> tensorflow.keras import<\/span> datasets, layers, models\nfrom<\/span> datasets import<\/span> load_dataset<\/span><\/pre>\n
Next we load the dataset:<\/p>\n
dataset = load_dataset(\"beans\"<\/span>, split=\"train\"<\/span>)<\/span><\/pre>\n
We then instantiate a Kangas DataGrid using our dataset:<\/p>\n
dg = kg.DataGrid(dataset)<\/span><\/pre>\n
To show what\u2019s in our DataGrid, we use the .show()<\/code> method:<\/p>\n
dg.show()<\/span><\/pre>\n
\n
\n
\"\"<\/figure>
<\/picture><\/div>\n<\/div>\n<\/figure>\n
Once the dataset is loaded, we begin to investigate its filtering and group-by capabilities. In particular, we examine the bean leaf images to identify any discernible differences. We observe that the leaves labeled 0 and 1 exhibit signs of disease, while the leaves labeled 2 appear to be healthy.<\/p>\n
\n
\n
\"\"<\/figure>
<\/picture><\/div>\n<\/div>\n<\/figure>\n
The photographs are then organized into each label using the \u201cgroup-by\u201d feature of the Kangas UI.<\/p>\n
We then go to building the ML model so we can track using Comet.<\/p>\n
df = pd.DataFrame(dataset)\nselected_columns = df.columns[:100<\/span>] # Select the first 100 columns<\/span>\ndf = df[selected_columns] # Subset the dataframe to include only the selected columns<\/span>\ny_train = y_train.reshape(-1<\/span>)\ny_train[:5<\/span>]<\/span><\/pre>\n
In the above code block, y_train = y_train.reshape(-1)<\/code> reshapes the array y_train<\/code> to a one-dimensional array.<\/p>\n
The second line of code y_train[:5]<\/code> selects the first five elements of the newly reshaped y_train<\/code> array and returns them.<\/p>\n
We then divide the beans dataset into classes.<\/p>\n
classes = [ 'angular leaf spot'<\/span>, 'bean rust'<\/span>, 'healthy'<\/span>]<\/span><\/pre>\n
The code below defines a function called plot_sample<\/code> that takes three arguments: a numpy array containing image data samples X<\/code>, a numpy array containing labels for each sample in y<\/code>, and an integer index index<\/code> indicating which sample to plot. The function converts the index to an integer, rounds it, and creates a new figure with dimensions 15×2. It then displays the image corresponding to the selected sample and sets the x-axis label to the class label of the selected sample. The class labels are obtained from a dictionary called classes<\/code>.<\/p>\n
def<\/span> plot_sample<\/span>(X, y, index<\/span>):\n    index = np.round<\/span>(index).astype(int<\/span>)\n    plt.figure(figsize = (15<\/span>,2<\/span>))\n    plt.imshow(X[index])\n    plt.xlabel(classes[y[index]])<\/span><\/pre>\n
\n
\"\"<\/figure>
<\/picture><\/div>
Fig 1<\/figcaption><\/figure>\n
After viewing the various categories of leaves we then start building our ML model.<\/p>\n
from<\/span> tensorflow import<\/span> keras\nfrom<\/span> tensorflow.keras import<\/span> layers\n\nmodel = keras.Sequential([\n    layers.Conv2D(filters=32<\/span>, kernel_size=(3<\/span>, 3<\/span>), activation='relu'<\/span>, input_shape=(500<\/span>, 500<\/span>, 3<\/span>)),\n    layers.MaxPooling2D((2<\/span>, 2<\/span>)),\n\n    layers.Conv2D(filters=64<\/span>, kernel_size=(3<\/span>, 3<\/span>), activation='relu'<\/span>),\n    layers.MaxPooling2D((2<\/span>, 2<\/span>)),\n\n    layers.Conv2D(filters=128<\/span>, kernel_size=(3<\/span>, 3<\/span>), activation='relu'<\/span>),\n    layers.MaxPooling2D((2<\/span>, 2<\/span>)),\n\n    layers.Flatten(),\n    layers.Dense(256<\/span>, activation='relu'<\/span>),\n    layers.Dense(3<\/span>, activation='softmax'<\/span>)\n])from<\/span> tensorflow import<\/span> keras\nfrom<\/span> tensorflow.keras import<\/span> layers\n\nmodel = keras.Sequential([\n    layers.Conv2D(filters=32<\/span>, kernel_size=(3<\/span>, 3<\/span>), activation='relu'<\/span>, input_shape=(500<\/span>, 500<\/span>, 3<\/span>)),\n    layers.MaxPooling2D((2<\/span>, 2<\/span>)),\n\n    layers.Conv2D(filters=64<\/span>, kernel_size=(3<\/span>, 3<\/span>), activation='relu'<\/span>),\n    layers.MaxPooling2D((2<\/span>, 2<\/span>)),\n\n    layers.Conv2D(filters=128<\/span>, kernel_size=(3<\/span>, 3<\/span>), activation='relu'<\/span>),\n    layers.MaxPooling2D((2<\/span>, 2<\/span>)),\n\n    layers.Flatten(),\n    layers.Dense(256<\/span>, activation='relu'<\/span>),\n    layers.Dense(3<\/span>, activation='softmax'<\/span>)\n])<\/span><\/pre>\n
To extract important features from input images, the model starts with a convolutional layer that uses 32 filters of size 3×3. ReLU activation function is used to deal with non-linearity. The input shape is set to (500, 500, 3), which indicates that the input image is 500×500 pixels with 3 color channels.<\/p>\n
To reduce the input image size, a max pooling layer with a pool size of 2×2 is added to the model.<\/p>\n
This process is then repeated two more times, with 64 and 128 filters in each convolutional layer. A max pooling layer with a pool size of 2×2 is added after each convolutional layer.<\/p>\n
The output from the convolutional layers is then converted into a 1D feature vector using a flatten layer. Then, two fully connected layers (Dense) are added to the model, with the first layer having 256 nodes and the second layer having 3 nodes. ReLU activation function is used for the first dense layer, and softmax activation function is used for the last layer.<\/p>\n
In multi-class classification problems, the softmax function helps normalize the output and gives a probability distribution across all classes.<\/p>\n
The next step involves compiling the model. This is where the network architecture is configured, and the learning process is initialized with parameters such as the loss function, optimizer, and performance metrics.<\/p>\n
model.compile<\/span>(loss='categorical_crossentropy'<\/span>,\n              optimizer='adam'<\/span>,\n              metrics=['accuracy'<\/span>])<\/span><\/pre>\n
The above code is a crucial step in configuring a neural network model for training and evaluation. It specifies the loss function, optimizer, and performance metrics that will be used during the learning process, and helps to ensure that the model is optimized for the specific problem at hand.<\/p>\n
from<\/span> datasets import<\/span> load_dataset\nfrom<\/span> tensorflow.keras.utils import<\/span> to_categorical\n\n# Load the \"beans\" dataset<\/span>\ndataset = load_dataset(\"beans\"<\/span>, split=\"train[:80%]\"<\/span>)\nval_dataset = load_dataset(\"beans\"<\/span>, split=\"train[80%:]\"<\/span>)\n\n# Convert the input images to NumPy arrays<\/span>\nX_train = np.array([np.array(image) for<\/span> image in<\/span> dataset['image'<\/span>]])\nX_val = np.array([np.array(image) for<\/span> image in<\/span> val_dataset['image'<\/span>]])\n\n# Convert the target labels to NumPy arrays and one-hot encode them<\/span>\ny_train = to_categorical(np.array(dataset['labels'<\/span>]))\ny_val = to_categorical(np.array(val_dataset['labels'<\/span>]))\n\n# Compile the model<\/span>\nmodel.compile<\/span>(loss='categorical_crossentropy'<\/span>,\n              optimizer='adam'<\/span>,\n              metrics=['accuracy'<\/span>])\n\n# Train the model<\/span>\nhistory = model.fit(X_train, y_train, batch_size=32<\/span>, epochs=10<\/span>, validation_data=(X_val, y_val))<\/span><\/pre>\n
The above code loads the dataset, splits it into training and validation data, converts the input images and target labels to NumPy arrays, compiles the model with the chosen optimizer, loss function, and metrics, and finally trains the model using the training data and validates it on the validation data. The training process is saved to the \u2018history\u2019 variable. This code demonstrates how to train a machine learning model on the \u201cbeans\u201d dataset using the Keras library. It loads the dataset, splits it into training and validation data, converts the input images and target labels to NumPy arrays, compiles the model with the chosen optimizer, loss function, and metrics, and finally trains the model using the training data and validates it on the validation data. The training process is saved to the \u2018history\u2019 variable.<\/p>\n
The next step involves evaluating the performance of a machine learning model on a validation dataset and logging the evaluation metrics<\/p>\n
loss, accuracy = model.evaluate(X_val,y_val)\nprint<\/span>(\"Loss: \"<\/span>, loss)\nprint<\/span>(\"Accuracy: \"<\/span>, accuracy)\n\nexperiment.log_metric(\"Loss\"<\/span>, loss, step=None<\/span>, include_context=True<\/span>)\nexperiment.log_metric(\"Accuracy\"<\/span>, accuracy, step=None<\/span>, include_context=True<\/span>)<\/span><\/pre>\n
The above code logs the evaluation metrics using a library called Comet for experiment tracking. The model.evaluate()<\/code> function calculates the loss and accuracy of the model on the validation dataset. The calculated loss and accuracy values are then logged using the experiment.log_metric()<\/code> function of the Comet library. This function logs the metric name, value, and context of the current experiment if required. Overall, the code evaluates the model’s performance, prints the results, and logs them using a library for experiment tracking.<\/p>\n
We then save our model and log it to Comet.<\/p>\n
model.save('model'<\/span>)\nexperiment.log_model(model, 'model'<\/span>)<\/span><\/pre>\n
Finally, we view our logged model in comet.<\/p>\n
\n
\n