{"id":4151,"date":"2022-10-13T14:25:31","date_gmt":"2022-10-13T22:25:31","guid":{"rendered":"https:\/\/live-cometml.pantheonsite.io\/?p=4151"},"modified":"2025-04-24T17:17:09","modified_gmt":"2025-04-24T17:17:09","slug":"fixing-object-detection-models-with-better-data","status":"publish","type":"post","link":"https:\/\/www.comet.com\/site\/blog\/fixing-object-detection-models-with-better-data\/","title":{"rendered":"Fixing Object Detection Models with Better Data"},"content":{"rendered":"\n
\n

Introduction<\/h2>\n

Object detection tasks can be particularly tedious to debug. If you\u2019ve worked with large object detection datasets in the past, chances are you\u2019ve run into incorrectly labelled data or data that\u2019s missing labels that end up killing your evaluation metrics. Identifying these issues usually involves manually inspecting the individual problematic examples in your dataset.<\/p>\n

The other issue with object detection is that these models usually output multiple detection boxes for a given image, and evaluation metrics have to be calculated based on different threshold parameters that control the strictness of our detection criteria.<\/p>\n

\n
\"\"<\/figure>
<\/picture><\/div>\n<\/figure>\n

Applying Non Max Suppression to Predicted Bounding Boxes to produce a Single Bounding Box Prediction<\/p>\n

The most important parameter in this case, is the Intersection over Union (IoU) threshold. This parameter controls the number of returned bounding boxes in the model\u2019s prediction via Non-Max Suppression (NMS), and it also determines how strictly we evaluate whether or not the model has detected a valid object.<\/p>\n

The Mean Average Precision (mAP) and Recall metrics for object detection models are usually reported at a particular IoU threshold.<\/p>\n

The Problem<\/h2>\n

Let\u2019s say you\u2019re evaluating an Aerial Object Detection model based on drone data. You have a sense of how your model is performing at a particular IoU threshold, but you can\u2019t help wonder what would happen if you\u2019d tried a higher threshold value when running the evaluation?<\/p>\n

Additionally, you find that your evaluation dataset has examples of incorrectly labelled data that you\u2019d like to remove. What do you do then?<\/p>\n

Ordinarily, these steps this would involve writing a script to remove bad examples from the dataset, and re-running your evaluation script or notebook with updated parameters. Depending on the size of the dataset and model, this can take quite a while. Also, if you\u2019re making edits to your dataset, you need to be able to track which examples are being removed or updated so that you can ensure you\u2019re not changing the distributions within your dataset.<\/p>\n

Wouldn\u2019t it be nice if we could just upload our data and predictions somewhere, compute evaluation metrics in a more ad-hoc manner, identify bad data points, and update the dataset without having to change or write additional code?<\/p>\n

Wouldn\u2019t it be nice if we could just upload our data and predictions somewhere, compute evaluation metrics in a more ad-hoc manner, identify bad data points, and update the dataset without having to change or write additional code?<\/p><\/blockquote>\n

It used to take weeks of work to build a system like this. But in this post, I\u2019m going to show you how you can set up this system in an afternoon using Comet<\/a> and Aquarium<\/a>.<\/p>\n

Aquarium<\/a> is an ML data management platform that focuses on improving models by providing tooling to explore and improve datasets.<\/p>\n

Comet<\/a> is an MLOps Platform that is designed to help Data Scientists and Teams build better models faster by providing tooling to track, explain, manage, and monitor models in a single place.<\/p>\n

The Proposed Solution<\/h2>\n

You can follow along with the steps to build this solution in this Colab Notebook<\/a><\/p>\n

Our objective is to build an evaluation system that lets us:<\/p>\n