{"id":6559,"date":"2023-06-29T11:23:23","date_gmt":"2023-06-29T19:23:23","guid":{"rendered":"https:\/\/live-cometml.pantheonsite.io\/?p=6559"},"modified":"2025-04-24T17:15:17","modified_gmt":"2025-04-24T17:15:17","slug":"real-time-object-detection-on-raspberry-pi-using-opencv-dnn","status":"publish","type":"post","link":"https:\/\/www.comet.com\/site\/blog\/real-time-object-detection-on-raspberry-pi-using-opencv-dnn\/","title":{"rendered":"Real-Time Object Detection on Raspberry Pi Using OpenCV DNN"},"content":{"rendered":"\n
\n
\"\"<\/figure>
\n
<\/picture><\/figure>\n<\/div>\n
\n
\n

Running deep learning models is computationally expensive. And when it comes to image processing with computer vision,<\/a> the first thing that comes to mind is high-end GPUs\u2014think the 1080ti and now the 2080ti.<\/p>\n

\n
\"\"<\/figure>
<\/picture><\/div>\n<\/figure>\n

But it\u2019s hard to run computer vision models on edge devices like Raspberry Pi, and making a portable solution is difficult with deep learning libraries like TensorFlow<\/a> or PyTorch<\/a>.<\/p>\n

For this task, it\u2019s almost compulsory to add OpenCV<\/a> to help pre-process data. And the good news is that OpenCV itself includes a deep neural network module, known as OpenCV DNN<\/a>. It runs much faster than other libraries, and conveniently, it only needs OpenCV in the environment. As a result, OpenCV DNN can run on a CPU\u2019s computational power with great speed.<\/p>\n

The best use case of OpenCV DNN is performing real-time object detection on a Raspberry Pi. This process can run in any environment where OpenCV can be installed and doesn’t depend on the hassle of installing deep learning libraries with GPU support. As such, this tutorial isn\u2019t centered on Raspberry Pi\u2014you can follow this process for any environment with OpenCV.<\/p>\n

How Does Object Detection with OpenCV DNN Work?<\/h1>\n

Previously, I wrote this piece:<\/p>\n

Without TensorFlow: Web app with an Object Detection API in Heroku and OpenCV [LINK]<\/a><\/p><\/blockquote>\n

While writing the above article, I realized there are lots of code examples available online, but I couldn\u2019t find any output analysis using OpenCV DNN for object detection. So I figured, why not explore the OpenCV DNN module?<\/p>\n

So in this tutorial, we\u2019ll be exploring how object detection works with OpenCV DNN and MobileNet-SSD <\/a>(in terms of inference<\/em>).<\/p>\n

We\u2019ll be using:<\/p>\n

    \n
  1. Python 3<\/li>\n
  2. OpenCV [Latest version]<\/li>\n
  3. MobileNet-SSD v2<\/li>\n<\/ol>\n

    OpenCV DNN supports models trained from various frameworks like Caffe and TensorFlow. It also supports various networks architectures based on YOLO,<\/a> MobileNet-SSD, Inception-SSD, Faster-RCNN Inception,Faster-RCNN ResNet, and Mask-RCNN Inception.<\/p>\n

    Because OpenCV supports multiple platforms (Android, Raspberry Pi) and languages (C++, Python, and Java), we can use this module for development on many different devices.<\/p>\n

    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/figure>\n

    Why OpenCV DNN?<\/h2>\n

    OpenCV DNN runs faster inference than the TensorFlow object detection API <\/a>with higher speed and low computational power. We will see the performance comparison in a future blog post.<\/p>\n

    Why MobileNet-SSD?<\/strong><\/h2>\n

    MobileNet-SSD can easily be trained with the TensorFlow-Object-Detection-API, Lightweight.<\/p>\n

    Check out the official docs<\/a> for more.<\/p>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n

    \n
    \n
    \n

    Getting Started<\/h1>\n

    First things first, here\u2019s a GitHub repo<\/a> I created that allows you to explore this module:<\/p>\n

    Installation<\/h2>\n

    Installing OpenCV for python<\/p>\n

    pip3 install opencv-python<\/code><\/p>\n

    Download the pre-trained model from the above link.<\/p>\n

    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/figure>\n

    We\u2019ll be using MobileNet-SSD v2 for our object detection model, as it\u2019s more popular\u2014let\u2019s download its weights and config.<\/p>\n

    From the weights folder (after unzipping), we use the frozen_inference_graph.pb<\/code> file.<\/p>\n

    Project Structure<\/h2>\n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/figure>\n

    (I\u2019m using virtualenv for this tutorial, so there is venv, but this isn\u2019t mandatory.)<\/p>\n

    Lets code<\/strong><\/h2>\n

    All of the code can be found in main.py<\/code><\/p>\n

    classNames is a dictionary that contains the 90 objects trained in the model and also in the background <\/span><\/pre>\n
    Loading models (Those we have downloaded)<\/strong><\/h2>\n
    model = cv2.dnn.readNetFromTensorflow('models\/frozen_inference_graph.pb', 'models\/ssd_mobilenet_v2_coco_2018_03_29.pbtxt')<\/span><\/pre>\n
    We need an image to detect objects (these can be captured as frames from live video)<\/strong><\/p><\/blockquote>\n
    For the purposes of this tutorial, let\u2019s use this image<\/a>:<\/p>\n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/figure>\n
    Download the image into the code directory; then read the image with OpenCV and show it:<\/p>\n
    image = cv2.imread(\"image.jpeg\")\ncv2.imshow('image',image)\ncv2.waitKey(0)\ncv2.destroyAllWindows()<\/span><\/pre>\n
    Feeding the image to the network<\/strong><\/h2>\n
    To feed image into the network, we have to convert the image to a blob<\/a>. A blob is a pre-processed image that serves as the input. Pre-processing techniques<\/a> like resizing according to the model, color swapping, cropping, and color channels mean subtraction (normalizing color channel values by subtracting a mean value).<\/p>\n
    model.setInput(cv2.dnn.blobFromImage(image, size=(300, 300), swapRB=True))<\/span><\/pre>\n
    We’re resizing the image to 300 x 300 as our pre-trained model supports and swapping the color channels from BGR to RGB. OpenCV reads in BGR, while RGB is commonly used in model training. RGB is more popular.<\/p>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n
    <\/div>\n\n\n\n
    \n
    \n
    \n
    Feeding forward the model<\/strong><\/h2>\n
    output = model.forward()<\/span><\/pre>\n
    We got the output, so now we need to understand it. For the input image we have given, the shape of the output matrix is (1, 1, 100, 7), Our main concern is output [0,0,:,:]<\/p>\n
    output[0,0,:,:].shape is (100, 7)<\/span><\/pre>\n
    Thresholding Results<\/strong><\/h2>\n
    for detection in output[0,0,:,:]:\n    confidence = detection[2]<\/span><\/pre>\n
    Each detection output gives a predicted confidence in a range of 0 to 1. But most of them are false positive (falsely detected). So we\u2019ll keep only objects of higher confidence. For that, let\u2019s set a threshold of .5. We\u2019ll consider everything up to that threshold when drawing the bounding box around the image.<\/p>\n
    if confidence > .5:<\/span><\/pre>\n
    For the predicted class, we get the id. From the id, we\u2019ll get the class name from the classNames<\/code> dictionary.<\/p>\n
    class_id = detection[1]<\/span><\/pre>\n
    Converting id to class name<\/strong><\/h2>\n
    This function below takes class_id<\/code> and returns the class name according to the id from classNames<\/code><\/p>\n
    def id_class_name(class_id, classes):\n    for key,value in classes.items():\n        if class_id == key:\n            return value<\/span><\/pre>\n
    In this loop, we\u2019re printing Class id, confidence, and class name for debugging purposes before drawing the box.<\/p>\n
    for detection in output[0, 0, :, :]:\n    confidence = detection[2]\n    if confidence > .5:\n        class_id = detection[1]\n        print(str(str(class_id) + \" \" + str(detection[2])  + \" \" + id_class_name(class_id,classNames)))<\/span><\/pre>\n
    This is the output we got from the above code:<\/p>\n
    1.0 0.6377985 person
    \n18.0 0.84042233 dog<\/code><\/p>\n
    The Bounding Box<\/strong><\/h2>\n
    It’s time to draw the box in the image. To draw the bounding box in the image for the predicted object, we need x<\/code>, y<\/code>, width<\/code>, and height<\/code>.<\/p>\n
    box_x=detection[3]\nbox_y=detection[4]\nbox_width=detection[5]\nbox_height=detection[6]<\/span><\/pre>\n
    But we need to scale the values of the box according to our image height and width. But the image is 3 dimensional, as it also includes color channels, and we\u2019re only taking height and width. So we skip the color channel input with \u201c_\u201d<\/p>\n
    image_height, image_width, _ = image.shape<\/span><\/pre>\n
    After scaling<\/strong><\/h2>\n
    box_x = detection[3] * image_width\nbox_y = detection[4] * image_height\nbox_width = detection[5] * image_width\nbox_height = detection[6] * image_height<\/span><\/pre>\n
    Once we\u2019ve scaled the image, we have to draw the box with those new values in the image<\/p>\n
    cv2.rectangle(image, (int(box_x), int(box_y)), (int(box_width), int(box_height)), (23, 230, 210), thickness=1)<\/span><\/pre>\n
    OpenCV\u2019s rectangle<\/code> function takes arguments for the image input, rectangle x, rectangle y, rectangle width, and rectangle height. It also takes color and thickness.<\/p>\n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/figure>\n
    Adding Text<\/strong><\/h2>\n
    Now that we\u2019ve drawn the bounding boxes, let\u2019s add the class text<\/code> in the box:<\/p>\n
    cv2.putText(image,class_name ,(int(box_x), int(box_y+.05*image_height)),cv2.FONT_HERSHEY_SIMPLEX,(.005*image_width),(0, 0, 255))\n<\/span><\/pre>\n
    OpenCV\u2019s putText<\/code> function takes arguments for image, text, starting x, starting y, font type, font size, and text color. In the above code snippet, We\u2019ve scaled the starting y<\/code> and font size<\/code> according to the image dimensions.<\/p>\n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/figure>\n
    Voil\u00e0! Now we\u2019ve got our desired bounding box in the detected objects, and we\u2019ve added labels to each of them.<\/p>\n
    Conclusion<\/h1>\n
    My hope is that this tutorial has provided an understanding of how we can use the OpenCV DNN module for object detection. And with MobileNet-SSD inference, we can use it for any kind of object detection use case or application.<\/p>\n
    Now that we have an understanding of the output matrix, we can use the output values according to our application\u2019s need. It gets more fun when you run a custom-trained model\u2014maybe we can see this in a future blog post!<\/p>\n
    Discuss this post on <\/strong>Hacker News<\/strong><\/a> and <\/strong>Reddit<\/strong><\/a>.<\/strong><\/p>\n
    Find me:<\/p>\n
    GitHub: https:\/\/github.com\/rdeepc<\/a><\/p>\n
    LinkedIn: https:\/\/www.linkedin.com\/in\/saumyashovanroy\/<\/a><\/p>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"
    Running deep learning models is computationally expensive. And when it comes to image processing with computer vision, the first thing that comes to mind is high-end GPUs\u2014think the 1080ti and now the 2080ti. But it\u2019s hard to run computer vision models on edge devices like Raspberry Pi, and making a portable solution is difficult with […]<\/p>\n","protected":false},"author":44,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"customer_name":"","customer_description":"","customer_industry":"","customer_technologies":"","customer_logo":"","footnotes":""},"categories":[6],"tags":[],"coauthors":[152],"class_list":["post-6559","post","type-post","status-publish","format-standard","hentry","category-machine-learning"],"yoast_head":"\nReal-Time Object Detection on Raspberry Pi Using OpenCV DNN - Comet<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.comet.com\/site\/blog\/real-time-object-detection-on-raspberry-pi-using-opencv-dnn\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Real-Time Object Detection on Raspberry Pi Using OpenCV DNN\" \/>\n<meta property=\"og:description\" content=\"Running deep learning models is computationally expensive. And when it comes to image processing with computer vision, the first thing that comes to mind is high-end GPUs\u2014think the 1080ti and now the 2080ti. But it\u2019s hard to run computer vision models on edge devices like Raspberry Pi, and making a portable solution is difficult with […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.comet.com\/site\/blog\/real-time-object-detection-on-raspberry-pi-using-opencv-dnn\/\" \/>\n<meta property=\"og:site_name\" content=\"Comet\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cometdotml\" \/>\n<meta property=\"article:published_time\" content=\"2023-06-29T19:23:23+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-04-24T17:15:17+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/miro.medium.com\/v2\/resize:fit:5616\/1*LyjNlbtMaXBWGEFzh2wArQ.jpeg\" \/>\n<meta name=\"author\" content=\"Saumya Shovan Roy\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@Cometml\" \/>\n<meta name=\"twitter:site\" content=\"@Cometml\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Saumya Shovan Roy\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Real-Time Object Detection on Raspberry Pi Using OpenCV DNN - Comet","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.comet.com\/site\/blog\/real-time-object-detection-on-raspberry-pi-using-opencv-dnn\/","og_locale":"en_US","og_type":"article","og_title":"Real-Time Object Detection on Raspberry Pi Using OpenCV DNN","og_description":"Running deep learning models is computationally expensive. And when it comes to image processing with computer vision, the first thing that comes to mind is high-end GPUs\u2014think the 1080ti and now the 2080ti. But it\u2019s hard to run computer vision models on edge devices like Raspberry Pi, and making a portable solution is difficult with […]","og_url":"https:\/\/www.comet.com\/site\/blog\/real-time-object-detection-on-raspberry-pi-using-opencv-dnn\/","og_site_name":"Comet","article_publisher":"https:\/\/www.facebook.com\/cometdotml","article_published_time":"2023-06-29T19:23:23+00:00","article_modified_time":"2025-04-24T17:15:17+00:00","og_image":[{"url":"https:\/\/miro.medium.com\/v2\/resize:fit:5616\/1*LyjNlbtMaXBWGEFzh2wArQ.jpeg","type":"","width":"","height":""}],"author":"Saumya Shovan Roy","twitter_card":"summary_large_image","twitter_creator":"@Cometml","twitter_site":"@Cometml","twitter_misc":{"Written by":"Saumya Shovan Roy","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.comet.com\/site\/blog\/real-time-object-detection-on-raspberry-pi-using-opencv-dnn\/#article","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/blog\/real-time-object-detection-on-raspberry-pi-using-opencv-dnn\/"},"author":{"name":"Saumya Shovan Roy","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/d3d1005b4f1f4bceea8f5e63f2ce8933"},"headline":"Real-Time Object Detection on Raspberry Pi Using OpenCV DNN","datePublished":"2023-06-29T19:23:23+00:00","dateModified":"2025-04-24T17:15:17+00:00","mainEntityOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/real-time-object-detection-on-raspberry-pi-using-opencv-dnn\/"},"wordCount":1092,"publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/real-time-object-detection-on-raspberry-pi-using-opencv-dnn\/#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:5616\/1*LyjNlbtMaXBWGEFzh2wArQ.jpeg","articleSection":["Machine Learning"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.comet.com\/site\/blog\/real-time-object-detection-on-raspberry-pi-using-opencv-dnn\/","url":"https:\/\/www.comet.com\/site\/blog\/real-time-object-detection-on-raspberry-pi-using-opencv-dnn\/","name":"Real-Time Object Detection on Raspberry Pi Using OpenCV DNN - Comet","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/real-time-object-detection-on-raspberry-pi-using-opencv-dnn\/#primaryimage"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/real-time-object-detection-on-raspberry-pi-using-opencv-dnn\/#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:5616\/1*LyjNlbtMaXBWGEFzh2wArQ.jpeg","datePublished":"2023-06-29T19:23:23+00:00","dateModified":"2025-04-24T17:15:17+00:00","breadcrumb":{"@id":"https:\/\/www.comet.com\/site\/blog\/real-time-object-detection-on-raspberry-pi-using-opencv-dnn\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.comet.com\/site\/blog\/real-time-object-detection-on-raspberry-pi-using-opencv-dnn\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/blog\/real-time-object-detection-on-raspberry-pi-using-opencv-dnn\/#primaryimage","url":"https:\/\/miro.medium.com\/v2\/resize:fit:5616\/1*LyjNlbtMaXBWGEFzh2wArQ.jpeg","contentUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:5616\/1*LyjNlbtMaXBWGEFzh2wArQ.jpeg"},{"@type":"BreadcrumbList","@id":"https:\/\/www.comet.com\/site\/blog\/real-time-object-detection-on-raspberry-pi-using-opencv-dnn\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.comet.com\/site\/"},{"@type":"ListItem","position":2,"name":"Real-Time Object Detection on Raspberry Pi Using OpenCV DNN"}]},{"@type":"WebSite","@id":"https:\/\/www.comet.com\/site\/#website","url":"https:\/\/www.comet.com\/site\/","name":"Comet","description":"Build Better Models Faster","publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.comet.com\/site\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.comet.com\/site\/#organization","name":"Comet ML, Inc.","alternateName":"Comet","url":"https:\/\/www.comet.com\/site\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","width":310,"height":310,"caption":"Comet ML, Inc."},"image":{"@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/cometdotml","https:\/\/x.com\/Cometml","https:\/\/www.youtube.com\/channel\/UCmN63HKvfXSCS-UwVwmK8Hw"]},{"@type":"Person","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/d3d1005b4f1f4bceea8f5e63f2ce8933","name":"Saumya Shovan Roy","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/image\/c447c437b478ee2126ae1e85cb61360b","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/08\/1539064602993-96x96.jpg","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/08\/1539064602993-96x96.jpg","caption":"Saumya Shovan Roy"},"url":"https:\/\/www.comet.com\/site\/blog\/author\/saumyashovanroy\/"}]}},"_links":{"self":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/6559","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/users\/44"}],"replies":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/comments?post=6559"}],"version-history":[{"count":1,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/6559\/revisions"}],"predecessor-version":[{"id":15608,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/6559\/revisions\/15608"}],"wp:attachment":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media?parent=6559"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/categories?post=6559"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/tags?post=6559"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/coauthors?post=6559"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}