![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*HQ3KhhLHXwxEpqIsa8d4VQ.jpeg\")
<\/figure>What is ONNX?<\/h2>\n
Open Neural Network Exchange, or ONNX, is a free and open-source ecosystem for deep learning model representation. Facebook and Microsoft created this tool in 2017 to make it simpler for academics and engineers to migrate models between various deep-learning frameworks and hardware platforms.<\/p>\n
One of ONNX\u2019s key benefits is that it makes it simple to export models from one framework, like PyTorch, and import them into another framework, like TensorFlow. Engineers who need to deploy models on several hardware platforms or academics who wish to test out various frameworks for training and deploying their models may find this extremely helpful.<\/p>\n
Benefits of Using ONNX<\/h2>\n\n- Interoperability: ONNX allows developers to switch between frameworks like PyTorch, TensorFlow, and Caffe2 without trouble. This compatibility makes it easier for businesses to integrate AI models developed with different tools into their platforms.<\/li>\n
- Portability or Platform Independence: ONNX supports a wide range of hardware, making it easier for developers to deploy their models on various devices without worrying about hardware optimizations. This allows for faster development and deployment.<\/li>\n
- Performance: ONNX is optimized for GPU and CPU, ensuring improved performance and training speed. Moreover, the ONNX runtime provides highly efficient performance across multiple platforms.<\/li>\n
- Flexibility: ONNX standardizes deep learning operations, enabling extensive customization for specific use cases.<\/li>\n
- Community Support: As an open-source project, ONNX has a vibrant community of researchers and developers dedicated to improving and supporting the ecosystem.<\/li>\n<\/ul>\n
Now that we have briefly introduced ONNX let\u2019s look at how it works and how the above benefits would apply through an example code.<\/p>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n
\n\n\nIn the example below, I will demonstrate how to create a simple neural network using PyTorch, convert it to ONNX format, and use ONNX Runtime for evaluation.<\/p>\n
Step 1:<\/h2>\npip install torch torchvision onnx<\/span><\/pre>\nThe above code snippet will install the PyTorch framework, TorchVision (a library that provides datasets and models), and ONNX library using the Python pip package manager.<\/p>\n
Step 2:<\/h2>\nimport<\/span> torch\nimport<\/span> torch.nn as<\/span> nn\nimport<\/span> torch.nn.functional as<\/span> F\n\nclass<\/span> SimpleModel<\/span>(nn.Module):\n def<\/span> __init__<\/span>(self<\/span>):\n super<\/span>(SimpleModel, self).__init__()\n self.fc1 = nn.Linear(28<\/span> * 28<\/span>, 100<\/span>)\n self.fc2 = nn.Linear(100<\/span>, 10<\/span>)\n\n def<\/span> forward<\/span>(self, x<\/span>):\n x = F.relu(self.fc1(x))\n x = F.softmax(self.fc2(x), dim=1<\/span>)\n return<\/span> x\n\nmodel = SimpleModel()<\/span><\/pre>\nHere, we create a simple, fully connected feed-forward neural network with a single hidden layer. In the __init__()<\/code> method, I have initialized two fully connected layers, where the first layer has an input size of 28*28 and 100 output units, and the second layer has 100 input units and 10 output units.<\/p>\nIn the forward()<\/code> method contains the forward pass of the neural network. It accepts an input tensor x<\/code> and applies the ReLU <\/strong>activation function after passing through the first layer. Then, it uses the Softmax <\/strong>activation function after passing through the second layer and returns the x<\/code> tensor as the output.<\/p>\nStep 3:<\/h2>\nimport<\/span> torch.onnx\n\ndummy_input = torch.randn(1<\/span>, 28<\/span>*28<\/span>)\nonnx_filename = \"simple_test_model.onnx\"<\/span>\n\ntorch.onnx.export(model, dummy_input, onnx_filename, verbose=True<\/span>, input_names=['input'<\/span>], output_names=['output'<\/span>])<\/span><\/pre>\nIn this step, we import the torch.onnx<\/code>package for ONNX conversion. We create a dummy_input<\/code>, which is a random tensor. The input tensor is expected to be of size (1, 28*28), where 1 represents the batch size and 28*28 is the input dimension.<\/p>\nWe then define the name of the ONNX model file as simple_test_model.onnx<\/code>. The torch.onnx.export()<\/code> function is used to convert the PyTorch model to ONNX format and save it in the file.<\/p>\nThe input_names<\/code> and output_names<\/code> arguments are optional but help identify the input and output tensors when using the ONNX model later.<\/p><\/blockquote>\nStep 4:<\/h2>\npip install onnxruntime<\/span><\/pre>\nThis above command will install the ONNX runtime, ONNX Runtime is a high-performance, cross-platform library for running ONNX standard models on various devices and platforms or languages.<\/p>\n
import<\/span> onnxruntime as<\/span> ort\nimport<\/span> numpy as<\/span> np\n\nORT_session = ort.InferenceSession(onnx_filename)\n\ndef<\/span> run_model<\/span>(input_data<\/span>):\n input_data = input_data.astype(np.float32)\n input_name = ORT_session.get_inputs()[0<\/span>].name\n output_name = ORT_session.get_outputs()[0<\/span>].name\n result = ORT_session.run([output_name], {input_name: input_data})\n return<\/span> result[0<\/span>]\n\n# Define your input data, for example, a random tensor<\/span>\ninput_data = np.random.randn(1<\/span>, 28<\/span>*28<\/span>)\n\n# Get output<\/span>\noutput = run_model(input_data)\nprint<\/span>(output)<\/span><\/pre>\nI have created an inference session using the saved ONNX model using the file name. Then, I created a function that first casts the input data type to float32<\/code> because it is necessary for ONNX Runtime. Finally, we can run the model using the ORT_session.run()<\/code> and get the result.<\/p>\nComparative Analysis<\/h2>\n
By examining the above example, we can better understand some of the benefits ONNX provides.<\/p>\n
It provides interoperability by supporting various deep-learning frameworks, allowing models to be converted from frameworks such as PyTorch and TensorFlow. In the above code, we converted the PyTorch model to ONNX format.<\/p>\n
ONNX enables easy deployment for models across various platforms and programming languages. In the above code, we ran the ONNX model using the ONNX Runtime library, which is available for different platforms. In my personal experience, I have converted a TensorFlow model to an ONNX model and deployed it on a NodeJS server using the ONNX Runtime library.<\/p>\n
ONNX Runtime is designed to provide optimized execution of models, both static and dynamic optimization. As shown in the code above, running inference with ONNX Runtime capitalizes on these optimizations to offer faster, low-latency inference than running the model directly in the training framework. This results in better resource utilization and efficient model deployments.<\/p>\n
Conclusion<\/h2>\n
ONNX: An indispensable asset for AI developers that provides unmatched flexibility in selecting tools based on individual requirements while ensuring utmost compatibility, portability, and performance. Our article offers detailed instructions on developing a straightforward neural network utilizing PyTorch before assigning it an ONNX format that allows for inference using the ONNX runtime.<\/p>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"
Photo by drmakete lab on Unsplash What is ONNX? Open Neural Network Exchange, or ONNX, is a free and open-source ecosystem for deep learning model representation. Facebook and Microsoft created this tool in 2017 to make it simpler for academics and engineers to migrate models between various deep-learning frameworks and hardware platforms. One of ONNX\u2019s […]<\/p>\n","protected":false},"author":29,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"customer_name":"","customer_description":"","customer_industry":"","customer_technologies":"","customer_logo":"","footnotes":""},"categories":[7],"tags":[],"coauthors":[110],"class_list":["post-8008","post","type-post","status-publish","format-standard","hentry","category-tutorials"],"yoast_head":"\n
Unlocking the Power of ONNX: Model Interoperability and Boosting Performance - Comet<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Unlocking the Power of ONNX: Model Interoperability and Boosting Performance\" \/>\n<meta property=\"og:description\" content=\"Photo by drmakete lab on Unsplash What is ONNX? Open Neural Network Exchange, or ONNX, is a free and open-source ecosystem for deep learning model representation. Facebook and Microsoft created this tool in 2017 to make it simpler for academics and engineers to migrate models between various deep-learning frameworks and hardware platforms. One of ONNX\u2019s […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance\" \/>\n<meta property=\"og:site_name\" content=\"Comet\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cometdotml\" \/>\n<meta property=\"article:published_time\" content=\"2023-10-24T16:03:32+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-04-24T17:05:18+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*HQ3KhhLHXwxEpqIsa8d4VQ.jpeg\" \/>\n<meta name=\"author\" content=\"Ravindu Senaratne\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@Cometml\" \/>\n<meta name=\"twitter:site\" content=\"@Cometml\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Ravindu Senaratne\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Unlocking the Power of ONNX: Model Interoperability and Boosting Performance - Comet","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance","og_locale":"en_US","og_type":"article","og_title":"Unlocking the Power of ONNX: Model Interoperability and Boosting Performance","og_description":"Photo by drmakete lab on Unsplash What is ONNX? Open Neural Network Exchange, or ONNX, is a free and open-source ecosystem for deep learning model representation. Facebook and Microsoft created this tool in 2017 to make it simpler for academics and engineers to migrate models between various deep-learning frameworks and hardware platforms. One of ONNX\u2019s […]","og_url":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance","og_site_name":"Comet","article_publisher":"https:\/\/www.facebook.com\/cometdotml","article_published_time":"2023-10-24T16:03:32+00:00","article_modified_time":"2025-04-24T17:05:18+00:00","og_image":[{"url":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*HQ3KhhLHXwxEpqIsa8d4VQ.jpeg","type":"","width":"","height":""}],"author":"Ravindu Senaratne","twitter_card":"summary_large_image","twitter_creator":"@Cometml","twitter_site":"@Cometml","twitter_misc":{"Written by":"Ravindu Senaratne","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance#article","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance\/"},"author":{"name":"Ravindu Senaratne","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/c201ed1fa6abe33140a67fd6a5d70691"},"headline":"Unlocking the Power of ONNX: Model Interoperability and Boosting Performance","datePublished":"2023-10-24T16:03:32+00:00","dateModified":"2025-04-24T17:05:18+00:00","mainEntityOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance\/"},"wordCount":801,"publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*HQ3KhhLHXwxEpqIsa8d4VQ.jpeg","articleSection":["Tutorials"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance\/","url":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance","name":"Unlocking the Power of ONNX: Model Interoperability and Boosting Performance - Comet","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance#primaryimage"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*HQ3KhhLHXwxEpqIsa8d4VQ.jpeg","datePublished":"2023-10-24T16:03:32+00:00","dateModified":"2025-04-24T17:05:18+00:00","breadcrumb":{"@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance#primaryimage","url":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*HQ3KhhLHXwxEpqIsa8d4VQ.jpeg","contentUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*HQ3KhhLHXwxEpqIsa8d4VQ.jpeg"},{"@type":"BreadcrumbList","@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.comet.com\/site\/"},{"@type":"ListItem","position":2,"name":"Unlocking the Power of ONNX: Model Interoperability and Boosting Performance"}]},{"@type":"WebSite","@id":"https:\/\/www.comet.com\/site\/#website","url":"https:\/\/www.comet.com\/site\/","name":"Comet","description":"Build Better Models Faster","publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.comet.com\/site\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.comet.com\/site\/#organization","name":"Comet ML, Inc.","alternateName":"Comet","url":"https:\/\/www.comet.com\/site\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","width":310,"height":310,"caption":"Comet ML, Inc."},"image":{"@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/cometdotml","https:\/\/x.com\/Cometml","https:\/\/www.youtube.com\/channel\/UCmN63HKvfXSCS-UwVwmK8Hw"]},{"@type":"Person","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/c201ed1fa6abe33140a67fd6a5d70691","name":"Ravindu Senaratne","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/image\/ae4b12e558e2fa062d8be814e51d034a","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/10\/1687099615398-96x96.jpg","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/10\/1687099615398-96x96.jpg","caption":"Ravindu Senaratne"},"url":"https:\/\/www.comet.com\/site\/blog\/author\/ravindusenaratne\/"}]}},"_links":{"self":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/8008","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/users\/29"}],"replies":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/comments?post=8008"}],"version-history":[{"count":1,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/8008\/revisions"}],"predecessor-version":[{"id":15489,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/8008\/revisions\/15489"}],"wp:attachment":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media?parent=8008"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/categories?post=8008"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/tags?post=8008"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/coauthors?post=8008"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
Now that we have briefly introduced ONNX let\u2019s look at how it works and how the above benefits would apply through an example code.<\/p>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n
In the example below, I will demonstrate how to create a simple neural network using PyTorch, convert it to ONNX format, and use ONNX Runtime for evaluation.<\/p>\n
Step 1:<\/h2>\npip install torch torchvision onnx<\/span><\/pre>\nThe above code snippet will install the PyTorch framework, TorchVision (a library that provides datasets and models), and ONNX library using the Python pip package manager.<\/p>\n
Step 2:<\/h2>\nimport<\/span> torch\nimport<\/span> torch.nn as<\/span> nn\nimport<\/span> torch.nn.functional as<\/span> F\n\nclass<\/span> SimpleModel<\/span>(nn.Module):\n def<\/span> __init__<\/span>(self<\/span>):\n super<\/span>(SimpleModel, self).__init__()\n self.fc1 = nn.Linear(28<\/span> * 28<\/span>, 100<\/span>)\n self.fc2 = nn.Linear(100<\/span>, 10<\/span>)\n\n def<\/span> forward<\/span>(self, x<\/span>):\n x = F.relu(self.fc1(x))\n x = F.softmax(self.fc2(x), dim=1<\/span>)\n return<\/span> x\n\nmodel = SimpleModel()<\/span><\/pre>\nHere, we create a simple, fully connected feed-forward neural network with a single hidden layer. In the __init__()<\/code> method, I have initialized two fully connected layers, where the first layer has an input size of 28*28 and 100 output units, and the second layer has 100 input units and 10 output units.<\/p>\nIn the forward()<\/code> method contains the forward pass of the neural network. It accepts an input tensor x<\/code> and applies the ReLU <\/strong>activation function after passing through the first layer. Then, it uses the Softmax <\/strong>activation function after passing through the second layer and returns the x<\/code> tensor as the output.<\/p>\nStep 3:<\/h2>\nimport<\/span> torch.onnx\n\ndummy_input = torch.randn(1<\/span>, 28<\/span>*28<\/span>)\nonnx_filename = \"simple_test_model.onnx\"<\/span>\n\ntorch.onnx.export(model, dummy_input, onnx_filename, verbose=True<\/span>, input_names=['input'<\/span>], output_names=['output'<\/span>])<\/span><\/pre>\nIn this step, we import the torch.onnx<\/code>package for ONNX conversion. We create a dummy_input<\/code>, which is a random tensor. The input tensor is expected to be of size (1, 28*28), where 1 represents the batch size and 28*28 is the input dimension.<\/p>\nWe then define the name of the ONNX model file as simple_test_model.onnx<\/code>. The torch.onnx.export()<\/code> function is used to convert the PyTorch model to ONNX format and save it in the file.<\/p>\nThe input_names<\/code> and output_names<\/code> arguments are optional but help identify the input and output tensors when using the ONNX model later.<\/p><\/blockquote>\nStep 4:<\/h2>\npip install onnxruntime<\/span><\/pre>\nThis above command will install the ONNX runtime, ONNX Runtime is a high-performance, cross-platform library for running ONNX standard models on various devices and platforms or languages.<\/p>\n
import<\/span> onnxruntime as<\/span> ort\nimport<\/span> numpy as<\/span> np\n\nORT_session = ort.InferenceSession(onnx_filename)\n\ndef<\/span> run_model<\/span>(input_data<\/span>):\n input_data = input_data.astype(np.float32)\n input_name = ORT_session.get_inputs()[0<\/span>].name\n output_name = ORT_session.get_outputs()[0<\/span>].name\n result = ORT_session.run([output_name], {input_name: input_data})\n return<\/span> result[0<\/span>]\n\n# Define your input data, for example, a random tensor<\/span>\ninput_data = np.random.randn(1<\/span>, 28<\/span>*28<\/span>)\n\n# Get output<\/span>\noutput = run_model(input_data)\nprint<\/span>(output)<\/span><\/pre>\nI have created an inference session using the saved ONNX model using the file name. Then, I created a function that first casts the input data type to float32<\/code> because it is necessary for ONNX Runtime. Finally, we can run the model using the ORT_session.run()<\/code> and get the result.<\/p>\nComparative Analysis<\/h2>\n
By examining the above example, we can better understand some of the benefits ONNX provides.<\/p>\n
It provides interoperability by supporting various deep-learning frameworks, allowing models to be converted from frameworks such as PyTorch and TensorFlow. In the above code, we converted the PyTorch model to ONNX format.<\/p>\n
ONNX enables easy deployment for models across various platforms and programming languages. In the above code, we ran the ONNX model using the ONNX Runtime library, which is available for different platforms. In my personal experience, I have converted a TensorFlow model to an ONNX model and deployed it on a NodeJS server using the ONNX Runtime library.<\/p>\n
ONNX Runtime is designed to provide optimized execution of models, both static and dynamic optimization. As shown in the code above, running inference with ONNX Runtime capitalizes on these optimizations to offer faster, low-latency inference than running the model directly in the training framework. This results in better resource utilization and efficient model deployments.<\/p>\n
Conclusion<\/h2>\n
ONNX: An indispensable asset for AI developers that provides unmatched flexibility in selecting tools based on individual requirements while ensuring utmost compatibility, portability, and performance. Our article offers detailed instructions on developing a straightforward neural network utilizing PyTorch before assigning it an ONNX format that allows for inference using the ONNX runtime.<\/p>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"
Photo by drmakete lab on Unsplash What is ONNX? Open Neural Network Exchange, or ONNX, is a free and open-source ecosystem for deep learning model representation. Facebook and Microsoft created this tool in 2017 to make it simpler for academics and engineers to migrate models between various deep-learning frameworks and hardware platforms. One of ONNX\u2019s […]<\/p>\n","protected":false},"author":29,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"customer_name":"","customer_description":"","customer_industry":"","customer_technologies":"","customer_logo":"","footnotes":""},"categories":[7],"tags":[],"coauthors":[110],"class_list":["post-8008","post","type-post","status-publish","format-standard","hentry","category-tutorials"],"yoast_head":"\n
Unlocking the Power of ONNX: Model Interoperability and Boosting Performance - Comet<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Unlocking the Power of ONNX: Model Interoperability and Boosting Performance\" \/>\n<meta property=\"og:description\" content=\"Photo by drmakete lab on Unsplash What is ONNX? Open Neural Network Exchange, or ONNX, is a free and open-source ecosystem for deep learning model representation. Facebook and Microsoft created this tool in 2017 to make it simpler for academics and engineers to migrate models between various deep-learning frameworks and hardware platforms. One of ONNX\u2019s […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance\" \/>\n<meta property=\"og:site_name\" content=\"Comet\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cometdotml\" \/>\n<meta property=\"article:published_time\" content=\"2023-10-24T16:03:32+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-04-24T17:05:18+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*HQ3KhhLHXwxEpqIsa8d4VQ.jpeg\" \/>\n<meta name=\"author\" content=\"Ravindu Senaratne\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@Cometml\" \/>\n<meta name=\"twitter:site\" content=\"@Cometml\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Ravindu Senaratne\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Unlocking the Power of ONNX: Model Interoperability and Boosting Performance - Comet","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance","og_locale":"en_US","og_type":"article","og_title":"Unlocking the Power of ONNX: Model Interoperability and Boosting Performance","og_description":"Photo by drmakete lab on Unsplash What is ONNX? Open Neural Network Exchange, or ONNX, is a free and open-source ecosystem for deep learning model representation. Facebook and Microsoft created this tool in 2017 to make it simpler for academics and engineers to migrate models between various deep-learning frameworks and hardware platforms. One of ONNX\u2019s […]","og_url":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance","og_site_name":"Comet","article_publisher":"https:\/\/www.facebook.com\/cometdotml","article_published_time":"2023-10-24T16:03:32+00:00","article_modified_time":"2025-04-24T17:05:18+00:00","og_image":[{"url":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*HQ3KhhLHXwxEpqIsa8d4VQ.jpeg","type":"","width":"","height":""}],"author":"Ravindu Senaratne","twitter_card":"summary_large_image","twitter_creator":"@Cometml","twitter_site":"@Cometml","twitter_misc":{"Written by":"Ravindu Senaratne","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance#article","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance\/"},"author":{"name":"Ravindu Senaratne","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/c201ed1fa6abe33140a67fd6a5d70691"},"headline":"Unlocking the Power of ONNX: Model Interoperability and Boosting Performance","datePublished":"2023-10-24T16:03:32+00:00","dateModified":"2025-04-24T17:05:18+00:00","mainEntityOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance\/"},"wordCount":801,"publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*HQ3KhhLHXwxEpqIsa8d4VQ.jpeg","articleSection":["Tutorials"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance\/","url":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance","name":"Unlocking the Power of ONNX: Model Interoperability and Boosting Performance - Comet","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance#primaryimage"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*HQ3KhhLHXwxEpqIsa8d4VQ.jpeg","datePublished":"2023-10-24T16:03:32+00:00","dateModified":"2025-04-24T17:05:18+00:00","breadcrumb":{"@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance#primaryimage","url":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*HQ3KhhLHXwxEpqIsa8d4VQ.jpeg","contentUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*HQ3KhhLHXwxEpqIsa8d4VQ.jpeg"},{"@type":"BreadcrumbList","@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.comet.com\/site\/"},{"@type":"ListItem","position":2,"name":"Unlocking the Power of ONNX: Model Interoperability and Boosting Performance"}]},{"@type":"WebSite","@id":"https:\/\/www.comet.com\/site\/#website","url":"https:\/\/www.comet.com\/site\/","name":"Comet","description":"Build Better Models Faster","publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.comet.com\/site\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.comet.com\/site\/#organization","name":"Comet ML, Inc.","alternateName":"Comet","url":"https:\/\/www.comet.com\/site\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","width":310,"height":310,"caption":"Comet ML, Inc."},"image":{"@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/cometdotml","https:\/\/x.com\/Cometml","https:\/\/www.youtube.com\/channel\/UCmN63HKvfXSCS-UwVwmK8Hw"]},{"@type":"Person","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/c201ed1fa6abe33140a67fd6a5d70691","name":"Ravindu Senaratne","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/image\/ae4b12e558e2fa062d8be814e51d034a","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/10\/1687099615398-96x96.jpg","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/10\/1687099615398-96x96.jpg","caption":"Ravindu Senaratne"},"url":"https:\/\/www.comet.com\/site\/blog\/author\/ravindusenaratne\/"}]}},"_links":{"self":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/8008","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/users\/29"}],"replies":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/comments?post=8008"}],"version-history":[{"count":1,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/8008\/revisions"}],"predecessor-version":[{"id":15489,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/8008\/revisions\/15489"}],"wp:attachment":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media?parent=8008"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/categories?post=8008"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/tags?post=8008"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/coauthors?post=8008"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
The above code snippet will install the PyTorch framework, TorchVision (a library that provides datasets and models), and ONNX library using the Python pip package manager.<\/p>\n
Step 2:<\/h2>\nimport<\/span> torch\nimport<\/span> torch.nn as<\/span> nn\nimport<\/span> torch.nn.functional as<\/span> F\n\nclass<\/span> SimpleModel<\/span>(nn.Module):\n def<\/span> __init__<\/span>(self<\/span>):\n super<\/span>(SimpleModel, self).__init__()\n self.fc1 = nn.Linear(28<\/span> * 28<\/span>, 100<\/span>)\n self.fc2 = nn.Linear(100<\/span>, 10<\/span>)\n\n def<\/span> forward<\/span>(self, x<\/span>):\n x = F.relu(self.fc1(x))\n x = F.softmax(self.fc2(x), dim=1<\/span>)\n return<\/span> x\n\nmodel = SimpleModel()<\/span><\/pre>\nHere, we create a simple, fully connected feed-forward neural network with a single hidden layer. In the __init__()<\/code> method, I have initialized two fully connected layers, where the first layer has an input size of 28*28 and 100 output units, and the second layer has 100 input units and 10 output units.<\/p>\nIn the forward()<\/code> method contains the forward pass of the neural network. It accepts an input tensor x<\/code> and applies the ReLU <\/strong>activation function after passing through the first layer. Then, it uses the Softmax <\/strong>activation function after passing through the second layer and returns the x<\/code> tensor as the output.<\/p>\nStep 3:<\/h2>\nimport<\/span> torch.onnx\n\ndummy_input = torch.randn(1<\/span>, 28<\/span>*28<\/span>)\nonnx_filename = \"simple_test_model.onnx\"<\/span>\n\ntorch.onnx.export(model, dummy_input, onnx_filename, verbose=True<\/span>, input_names=['input'<\/span>], output_names=['output'<\/span>])<\/span><\/pre>\nIn this step, we import the torch.onnx<\/code>package for ONNX conversion. We create a dummy_input<\/code>, which is a random tensor. The input tensor is expected to be of size (1, 28*28), where 1 represents the batch size and 28*28 is the input dimension.<\/p>\nWe then define the name of the ONNX model file as simple_test_model.onnx<\/code>. The torch.onnx.export()<\/code> function is used to convert the PyTorch model to ONNX format and save it in the file.<\/p>\nThe input_names<\/code> and output_names<\/code> arguments are optional but help identify the input and output tensors when using the ONNX model later.<\/p><\/blockquote>\nStep 4:<\/h2>\npip install onnxruntime<\/span><\/pre>\nThis above command will install the ONNX runtime, ONNX Runtime is a high-performance, cross-platform library for running ONNX standard models on various devices and platforms or languages.<\/p>\n
import<\/span> onnxruntime as<\/span> ort\nimport<\/span> numpy as<\/span> np\n\nORT_session = ort.InferenceSession(onnx_filename)\n\ndef<\/span> run_model<\/span>(input_data<\/span>):\n input_data = input_data.astype(np.float32)\n input_name = ORT_session.get_inputs()[0<\/span>].name\n output_name = ORT_session.get_outputs()[0<\/span>].name\n result = ORT_session.run([output_name], {input_name: input_data})\n return<\/span> result[0<\/span>]\n\n# Define your input data, for example, a random tensor<\/span>\ninput_data = np.random.randn(1<\/span>, 28<\/span>*28<\/span>)\n\n# Get output<\/span>\noutput = run_model(input_data)\nprint<\/span>(output)<\/span><\/pre>\nI have created an inference session using the saved ONNX model using the file name. Then, I created a function that first casts the input data type to float32<\/code> because it is necessary for ONNX Runtime. Finally, we can run the model using the ORT_session.run()<\/code> and get the result.<\/p>\nComparative Analysis<\/h2>\n
By examining the above example, we can better understand some of the benefits ONNX provides.<\/p>\n
It provides interoperability by supporting various deep-learning frameworks, allowing models to be converted from frameworks such as PyTorch and TensorFlow. In the above code, we converted the PyTorch model to ONNX format.<\/p>\n
ONNX enables easy deployment for models across various platforms and programming languages. In the above code, we ran the ONNX model using the ONNX Runtime library, which is available for different platforms. In my personal experience, I have converted a TensorFlow model to an ONNX model and deployed it on a NodeJS server using the ONNX Runtime library.<\/p>\n
ONNX Runtime is designed to provide optimized execution of models, both static and dynamic optimization. As shown in the code above, running inference with ONNX Runtime capitalizes on these optimizations to offer faster, low-latency inference than running the model directly in the training framework. This results in better resource utilization and efficient model deployments.<\/p>\n
Conclusion<\/h2>\n
ONNX: An indispensable asset for AI developers that provides unmatched flexibility in selecting tools based on individual requirements while ensuring utmost compatibility, portability, and performance. Our article offers detailed instructions on developing a straightforward neural network utilizing PyTorch before assigning it an ONNX format that allows for inference using the ONNX runtime.<\/p>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"
Photo by drmakete lab on Unsplash What is ONNX? Open Neural Network Exchange, or ONNX, is a free and open-source ecosystem for deep learning model representation. Facebook and Microsoft created this tool in 2017 to make it simpler for academics and engineers to migrate models between various deep-learning frameworks and hardware platforms. One of ONNX\u2019s […]<\/p>\n","protected":false},"author":29,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"customer_name":"","customer_description":"","customer_industry":"","customer_technologies":"","customer_logo":"","footnotes":""},"categories":[7],"tags":[],"coauthors":[110],"class_list":["post-8008","post","type-post","status-publish","format-standard","hentry","category-tutorials"],"yoast_head":"\n
Unlocking the Power of ONNX: Model Interoperability and Boosting Performance - Comet<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Unlocking the Power of ONNX: Model Interoperability and Boosting Performance\" \/>\n<meta property=\"og:description\" content=\"Photo by drmakete lab on Unsplash What is ONNX? Open Neural Network Exchange, or ONNX, is a free and open-source ecosystem for deep learning model representation. Facebook and Microsoft created this tool in 2017 to make it simpler for academics and engineers to migrate models between various deep-learning frameworks and hardware platforms. One of ONNX\u2019s […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance\" \/>\n<meta property=\"og:site_name\" content=\"Comet\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cometdotml\" \/>\n<meta property=\"article:published_time\" content=\"2023-10-24T16:03:32+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-04-24T17:05:18+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*HQ3KhhLHXwxEpqIsa8d4VQ.jpeg\" \/>\n<meta name=\"author\" content=\"Ravindu Senaratne\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@Cometml\" \/>\n<meta name=\"twitter:site\" content=\"@Cometml\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Ravindu Senaratne\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Unlocking the Power of ONNX: Model Interoperability and Boosting Performance - Comet","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance","og_locale":"en_US","og_type":"article","og_title":"Unlocking the Power of ONNX: Model Interoperability and Boosting Performance","og_description":"Photo by drmakete lab on Unsplash What is ONNX? Open Neural Network Exchange, or ONNX, is a free and open-source ecosystem for deep learning model representation. Facebook and Microsoft created this tool in 2017 to make it simpler for academics and engineers to migrate models between various deep-learning frameworks and hardware platforms. One of ONNX\u2019s […]","og_url":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance","og_site_name":"Comet","article_publisher":"https:\/\/www.facebook.com\/cometdotml","article_published_time":"2023-10-24T16:03:32+00:00","article_modified_time":"2025-04-24T17:05:18+00:00","og_image":[{"url":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*HQ3KhhLHXwxEpqIsa8d4VQ.jpeg","type":"","width":"","height":""}],"author":"Ravindu Senaratne","twitter_card":"summary_large_image","twitter_creator":"@Cometml","twitter_site":"@Cometml","twitter_misc":{"Written by":"Ravindu Senaratne","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance#article","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance\/"},"author":{"name":"Ravindu Senaratne","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/c201ed1fa6abe33140a67fd6a5d70691"},"headline":"Unlocking the Power of ONNX: Model Interoperability and Boosting Performance","datePublished":"2023-10-24T16:03:32+00:00","dateModified":"2025-04-24T17:05:18+00:00","mainEntityOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance\/"},"wordCount":801,"publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*HQ3KhhLHXwxEpqIsa8d4VQ.jpeg","articleSection":["Tutorials"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance\/","url":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance","name":"Unlocking the Power of ONNX: Model Interoperability and Boosting Performance - Comet","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance#primaryimage"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*HQ3KhhLHXwxEpqIsa8d4VQ.jpeg","datePublished":"2023-10-24T16:03:32+00:00","dateModified":"2025-04-24T17:05:18+00:00","breadcrumb":{"@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance#primaryimage","url":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*HQ3KhhLHXwxEpqIsa8d4VQ.jpeg","contentUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*HQ3KhhLHXwxEpqIsa8d4VQ.jpeg"},{"@type":"BreadcrumbList","@id":"https:\/\/www.comet.com\/site\/blog\/unlocking-the-power-of-onnx-model-interoperability-and-boosting-performance#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.comet.com\/site\/"},{"@type":"ListItem","position":2,"name":"Unlocking the Power of ONNX: Model Interoperability and Boosting Performance"}]},{"@type":"WebSite","@id":"https:\/\/www.comet.com\/site\/#website","url":"https:\/\/www.comet.com\/site\/","name":"Comet","description":"Build Better Models Faster","publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.comet.com\/site\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.comet.com\/site\/#organization","name":"Comet ML, Inc.","alternateName":"Comet","url":"https:\/\/www.comet.com\/site\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","width":310,"height":310,"caption":"Comet ML, Inc."},"image":{"@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/cometdotml","https:\/\/x.com\/Cometml","https:\/\/www.youtube.com\/channel\/UCmN63HKvfXSCS-UwVwmK8Hw"]},{"@type":"Person","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/c201ed1fa6abe33140a67fd6a5d70691","name":"Ravindu Senaratne","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/image\/ae4b12e558e2fa062d8be814e51d034a","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/10\/1687099615398-96x96.jpg","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/10\/1687099615398-96x96.jpg","caption":"Ravindu Senaratne"},"url":"https:\/\/www.comet.com\/site\/blog\/author\/ravindusenaratne\/"}]}},"_links":{"self":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/8008","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/users\/29"}],"replies":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/comments?post=8008"}],"version-history":[{"count":1,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/8008\/revisions"}],"predecessor-version":[{"id":15489,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/8008\/revisions\/15489"}],"wp:attachment":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media?parent=8008"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/categories?post=8008"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/tags?post=8008"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/coauthors?post=8008"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
Here, we create a simple, fully connected feed-forward neural network with a single hidden layer. In the In the In this step, we import the We then define the name of the ONNX model file as The This above command will install the ONNX runtime, ONNX Runtime is a high-performance, cross-platform library for running ONNX standard models on various devices and platforms or languages.<\/p>\n I have created an inference session using the saved ONNX model using the file name. Then, I created a function that first casts the input data type to By examining the above example, we can better understand some of the benefits ONNX provides.<\/p>\n It provides interoperability by supporting various deep-learning frameworks, allowing models to be converted from frameworks such as PyTorch and TensorFlow. In the above code, we converted the PyTorch model to ONNX format.<\/p>\n ONNX enables easy deployment for models across various platforms and programming languages. In the above code, we ran the ONNX model using the ONNX Runtime library, which is available for different platforms. In my personal experience, I have converted a TensorFlow model to an ONNX model and deployed it on a NodeJS server using the ONNX Runtime library.<\/p>\n ONNX Runtime is designed to provide optimized execution of models, both static and dynamic optimization. As shown in the code above, running inference with ONNX Runtime capitalizes on these optimizations to offer faster, low-latency inference than running the model directly in the training framework. This results in better resource utilization and efficient model deployments.<\/p>\n ONNX: An indispensable asset for AI developers that provides unmatched flexibility in selecting tools based on individual requirements while ensuring utmost compatibility, portability, and performance. Our article offers detailed instructions on developing a straightforward neural network utilizing PyTorch before assigning it an ONNX format that allows for inference using the ONNX runtime.<\/p>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":" Photo by drmakete lab on Unsplash What is ONNX? Open Neural Network Exchange, or ONNX, is a free and open-source ecosystem for deep learning model representation. Facebook and Microsoft created this tool in 2017 to make it simpler for academics and engineers to migrate models between various deep-learning frameworks and hardware platforms. One of ONNX\u2019s […]<\/p>\n","protected":false},"author":29,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"customer_name":"","customer_description":"","customer_industry":"","customer_technologies":"","customer_logo":"","footnotes":""},"categories":[7],"tags":[],"coauthors":[110],"class_list":["post-8008","post","type-post","status-publish","format-standard","hentry","category-tutorials"],"yoast_head":"\n__init__()<\/code> method, I have initialized two fully connected layers, where the first layer has an input size of 28*28 and 100 output units, and the second layer has 100 input units and 10 output units.<\/p>\nforward()<\/code> method contains the forward pass of the neural network. It accepts an input tensor x<\/code> and applies the ReLU <\/strong>activation function after passing through the first layer. Then, it uses the Softmax <\/strong>activation function after passing through the second layer and returns the x<\/code> tensor as the output.<\/p>\nStep 3:<\/h2>\n
import<\/span> torch.onnx\n\ndummy_input = torch.randn(1<\/span>, 28<\/span>*28<\/span>)\nonnx_filename = \"simple_test_model.onnx\"<\/span>\n\ntorch.onnx.export(model, dummy_input, onnx_filename, verbose=True<\/span>, input_names=['input'<\/span>], output_names=['output'<\/span>])<\/span><\/pre>\ntorch.onnx<\/code>package for ONNX conversion. We create a dummy_input<\/code>, which is a random tensor. The input tensor is expected to be of size (1, 28*28), where 1 represents the batch size and 28*28 is the input dimension.<\/p>\nsimple_test_model.onnx<\/code>. The torch.onnx.export()<\/code> function is used to convert the PyTorch model to ONNX format and save it in the file.<\/p>\ninput_names<\/code> and output_names<\/code> arguments are optional but help identify the input and output tensors when using the ONNX model later.<\/p><\/blockquote>\nStep 4:<\/h2>\n
pip install onnxruntime<\/span><\/pre>\nimport<\/span> onnxruntime as<\/span> ort\nimport<\/span> numpy as<\/span> np\n\nORT_session = ort.InferenceSession(onnx_filename)\n\ndef<\/span> run_model<\/span>(input_data<\/span>):\n input_data = input_data.astype(np.float32)\n input_name = ORT_session.get_inputs()[0<\/span>].name\n output_name = ORT_session.get_outputs()[0<\/span>].name\n result = ORT_session.run([output_name], {input_name: input_data})\n return<\/span> result[0<\/span>]\n\n# Define your input data, for example, a random tensor<\/span>\ninput_data = np.random.randn(1<\/span>, 28<\/span>*28<\/span>)\n\n# Get output<\/span>\noutput = run_model(input_data)\nprint<\/span>(output)<\/span><\/pre>\nfloat32<\/code> because it is necessary for ONNX Runtime. Finally, we can run the model using the ORT_session.run()<\/code> and get the result.<\/p>\nComparative Analysis<\/h2>\n
Conclusion<\/h2>\n