{"id":7397,"date":"2023-09-07T10:26:15","date_gmt":"2023-09-07T18:26:15","guid":{"rendered":"https:\/\/live-cometml.pantheonsite.io\/?p=7397"},"modified":"2025-04-24T17:14:20","modified_gmt":"2025-04-24T17:14:20","slug":"guide-to-loss-functions-for-machine-learning-models","status":"publish","type":"post","link":"https:\/\/www.comet.com\/site\/blog\/guide-to-loss-functions-for-machine-learning-models\/","title":{"rendered":"Guide to Loss Functions for Machine Learning Models"},"content":{"rendered":"\n\n\n\n\n
\n
\n
\n
\n
\n
\"\"<\/figure>
<\/picture><\/div>\n<\/div>
Photo by Alexandre Debi\u00e8ve<\/a> on Unsplash<\/a><\/figcaption><\/figure>\n

In machine learning, a loss function is used to measure the loss, or cost, of a specific machine learning model. These loss functions calculate the amount of error in a specific machine learning model using some mathematical formula and measure the performance of that specific model.<\/p>\n

There are various loss functions that are used in machine learning for regression and classification problems. In the process of the machine learning model building, our aim is to minimize the loss\/cost function and therefore increase the accuracy of the model.<\/p>\n

In this article, we will go through various loss functions used in machine learning models for regression and classification problems. Let\u2019s get started!<\/p>\n

Loss functions for Regression<\/h1>\n

Mean Squared Error<\/h2>\n

The mean squared error (MSE) measures the amount of error in machine learning regression models by calculating the average squared distance between the observed and predicted values in our data.<\/p>\n

This loss function penalizes the model for large errors by squaring them and therefore making the model less<\/strong> robust<\/strong> to the outliers. We should not use this loss function when the dataset is prone to too many outliers.<\/p>\n

For a good model\/estimator, the value of the MSE should be closer to zero.<\/p>\n

If we have a data point Yi and the predicted value for this point is \u0176i, then the mean squared error can be calculated as:<\/p>\n

\n
\"\"<\/figure>
<\/picture><\/div>\n<\/figure>\n

Where n is the total number of observations in our dataset.<\/p>\n

Example<\/strong>:<\/p>\n

import<\/strong> numpy as<\/strong> np<\/span>def mean_squared_error<\/strong>(true_val, pred):\n    squared_error = np.square(true_val - pred)\n    sum_squared_error = np.sum(squared_error)\n    mse_loss = sum_squared_error \/ true.size\n\n    return<\/strong> mse_loss<\/span><\/pre>\n
In machine learning, we aim to lower the MSE value to increase the accuracy of the model.<\/p>\n
Mean Absolute Error<\/h2>\n
The mean absolute error (MAE) measures the amount of error in machine learning models by calculating the total absolute difference between the actual value and the predicted value in our data. This error is also known as L1 loss<\/strong>.<\/p>\n
This mean absolute error loss function is more robust<\/strong> to the outliers in comparison with the MSE loss function. Therefore, we can use this loss function when the dataset is prone to too many outliers.<\/p>\n
If we have data point Yi and the corresponding predicted value for this data point is \u0176i, then the mean absolute error can be calculated as:<\/p>\n
\n
\"\"<\/figure>
<\/picture><\/div>\n<\/figure>\n
Where n is the total number of observations in our dataset.<\/p>\n
Example<\/strong>:<\/p>\n
from<\/strong> sklearn.metrics import<\/strong> mean_absolute_error<\/span>#sample data\nactual =<\/strong> [2, 3, 5, 5, 9]\npredicted =<\/strong> [3, 3, 8, 7, 6]<\/span>#calculate MAE<\/span>error =<\/strong> mean_absolute_error<\/strong>(actual, predicted)\nprint(error)<\/span><\/pre>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n
\n
\n
\n
Struggling to track and reproduce complex experiment parameters? Artifacts are just one of the many tools in the Comet toolbox to help ease model management. Read our PetCam scenario to learn more.<\/a><\/p><\/blockquote>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n
\n
\n
\n
Mean Absolute Percentage Error<\/h2>\n
The mean absolute percentage error (MAPE) loss function measures the amount of error by taking the absolute difference between the actual value and the predicted value in our data and then dividing it by the actual value. We apply the absolute percentage to this value and then average it across our dataset.<\/p>\n
This loss function measures the error better than the MSE as it does not penalize large errors. This loss function is commonly used as it normalizes all the errors on a common scale and is easy to interpret.<\/p>\n
If we have data point Yi and the corresponding predicted value for this data point is \u0176i, then the mean absolute percentage error can be calculated as:<\/p>\n
\n
\n
\"\"<\/figure>
<\/picture><\/div>\n<\/div>\n<\/figure>\n
Where n is the total number of observations in our dataset.<\/p>\n
Example:<\/strong><\/p>\n
import<\/strong> numpy as<\/strong> np<\/span>def mean_absolute_percentage_error<\/strong>(actual_value, pred):\n    abs_error = (np.abs(actual_value - pred)) \/ actual_value\n    sum_abs_error = np.sum(abs_error)\n    mape_loss_perc = (sum_abs_error \/ true.size) * 100<\/span>    return<\/strong> mape_loss_perc<\/span><\/pre>\n
The smaller the value of MAPE, the better will be the accuracy of the model.<\/p>\n
Mean Squared Logarithmic Error<\/h2>\n
The mean squared logarithmic error (MSLE) measures the amount of error in the machine learning models by taking the ratio between the actual value and the predicted value in our data.<\/p>\n
As the name suggests, this loss function is a variation of mean squared error (MSE). We use this loss function when we do not want to significantly penalize the model (as mean squared error) for large errors than the smaller errors.<\/p>\n
If we have data point Yi and the corresponding predicted value for this data point is \u0176i, then the mean absolute logarithmic error can be calculated as:<\/p>\n
\n
\n
\"\"<\/figure>
<\/picture><\/div>\n<\/div>\n<\/figure>\n
Where n is the total number of observations in our dataset.<\/p>\n
Example<\/strong>:<\/p>\n
from<\/strong> sklearn.metrics import<\/strong> mean_squared_log_error<\/span>#sample data\nactual = [3, 5, 2.5, 7, 11]\npredicted = [2.5, 5, 4, 8, 10.5]<\/span>#calculate MSLE<\/span>error = mean_squared_log_error<\/strong>(actual, predicted)\nprint(error)<\/span><\/pre>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n
\n
\n
\n
Loss functions for Classification<\/h1>\n
Binary Cross-Entropy Loss<\/h2>\n
This binary cross-entropy loss is a default loss function for binary classifiers in machine learning algorithms. It measures the performance of a classification model by comparing the value of each predicted probability with the actual class output such as 0 <\/strong>or 1<\/strong>.<\/p>\n
If the difference between the predicted probability and the value of actual class output increases, then the cross-entropy loss increases as well.<\/p>\n
Suppose a classifier predicts a probability of 0.09 and the actual observation class is 1, then it would result in a high cross-entropy loss value. The value of this loss function for a model should be closer to 0 ideally, to achieve high accuracy.<\/p>\n
Here\u2019s the formula for the cross-entropy loss function:<\/p>\n
\n
\n
\"\"<\/figure>
<\/picture><\/div>\n<\/div>\n<\/figure>\n
Here, h\u03b8(xi) is the probability of class 1 and (1-h\u03b8(xi)) is the probability of class 0.<\/p>\n
Example<\/strong>:<\/p>\n
def binary_cross_entropy<\/strong>(actual, predicted):\n\tsum_score = 0.0\n\tfor i in range<\/strong>(len(actual)):\n\t\tsum_score += actual[i] * log(1e-15 + predicted[i])\n\tmean_sum_score = 1.0 \/ len(actual) * sum_score<\/span>\treturn<\/strong> -mean_sum_score<\/span><\/pre>\n
Hinge Loss<\/h2>\n
The hinge loss is an alternative to the cross-entropy loss function. It was initially developed for the performance evaluation of the support vector machine (SVM) algorithm to calculate the maximum margin from the hyperplane to the labelled class.<\/p>\n
It is used with binary classification where the target values fall in the range of -1<\/strong> to 1<\/strong>. This loss function penalizes the wrong predictions by assigning more errors when there is more difference between the actual and predicted value class.<\/p>\n
Here\u2019s the formula for the hinge loss function:<\/p>\n
\n
\n
\"\"<\/figure>
<\/picture><\/div>\n<\/div>\n<\/figure>\n
Where Sj<\/strong> is the actual value and Syi<\/strong> is the predicted value.<\/p>\n
Example<\/strong>:<\/p>\n
import<\/strong> numpy as<\/strong> np<\/span>def hinge_loss<\/strong>(y, y_pred):\n    l = 0\n    size = np.size(y)\n    for i in range(size):\n        l = l + max(0, 1 - y[i] * y_pred[i])<\/span>    return<\/strong> l \/ size<\/span><\/pre>\n
Conclusion<\/h1>\n
That\u2019s all from this article. In this article, we learned about various loss functions in machine learning that are used with regression and classification problems. To achieve better outcomes, it is important to choose the loss function that is the best fitting for our data.<\/p>\n
Thanks for reading!<\/p>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"
Photo by Alexandre Debi\u00e8ve on Unsplash In machine learning, a loss function is used to measure the loss, or cost, of a specific machine learning model. These loss functions calculate the amount of error in a specific machine learning model using some mathematical formula and measure the performance of that specific model. There are various […]<\/p>\n","protected":false},"author":88,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"customer_name":"","customer_description":"","customer_industry":"","customer_technologies":"","customer_logo":"","footnotes":""},"categories":[6,7],"tags":[],"coauthors":[185],"class_list":["post-7397","post","type-post","status-publish","format-standard","hentry","category-machine-learning","category-tutorials"],"yoast_head":"\nGuide to Loss Functions for Machine Learning Models - Comet<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.comet.com\/site\/blog\/guide-to-loss-functions-for-machine-learning-models\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Guide to Loss Functions for Machine Learning Models\" \/>\n<meta property=\"og:description\" content=\"Photo by Alexandre Debi\u00e8ve on Unsplash In machine learning, a loss function is used to measure the loss, or cost, of a specific machine learning model. These loss functions calculate the amount of error in a specific machine learning model using some mathematical formula and measure the performance of that specific model. There are various […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.comet.com\/site\/blog\/guide-to-loss-functions-for-machine-learning-models\/\" \/>\n<meta property=\"og:site_name\" content=\"Comet\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cometdotml\" \/>\n<meta property=\"article:published_time\" content=\"2023-09-07T18:26:15+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-04-24T17:14:20+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*ag5QD6HncPkXdd0P\" \/>\n<meta name=\"author\" content=\"Pralabh Saxena\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@Cometml\" \/>\n<meta name=\"twitter:site\" content=\"@Cometml\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Pralabh Saxena\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Guide to Loss Functions for Machine Learning Models - Comet","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.comet.com\/site\/blog\/guide-to-loss-functions-for-machine-learning-models\/","og_locale":"en_US","og_type":"article","og_title":"Guide to Loss Functions for Machine Learning Models","og_description":"Photo by Alexandre Debi\u00e8ve on Unsplash In machine learning, a loss function is used to measure the loss, or cost, of a specific machine learning model. These loss functions calculate the amount of error in a specific machine learning model using some mathematical formula and measure the performance of that specific model. There are various […]","og_url":"https:\/\/www.comet.com\/site\/blog\/guide-to-loss-functions-for-machine-learning-models\/","og_site_name":"Comet","article_publisher":"https:\/\/www.facebook.com\/cometdotml","article_published_time":"2023-09-07T18:26:15+00:00","article_modified_time":"2025-04-24T17:14:20+00:00","og_image":[{"url":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*ag5QD6HncPkXdd0P","type":"","width":"","height":""}],"author":"Pralabh Saxena","twitter_card":"summary_large_image","twitter_creator":"@Cometml","twitter_site":"@Cometml","twitter_misc":{"Written by":"Pralabh Saxena","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.comet.com\/site\/blog\/guide-to-loss-functions-for-machine-learning-models\/#article","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/blog\/guide-to-loss-functions-for-machine-learning-models\/"},"author":{"name":"Pralabh Saxena","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/661df331deec9788343ef011c9467cc8"},"headline":"Guide to Loss Functions for Machine Learning Models","datePublished":"2023-09-07T18:26:15+00:00","dateModified":"2025-04-24T17:14:20+00:00","mainEntityOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/guide-to-loss-functions-for-machine-learning-models\/"},"wordCount":969,"publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/guide-to-loss-functions-for-machine-learning-models\/#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*ag5QD6HncPkXdd0P","articleSection":["Machine Learning","Tutorials"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.comet.com\/site\/blog\/guide-to-loss-functions-for-machine-learning-models\/","url":"https:\/\/www.comet.com\/site\/blog\/guide-to-loss-functions-for-machine-learning-models\/","name":"Guide to Loss Functions for Machine Learning Models - Comet","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/guide-to-loss-functions-for-machine-learning-models\/#primaryimage"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/guide-to-loss-functions-for-machine-learning-models\/#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*ag5QD6HncPkXdd0P","datePublished":"2023-09-07T18:26:15+00:00","dateModified":"2025-04-24T17:14:20+00:00","breadcrumb":{"@id":"https:\/\/www.comet.com\/site\/blog\/guide-to-loss-functions-for-machine-learning-models\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.comet.com\/site\/blog\/guide-to-loss-functions-for-machine-learning-models\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/blog\/guide-to-loss-functions-for-machine-learning-models\/#primaryimage","url":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*ag5QD6HncPkXdd0P","contentUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*ag5QD6HncPkXdd0P"},{"@type":"BreadcrumbList","@id":"https:\/\/www.comet.com\/site\/blog\/guide-to-loss-functions-for-machine-learning-models\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.comet.com\/site\/"},{"@type":"ListItem","position":2,"name":"Guide to Loss Functions for Machine Learning Models"}]},{"@type":"WebSite","@id":"https:\/\/www.comet.com\/site\/#website","url":"https:\/\/www.comet.com\/site\/","name":"Comet","description":"Build Better Models Faster","publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.comet.com\/site\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.comet.com\/site\/#organization","name":"Comet ML, Inc.","alternateName":"Comet","url":"https:\/\/www.comet.com\/site\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","width":310,"height":310,"caption":"Comet ML, Inc."},"image":{"@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/cometdotml","https:\/\/x.com\/Cometml","https:\/\/www.youtube.com\/channel\/UCmN63HKvfXSCS-UwVwmK8Hw"]},{"@type":"Person","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/661df331deec9788343ef011c9467cc8","name":"Pralabh Saxena","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/image\/af2f89cb395a3afe9b42605f70d9c6a7","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/09\/1689749938719-96x96.jpg","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/09\/1689749938719-96x96.jpg","caption":"Pralabh Saxena"},"url":"https:\/\/www.comet.com\/site\/blog\/author\/pralabh-saxena2014gmail-com\/"}]}},"_links":{"self":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/7397","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/users\/88"}],"replies":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/comments?post=7397"}],"version-history":[{"count":1,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/7397\/revisions"}],"predecessor-version":[{"id":15556,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/7397\/revisions\/15556"}],"wp:attachment":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media?parent=7397"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/categories?post=7397"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/tags?post=7397"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/coauthors?post=7397"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}