{"id":7876,"date":"2023-10-06T15:37:09","date_gmt":"2023-10-06T23:37:09","guid":{"rendered":"https:\/\/live-cometml.pantheonsite.io\/?p=7876"},"modified":"2025-04-24T17:05:39","modified_gmt":"2025-04-24T17:05:39","slug":"major-problems-of-machine-learning-datasets-part-2","status":"publish","type":"post","link":"https:\/\/www.comet.com\/site\/blog\/major-problems-of-machine-learning-datasets-part-2\/","title":{"rendered":"Major Problems of Machine Learning Datasets: Part 2"},"content":{"rendered":"\n\n\n\n\n
\n
\n
\n
\n
\n
\"\"<\/figure>
<\/picture><\/div>\n<\/div>
Photo by Elisa Ventur<\/a> on Unsplash<\/a><\/figcaption><\/figure>\n

Outliers in data<\/h1>\n

Outliers are unusual data points that differ significantly from other values in the sample of a population. Outliers sometimes represent errors in measurement or data collection, and can have significant effects on descriptive statistics and machine learning model outcomes. There are several ways to detect outliers in our data, and here we will discuss two methods: standard deviation and box plots.<\/p>\n

1. Standard deviation<\/strong>: In this method, we choose a minimum and maximum standard deviation threshold, and data points outside this limit are considered outliers.<\/p>\n

2. Box Plots<\/strong>: Box plots are widely used graphical representations that make use of the 25th, 50th (median), and 75th quartiles of a data\u2019s distribution to show a visual representation of outliers.<\/p>\n

How To Deal With Outliers?<\/h2>\n

Once we\u2019ve detected outliers, we handle them by removing or replacing them with some value.<\/p>\n

1. Dropping outliers: <\/strong>This is one of the simplest methods to handle outliers. Here, we remove all data points that fall outside a specified threshold or boundary. Typically, these thresholds are defined as numbers of standard deviations from the mean, but for simplicity, in the example below, we set a lower limit at 5% and an upper limit at 95% of values.<\/p>\n

Disadvantage<\/strong>: The biggest disadvantage of dropping outliers is the loss of data. Especially when outliers are not due to error, they may contain some very valuable information.<\/p>\n

import matplotlib.pyplot as plt\nimport seaborn as sns\nimport pandas as pd<\/span>fig, axes = plt.subplots(1, 2)\nplt.tight_layout(0.2) <\/span>df = pd.read_csv('titanic_with_no_nan.csv<\/a>')\nprint(\"Before Shape:\", df.shape) <\/span>max_val = df.Age.quantile(0.95)\nmin_val = df.Age.quantile(0.05) <\/span>df2 = df[(df['Age'] > min_val) & (df['Age'] < max_val)]\nprint(\"After Shape:\", df2.shape)<\/span>sns.boxplot(df['Age'], orient='v', ax=axes[0])\naxes[0].title.set_text(\"Before\")\nsns.boxplot(df2['Age'], orient='v', ax=axes[1])\naxes[1].title.set_text(\"After\")\nplt.show()<\/span><\/pre>\n
\n
\"\"<\/figure>
<\/picture><\/div>\n<\/figure>\n
2. Replacing outliers with custom percentiles<\/strong>: Using this method, instead of dropping values outside a particular threshold range, we replace them with minimum and maximum threshold values.<\/p>\n
import matplotlib.pyplot as plt\nimport seaborn as sns\nimport pandas as pd<\/span>fig, axes = plt.subplots(1, 2)\nplt.tight_layout(0.2) <\/span>df = pd.read_csv('titanic_with_no_nan.csv<\/a>')\nprint(\"Before Shape:\", df.shape) <\/span>max_val = df.Age.quantile(0.95)\nmin_val = df.Age.quantile(0.05) <\/span>df2 = df[(df['Age'] > min_val) & (df['Age'] < max_val)]\nprint(\"After Shape:\", df2.shape)<\/span>sns.boxplot(df['Age'], orient='v', ax=axes[0])\naxes[0].title.set_text(\"Before\")\nsns.boxplot(df2['Age'], orient='v', ax=axes[1])\naxes[1].title.set_text(\"After\")\nplt.show()<\/span><\/pre>\n
3. Replacing outliers using IQR<\/strong>: The Interquartile range (IQR) is a measure of statistical dispersion that divides datapoints into units, or quantiles. IQR helps measure the spread of data.<\/p>\n
IQR primarily utilizes three quantiles: Q1, which represents the 25th quantile, Q2 which represents the 50th quantile, and Q3, which represents the 75th quantile. Additionally, Q1 is also the median of the first half of the data, Q2 is the median of the whole dataset and Q3 is the median of the second half of the data.<\/p>\n
The interquartile range (IQR) is calculated by subtracting Q3 from Q1. One method of eliminating outliers is to place thresholds at 1.5 IQRs past the second and third quantiles in each direction.<\/p>\n
import numpy as np\nimport matplotlib.pyplot as plt\nimport warnings\nwarnings.filterwarnings(\"ignore\")\nfig, axes = plt.subplots(1, 2)\nplt.tight_layout(0.2)<\/span>df = pd.read_csv('data\/titanic_with_no_nan.csv')\nprint(\"Previous Shape With Outlier: \", df.shape)\nsns.boxplot(df['Age'], orient='v', ax=axes[0])\naxes[0].title.set_text(\"Before\")<\/span>Q1 = df.Age.quantile(0.25)\nQ3 = df.Age.quantile(0.75)\nprint(Q1, Q3)<\/span>IQR = Q3-Q1\nprint(IQR)<\/span>lower_limit = Q1 - 1.5*IQR\nupper_limit = Q3 + 1.5*IQR\nprint(lower_limit, upper_limit)<\/span>df2 = df.copy()\ndf2['Age'] = np.where(df2['Age']>upper_limit,upper_limit,df2['Age'])\ndf2['Age'] = np.where(df2['Age']<lower_limit,lower_limit,df2['Age'])<\/span>print(\"Shape After Removing Outliers:\", df2.shape)\nsns.boxplot(df2['Age'], orient='v', ax=axes[1])\naxes[1].title.set_text(\"After\")\nplt.show()<\/span><\/pre>\n
Feature Selection<\/h1>\n
Unwanted features<\/h2>\n
Machine learning datasets are often collected with the help of web scraping or APIs, and may likely contain features we are not interested in. For example, a dataset for predicting health outcomes might contain names or other PII we\u2019d rather not include in our project. Or maybe we\u2019re looking to predict income, and have found a dataset that also contains information about individuals\u2019 pets. In situations where your data contains features that are irrelevant, or potentially unethical, it is better to drop all of these columns and not include them in the model-building process. Later in the article we will also discuss how to handle features that may be correlated.<\/p>\n
Determining which features are most relevant to a particular task is not always so straightforward as in the examples above, however. In these situations, it may be helpful to use a feature selection technique to help us identify features with less importance. These techniques often rank features based on their importance in predicting an outcome. You can then filter all the features using a threshold value for extracting the most important features.<\/p>\n
    \n
  1. Extra Trees model<\/strong>: The Extra Trees estimator fits n<\/code> number of decision trees to create a more generalized model with less bias. The Extra Trees estimator class has an attribute called feature_importance<\/code> that makes use of the Gini Index to calculate the importance of features on predicting the outcome.<\/li>\n<\/ol>\n
    from sklearn.ensemble import ExtraTreesClassifier\nmodel=ExtraTreesClassifier()\nmodel.fit(X,y)\nprint(model.feature_importances_)<\/span><\/pre>\n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/figure>\n
    We can make this a bit more attractive, by plotting the results above with matplotlib<\/code>.<\/p>\n
    from matplotlib import pyplot as plt\ntop_features=pd.Series(model.feature_importances_, index=X.columns)\ntop_features.nlargest(10).plot(kind='barh')\nplt.show()<\/span><\/pre>\n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/figure>\n
    2. Mutual Information<\/strong>: In probability and information theory, mutual information is the measurement of mutual dependence between two variables. A measurement of 0 suggests two variables are completely independent of each other. Note this method may only be used on discrete numerical features and targets.<\/p>\n
    from sklearn.feature_selection import mutual_info_classif\nmutual_info = mutual_info_classif(X,y)\nmutual_data = pd.Series(mutual_info,index=X.columns)\nmutual_data.sort_values(ascending=False)[:10]<\/span><\/pre>\n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/figure>\n
    3. <\/strong>SelectKBest<\/strong><\/a>: Scikitlearn provides the SelectKBest<\/code> class for selecting the best (most important) features from a given dataset using a user-defined score metric (default is ANOVA F-value). It selects K<\/code> top features according to the highest score. Below, we use this method with the chi2<\/code> scoring function.<\/p>\n
    from sklearn.feature_selection import SelectKBest\nfrom sklearn.feature_selection import chi2<\/span>ordered_rank_features= SelectKBest(score_func= chi2, k= 10)\nordered_feature= ordered_rank_features.fit(X, y)<\/span>dfscores= pd.DataFrame(ordered_feature.scores_, columns= [\"Score\"])\ndfcolumns= pd.DataFrame(X.columns)<\/span>features_rank= pd.concat([dfcolumns, dfscores], axis=1)\nfeatures_rank.columns= ['Features', 'Score']\ntop_10 = features_rank.nlargest(10, 'Score')<\/span>plt.figure(figsize= (15, 8))\nplt.bar(data= top_10,x= 'Features', height= 'Score')<\/span><\/pre>\n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/figure>\n
    Some other methods that provide options for feature selection include Lasso Regression<\/a>, Anova Test<\/a>, and more.<\/p>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n
    \n
    \n
    \n
    Innovation and academia go hand-in-hand. Listen to our own CEO Gideon Mendels chat with the Stanford MLSys Seminar Series team about the future of MLOps and give the Comet platform a try for free<\/a>!<\/p><\/blockquote>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n
    \n
    \n
    \n
    Correlation<\/h1>\n
    Correlation is a statistical measure that expresses the relation between two variables. Variables can be positively or negatively correlated to each other. A positive correlation occurs when an increase in variable A leads to an increase in variable B. On the other hand, a negative correlation occurs when an increase in variable A leads to a decrease in variable B.<\/p>\n
    The range of correlation values is -1 to 1, where 1 represents completely, positively correlated features, and -1 represents completely negatively correlated features.<\/p>\n
    Having two or more highly correlated features in our training data will lead to the problem of multicollinearity<\/strong>, which affects model performance.<\/p>\n
    How to deal with correlated features?<\/h2>\n
    The pandas DataFrame class offers the corr<\/code> method, which computes the pairwise correlation of columns, excluding NaN<\/code> or null values.<\/p>\n
    DataFrame.corr(method='pearson', min_periods=1)<\/span><\/pre>\n
    The corr<\/code> method can calculate three types correlation metrics: Pearson<\/a>(default), Kendall<\/a>, and spearman<\/a>.<\/em><\/p>\n
    import pandas as pd<\/span>import seaborn as sns\nimport matplotlib.pyplot as plt<\/span>df = pd.read_csv('https:\/\/raw.githubusercontent.com\/Abhayparashar31\/datasets\/master\/titanic_with_no_nan.csv')<\/span>df = df.drop(['Name','PassengerId','Cabin'],axis=1)\ndf = pd.get_dummies(df,columns=['Embarked','Sex'])\ndf.corr()<\/span><\/pre>\n
    \n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/div>\n<\/figure>\n
    Analyzing the correlation matrix in this format can be a bit hard. Let\u2019s visualize it using a seaborn heatmap instead:<\/p>\n
    sns.heatmap(df.corr(),annot=True,cmap='RdYlGn',linewidths=0.2)\nfig=plt.gcf()\nfig.set_size_inches(20,12)\nplt.show()<\/span><\/pre>\n
    \n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/div>\n<\/figure>\n
    By looking at the above heatmap, we can clearly see the highest positive correlation value is 0.41, which is between our features Parch<\/code> and SibSp<\/code>. On the other hand highest negative correlation value is -1 which is between the sex categories female<\/code> and male<\/code> (because they read as binary variables by the algorithm in this particular case).<\/p>\n
    To remove highly correlated features from our data we can set a threshold value and filter all the features that are correlated with another feature by, for example, more than 60%.<\/p>\n
    Based on our data and heatmap we should get one column between (Sex_male<\/code>, Sex_female<\/code>) and one column between (Embarked_S<\/code>, Embarked_C<\/code>):<\/p>\n
    # Threshold Value\nthreshold = 0.60<\/span># Empty List to Store Column Names\ncol_corr = []<\/span># Correlation matrix\ncorr_matrix = df.corr()<\/span>for i in range(len(corr_matrix.columns)):\n  for j in range(i):\n    if abs(corr_matrix.iloc[i, j]) > threshold:\n      colname = corr_matrix.columns[i]\n      col_corr.append(colname)<\/span>print(col_corr)<\/span><\/pre>\n
    \n
    \"\"<\/figure>
    <\/picture><\/div>\n<\/figure>\n
    Now that we have names of highly correlated features (according to our pre-set threshold), we can simply remove them from our data frame using the drop function.<\/p>\n
    df.drop(columns=col_corr, axis=1, inplace=True)<\/span><\/pre>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"
    Photo by Elisa Ventur on Unsplash Outliers in data Outliers are unusual data points that differ significantly from other values in the sample of a population. Outliers sometimes represent errors in measurement or data collection, and can have significant effects on descriptive statistics and machine learning model outcomes. There are several ways to detect outliers […]<\/p>\n","protected":false},"author":8,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"customer_name":"","customer_description":"","customer_industry":"","customer_technologies":"","customer_logo":"","footnotes":""},"categories":[6],"tags":[],"coauthors":[140],"class_list":["post-7876","post","type-post","status-publish","format-standard","hentry","category-machine-learning"],"yoast_head":"\nMajor Problems of Machine Learning Datasets: Part 2 - Comet<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.comet.com\/site\/blog\/major-problems-of-machine-learning-datasets-part-2\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Major Problems of Machine Learning Datasets: Part 2\" \/>\n<meta property=\"og:description\" content=\"Photo by Elisa Ventur on Unsplash Outliers in data Outliers are unusual data points that differ significantly from other values in the sample of a population. Outliers sometimes represent errors in measurement or data collection, and can have significant effects on descriptive statistics and machine learning model outcomes. There are several ways to detect outliers […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.comet.com\/site\/blog\/major-problems-of-machine-learning-datasets-part-2\/\" \/>\n<meta property=\"og:site_name\" content=\"Comet\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cometdotml\" \/>\n<meta property=\"article:published_time\" content=\"2023-10-06T23:37:09+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-04-24T17:05:39+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*mJOf8ke5gq4iCFFk\" \/>\n<meta name=\"author\" content=\"Abhay Parashar\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@Cometml\" \/>\n<meta name=\"twitter:site\" content=\"@Cometml\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Abhay Parashar\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"9 minutes\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Major Problems of Machine Learning Datasets: Part 2 - Comet","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.comet.com\/site\/blog\/major-problems-of-machine-learning-datasets-part-2\/","og_locale":"en_US","og_type":"article","og_title":"Major Problems of Machine Learning Datasets: Part 2","og_description":"Photo by Elisa Ventur on Unsplash Outliers in data Outliers are unusual data points that differ significantly from other values in the sample of a population. Outliers sometimes represent errors in measurement or data collection, and can have significant effects on descriptive statistics and machine learning model outcomes. There are several ways to detect outliers […]","og_url":"https:\/\/www.comet.com\/site\/blog\/major-problems-of-machine-learning-datasets-part-2\/","og_site_name":"Comet","article_publisher":"https:\/\/www.facebook.com\/cometdotml","article_published_time":"2023-10-06T23:37:09+00:00","article_modified_time":"2025-04-24T17:05:39+00:00","og_image":[{"url":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*mJOf8ke5gq4iCFFk","type":"","width":"","height":""}],"author":"Abhay Parashar","twitter_card":"summary_large_image","twitter_creator":"@Cometml","twitter_site":"@Cometml","twitter_misc":{"Written by":"Abhay Parashar","Est. reading time":"9 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.comet.com\/site\/blog\/major-problems-of-machine-learning-datasets-part-2\/#article","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/blog\/major-problems-of-machine-learning-datasets-part-2\/"},"author":{"name":"Team Comet Digital","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/6266601170c60a7a82b3e0043fbe8ddf"},"headline":"Major Problems of Machine Learning Datasets: Part 2","datePublished":"2023-10-06T23:37:09+00:00","dateModified":"2025-04-24T17:05:39+00:00","mainEntityOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/major-problems-of-machine-learning-datasets-part-2\/"},"wordCount":1086,"publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/major-problems-of-machine-learning-datasets-part-2\/#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*mJOf8ke5gq4iCFFk","articleSection":["Machine Learning"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.comet.com\/site\/blog\/major-problems-of-machine-learning-datasets-part-2\/","url":"https:\/\/www.comet.com\/site\/blog\/major-problems-of-machine-learning-datasets-part-2\/","name":"Major Problems of Machine Learning Datasets: Part 2 - Comet","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/major-problems-of-machine-learning-datasets-part-2\/#primaryimage"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/major-problems-of-machine-learning-datasets-part-2\/#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*mJOf8ke5gq4iCFFk","datePublished":"2023-10-06T23:37:09+00:00","dateModified":"2025-04-24T17:05:39+00:00","breadcrumb":{"@id":"https:\/\/www.comet.com\/site\/blog\/major-problems-of-machine-learning-datasets-part-2\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.comet.com\/site\/blog\/major-problems-of-machine-learning-datasets-part-2\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/blog\/major-problems-of-machine-learning-datasets-part-2\/#primaryimage","url":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*mJOf8ke5gq4iCFFk","contentUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*mJOf8ke5gq4iCFFk"},{"@type":"BreadcrumbList","@id":"https:\/\/www.comet.com\/site\/blog\/major-problems-of-machine-learning-datasets-part-2\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.comet.com\/site\/"},{"@type":"ListItem","position":2,"name":"Major Problems of Machine Learning Datasets: Part 2"}]},{"@type":"WebSite","@id":"https:\/\/www.comet.com\/site\/#website","url":"https:\/\/www.comet.com\/site\/","name":"Comet","description":"Build Better Models Faster","publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.comet.com\/site\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.comet.com\/site\/#organization","name":"Comet ML, Inc.","alternateName":"Comet","url":"https:\/\/www.comet.com\/site\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","width":310,"height":310,"caption":"Comet ML, Inc."},"image":{"@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/cometdotml","https:\/\/x.com\/Cometml","https:\/\/www.youtube.com\/channel\/UCmN63HKvfXSCS-UwVwmK8Hw"]},{"@type":"Person","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/6266601170c60a7a82b3e0043fbe8ddf","name":"Team Comet Digital","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/image\/4f0c0a8cc7c0e87c636ff6a420a6647c","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/08\/Screen-Shot-2023-08-12-at-8.58.50-AM-96x96.png","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/08\/Screen-Shot-2023-08-12-at-8.58.50-AM-96x96.png","caption":"Team Comet Digital"},"sameAs":["https:\/\/www.comet.ml\/"],"url":"https:\/\/www.comet.com\/site\/blog\/author\/teamcometdigital\/"}]}},"_links":{"self":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/7876","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/comments?post=7876"}],"version-history":[{"count":1,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/7876\/revisions"}],"predecessor-version":[{"id":15503,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/7876\/revisions\/15503"}],"wp:attachment":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media?parent=7876"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/categories?post=7876"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/tags?post=7876"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/coauthors?post=7876"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
There are other methods that make use of unsupervised machine learning to detect outliers, like Elliptic Envelope<\/a>, Isolation Forest<\/a>, One-class SVM<\/a>, and Local Outlier Factor<\/a>. You can read about all of them here<\/strong><\/a>.<\/p>\n