\n
Data Preparation<\/h1>\n You need to convert a Pandas Data Frame to a Time Series Data format using the \u201cTimeSeriesData\u201d method. Also, it requires the time dimension and the value dimension, which would require you to rename the Data Frame columns.<\/p>\n
Let\u2019s do this step by step.<\/p>\n
Import Required Libraries<\/strong><\/p>\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom kats.consts import TimeSeriesData<\/span><\/pre>\nRead CSV data and view the top five rows.<\/strong><\/p>\nap_df = pd.read_csv(\"AirPassengers.csv\")\nap_df.head()<\/span><\/pre>\n\n<\/picture><\/div>\nSource: Author<\/figcaption>\n<\/figure>\nThis looks great: you have monthly air passenger numbers from January 1949 to December 1960. You can get this by:<\/p>\n
# Start month in the Data Frame\nap_df.min()<\/span># End month in the Date Frame\nap_df.max()<\/span><\/pre>\nPlot the Air Passenger data.<\/strong><\/p>\nSet the figure size as per your need, then plot a line chart, and finally log your chart to the experiment in the Comet project using log_figure(). This can be viewed under the \u201cGraphics\u201d tab in the project web interface.<\/p>\n\n\n
<\/picture><\/div>\n<\/div>\nSource: Author<\/figcaption>\n<\/figure>\nCode for plotting and logging the figure is as under:<\/p>\n
plt.figure(figsize=(35,20))\nfig = plt.plot(ap_df['Month'], ap_df[\"#Passengers\"])\nplt.xticks(rotation=90)\nexperiment.log_figure(figure_name = \"Complete Data\", figure=fig, overwrite=False)<\/span><\/pre>\n\n\n
<\/picture><\/div>\n<\/div>\nSource: Author<\/figcaption>\n<\/figure>\nFrom the chart, you can observe that there is an increasing trend in the number of air passengers along with an overall increase in the amplitude of variation in yearly traffic. As expected, air travel has a 12-month seasonality.<\/p>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n
\n
\n
\n
Outlier Detection<\/h1>\n Air Passenger data is a model dataset which doesn\u2019t contain any anomalies or outliers. For the purpose of this exercise, let\u2019s add an artificial outlier and observe if Kats is able to find the mouse. Uh oh! the outlier!<\/p>\n
Let\u2019s change one of the values in the Data Frame to a very large value i.e. 700 in this case.<\/p>\n
ap_df['#Passengers'][50]=700<\/span><\/pre>\nLet\u2019s visualize the data to represent the outlier.<\/p>\n
plt.figure(figsize=(15,10))\nfig = plt.plot(ap_df['Month'], ap_df[\"#Passengers\"])\nexperiment.log_figure(figure_name = \"Complete Data with Anomaly\", figure=fig, overwrite=False)<\/span><\/pre>\n\n\n
<\/picture><\/div>\n<\/div>\nSource: Author<\/figcaption>\n<\/figure>\nFor using Kats, let\u2019s do the following:<\/p>\n
\nConvert the \u2018Month\u2019 column to the DateTime format.<\/li>\n Rename the columns in the data frame to \u2018time\u2019 and \u2018value\u2019 respectively.<\/li>\n Convert data frame object to Time Series data object.<\/li>\n Then use the Outlier Detector method to identify outliers.<\/li>\n Now output the top one outlier at zero indexes of the detector object.<\/li>\n<\/ul>\nap_df['Month'] = pd.to_datetime(ap_df['Month'])\nap_df.columns = ['time', 'value']\noutlier_ts = TimeSeriesData(ap_df)\nts_outlierDetection = OutlierDetector(outlier_ts, 'multiplicative')\nts_outlierDetection.detector()<\/span>ts_outlierDetection.outliers[0]<\/span><\/pre>\n\n<\/picture><\/div>\nSource: Author<\/figcaption>\n<\/figure>\nOnce you identify an outlier you can either decide to remove it or replace it with another value. This is called interpolation. Use the code below to interpolate the outlier.<\/p>\n
outlier_ts_interpolated = ts_outlierDetection.remover(interpolate = True)<\/span><\/pre>\nConvert the Time Series Data object to the Data Frame object and rename the columns to the original names.<\/p>\n
ap_df = outlier_ts_interpolated.to_dataframe()\nap_df.columns = ['Month', \"#Passengers\"]<\/span><\/pre>\nNow, let\u2019s visualize the data again to see if the outlier is interpolated well by the algorithm.<\/p>\n
plt.figure(figsize=(15,10))\nfig = plt.plot(ap_df['Month'], ap_df[\"#Passengers\"])\nexperiment.log_figure(figure_name = \"Anomaly Removed\", figure=fig, overwrite=False)<\/span><\/pre>\n\n\n
<\/picture><\/div>\n<\/div>\nSource: Author<\/figcaption>\n<\/figure>\nVoila! You got back the original clean data again.<\/p>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n
\n
\n
\n
Note: If you are using a notebook for running your experiments, don\u2019t forget to call experiement.end()<\/em><\/strong> when you are done.<\/p><\/blockquote>\nSummary<\/h1>\n In this post, you learned how to handle outliers with the Facebook Kats library. You also learned how to create a free community account with comet.com and to log charts using Comet\u2019s Python API.<\/p>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"
Photo by Jake Hills on Unsplash Introduction Time series applications are ubiquitous and find applications in various industries such as supply chain, e-commerce, finance, retail, biotechnology, weather prediction, oil and energy, manufacturing, astronomy, etc. These applications generate data that can be noisy in the real world as some unaccounted factors can influence the measurements. For […]<\/p>\n","protected":false},"author":80,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"customer_name":"","customer_description":"","customer_industry":"","customer_technologies":"","customer_logo":"","footnotes":""},"categories":[6],"tags":[],"coauthors":[177],"class_list":["post-7270","post","type-post","status-publish","format-standard","hentry","category-machine-learning"],"yoast_head":"\n
Outlier Detection in Time Series with Kats and Comet - Comet<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.comet.com\/site\/blog\/outlier-detection-in-time-series-with-kats-and-comet\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Outlier Detection in Time Series with Kats and Comet\" \/>\n<meta property=\"og:description\" content=\"Photo by Jake Hills on Unsplash Introduction Time series applications are ubiquitous and find applications in various industries such as supply chain, e-commerce, finance, retail, biotechnology, weather prediction, oil and energy, manufacturing, astronomy, etc. These applications generate data that can be noisy in the real world as some unaccounted factors can influence the measurements. For […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.comet.com\/site\/blog\/outlier-detection-in-time-series-with-kats-and-comet\/\" \/>\n<meta property=\"og:site_name\" content=\"Comet\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cometdotml\" \/>\n<meta property=\"article:published_time\" content=\"2023-08-21T17:45:49+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-04-24T17:14:34+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*ZXRoM8gPPcCXwiue\" \/>\n<meta name=\"author\" content=\"Ankit Malik\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@Cometml\" \/>\n<meta name=\"twitter:site\" content=\"@Cometml\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Ankit Malik\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"8 minutes\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Outlier Detection in Time Series with Kats and Comet - Comet","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.comet.com\/site\/blog\/outlier-detection-in-time-series-with-kats-and-comet\/","og_locale":"en_US","og_type":"article","og_title":"Outlier Detection in Time Series with Kats and Comet","og_description":"Photo by Jake Hills on Unsplash Introduction Time series applications are ubiquitous and find applications in various industries such as supply chain, e-commerce, finance, retail, biotechnology, weather prediction, oil and energy, manufacturing, astronomy, etc. These applications generate data that can be noisy in the real world as some unaccounted factors can influence the measurements. For […]","og_url":"https:\/\/www.comet.com\/site\/blog\/outlier-detection-in-time-series-with-kats-and-comet\/","og_site_name":"Comet","article_publisher":"https:\/\/www.facebook.com\/cometdotml","article_published_time":"2023-08-21T17:45:49+00:00","article_modified_time":"2025-04-24T17:14:34+00:00","og_image":[{"url":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*ZXRoM8gPPcCXwiue","type":"","width":"","height":""}],"author":"Ankit Malik","twitter_card":"summary_large_image","twitter_creator":"@Cometml","twitter_site":"@Cometml","twitter_misc":{"Written by":"Ankit Malik","Est. reading time":"8 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.comet.com\/site\/blog\/outlier-detection-in-time-series-with-kats-and-comet\/#article","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/blog\/outlier-detection-in-time-series-with-kats-and-comet\/"},"author":{"name":"Ankit Malik","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/3f1178f2089090fdd2ffe0a43bccad23"},"headline":"Outlier Detection in Time Series with Kats and Comet","datePublished":"2023-08-21T17:45:49+00:00","dateModified":"2025-04-24T17:14:34+00:00","mainEntityOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/outlier-detection-in-time-series-with-kats-and-comet\/"},"wordCount":1021,"publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/outlier-detection-in-time-series-with-kats-and-comet\/#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*ZXRoM8gPPcCXwiue","articleSection":["Machine Learning"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.comet.com\/site\/blog\/outlier-detection-in-time-series-with-kats-and-comet\/","url":"https:\/\/www.comet.com\/site\/blog\/outlier-detection-in-time-series-with-kats-and-comet\/","name":"Outlier Detection in Time Series with Kats and Comet - Comet","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/outlier-detection-in-time-series-with-kats-and-comet\/#primaryimage"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/outlier-detection-in-time-series-with-kats-and-comet\/#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*ZXRoM8gPPcCXwiue","datePublished":"2023-08-21T17:45:49+00:00","dateModified":"2025-04-24T17:14:34+00:00","breadcrumb":{"@id":"https:\/\/www.comet.com\/site\/blog\/outlier-detection-in-time-series-with-kats-and-comet\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.comet.com\/site\/blog\/outlier-detection-in-time-series-with-kats-and-comet\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/blog\/outlier-detection-in-time-series-with-kats-and-comet\/#primaryimage","url":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*ZXRoM8gPPcCXwiue","contentUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*ZXRoM8gPPcCXwiue"},{"@type":"BreadcrumbList","@id":"https:\/\/www.comet.com\/site\/blog\/outlier-detection-in-time-series-with-kats-and-comet\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.comet.com\/site\/"},{"@type":"ListItem","position":2,"name":"Outlier Detection in Time Series with Kats and Comet"}]},{"@type":"WebSite","@id":"https:\/\/www.comet.com\/site\/#website","url":"https:\/\/www.comet.com\/site\/","name":"Comet","description":"Build Better Models Faster","publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.comet.com\/site\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.comet.com\/site\/#organization","name":"Comet ML, Inc.","alternateName":"Comet","url":"https:\/\/www.comet.com\/site\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","width":310,"height":310,"caption":"Comet ML, Inc."},"image":{"@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/cometdotml","https:\/\/x.com\/Cometml","https:\/\/www.youtube.com\/channel\/UCmN63HKvfXSCS-UwVwmK8Hw"]},{"@type":"Person","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/3f1178f2089090fdd2ffe0a43bccad23","name":"Ankit Malik","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/image\/970af032afc7d64150dc70c5b5047e00","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/08\/1687779949530-96x96.jpg","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/08\/1687779949530-96x96.jpg","caption":"Ankit Malik"},"url":"https:\/\/www.comet.com\/site\/blog\/author\/ank-dsmgmail-com\/"}]}},"_links":{"self":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/7270","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/users\/80"}],"replies":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/comments?post=7270"}],"version-history":[{"count":1,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/7270\/revisions"}],"predecessor-version":[{"id":15570,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/7270\/revisions\/15570"}],"wp:attachment":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media?parent=7270"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/categories?post=7270"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/tags?post=7270"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/coauthors?post=7270"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:466\/1*chEz8_Q4TNCVyUNWH23p2g.png\")
<\/figure>![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*qc6hzFuExdIMZr1a68pxlA.jpeg\")
<\/figure>![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*be_fSFxjzNeyB10qpl2bnA.png\")
<\/figure>![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*Hn3tccNVhZE9ySYlQNl_yw.png\")
<\/figure>![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*OrpUOFKHgbSc-ZUEmPIiYA.png\")
<\/figure>![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*MEf-BB-8Zw1RBtkIH-5LAQ.png\")
<\/figure>![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*pozkykz32CiGlpeSK_S78Q.png\")
<\/figure>![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:344\/1*88YffRddQs7241dhGz2Oeg.png\")
<\/figure>![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*EQdF_xx42rikGQu9SZmTFQ.png\")
<\/figure>![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*7-Z2TkE6_VhMlygkctHWxw.png\")
<\/figure>![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*I4Cvu7qZ0FCc4dU9h3Ft7g.png\")
<\/figure>![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:584\/1*pmhf7itKqhPVNso_tz8kIQ.png\")
<\/figure>![\"\"](\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*Rq7rKJvTHavNdyrIRHUqbA.png\")
<\/figure>