{"id":1449,"date":"2022-03-30T16:10:30","date_gmt":"2022-03-31T00:10:30","guid":{"rendered":"https:\/\/live-cometml.pantheonsite.io\/?post_type=press_release&amp;p=1449"},"modified":"2022-03-30T16:10:30","modified_gmt":"2022-03-31T00:10:30","slug":"comet-releases-new-survey-highlighting-ais-latest-challenges-too-much-friction-too-little-ml","status":"publish","type":"press_release","link":"https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/comet-releases-new-survey-highlighting-ais-latest-challenges-too-much-friction-too-little-ml\/","title":{"rendered":"Comet Releases New Survey Highlighting AI\u2019s Latest Challenges: Too Much Friction, Too Little ML"},"content":{"rendered":"<p><a href=\"https:\/\/live-cometml.pantheonsite.io\/\">Comet<\/a>, provider of the leading development platform for enterprise machine learning (ML) teams, today announced the results of its recent survey of machine learning professionals. Hundreds of enterprise ML practitioners were asked about their experiences and the factors that affected their teams\u2019 ability to deliver the level of business value their organizations expected from ML initiatives. Rather than attaining desired outcomes, however, many survey respondents revealed that they lack the right resources, or they shared that the resources they have are often misaligned. As a result, many AI initiatives have been far less productive than they could be.<\/p>\n<p>Issues include the fact that many experiments are abandoned because some part of the data science lifecycle was mismanaged. This is due in large part to the manual tracking processes that organizations often put in place, which hinder effective team collaboration, and are not adaptable, scalable, or reliable. Of models actually deployed into production, nearly one quarter failed in the real world for more than half (56. 5%) of the companies surveyed. As such, the full business value of machine learning is rarely captured.<\/p>\n<p>\u201cThere has been so much enthusiasm around AI, and ML specifically, over the past several years based on its potential, but the realities of generating experiments and deploying models have often fallen well short of expectations,\u201d said Gideon Mendels, CEO and co-founder of Comet. \u201cWe wanted to look deeper into where the friction lies so that issues can be addressed.\u201d<\/p>\n<p><strong>By the Numbers<\/strong><br \/>\nAccording to survey respondents:<\/p>\n<ul>\n<li><strong>Significant time, resources and budgets are being wasted.<\/strong> While teams expect to run, adjust, and re-run experiments as part of model development, 68% of respondents admit to scrapping a whopping 40 \u2013 80% of their experiments altogether. This was due to breakdowns that occur throughout the machine learning lifecycle outside of the normal iterative process of experimentation.<\/li>\n<li><strong>There is a serious lag in model deployment:<\/strong> Only 6% of teams surveyed have been able to take a model live in under 30 days. By contrast, 47% of ML teams require four to six months to deploy a single ML project, while another 43% take up to three months. This can cause unnecessary delays in delivering value to the respective lines of business.<\/li>\n<li><strong>Budgets for tools that could address issues are woefully inadequate:<\/strong> Despite the enthusiasm for ML overall, 88% of respondents have an annual budget of less than $75,000 for machine learning tools and infrastructure. This is far less than the average salary for a single data scientist and dwarfed by the opportunity cost resulting from under-investment.<\/li>\n<li><strong>58% of machine learning teams track all or at least some piece of their experiments manually.<\/strong> This places an enormous strain on workers, causes projects to take far longer to complete, creates challenges for team collaboration and model lineage tracking, hinders model auditability, and leads to unintentional mistakes.<\/li>\n<li><strong>Companies are not intentionally withholding budgets or misallocating ML resources.<\/strong> They simply \u201cdon\u2019t know what they don\u2019t know,\u201d as ML is still an emerging discipline. Of survey respondents, 63% said their organizations would increase ML budgets for 2022, but whether or not these funds will be devoted to the right tools and resources remains to be seen.<\/li>\n<\/ul>\n<p><strong>State of Enterprise ML Today<\/strong><br \/>\nML has demonstrated that it can deliver outsized business value and exciting technological innovation, and as such more companies are seeking to apply it, only to find the tools and processes to be nascent, disconnected, and complex. This makes it difficult to collaborate among teams and uncover the insights that drive business forward. Even though many organizations are good at identifying ML use cases and initiating projects, they fall short in investing in <a href=\"https:\/\/live-cometml.pantheonsite.io\/blog\/comet-office-hoursseven-simple-steps-to-standardizing-the-ml-experiment2-23-22\/\">ML operations best practices<\/a> and the tools and resources needed to make these machine learning initiatives as clear, efficient, and scalable as possible.<\/p>\n<p>Developing effective ML models requires a lot of experiments. These experiments can involve changing the model itself or tweaking its hyperparameters. They may utilize different datasets or involve changing code to evaluate how the algorithms behave differently. When developing and training an ML model, all these changes happen repeatedly, sometimes with only minute differences each time. This makes it difficult for <a href=\"https:\/\/www.comet.com\/site\/products\/ml-experiment-tracking\/\">keep track of which experiments<\/a> and which parameters produced which results\u2014including details such as scripts, the runtime environment, configuration files, data versions, hyperparameters, metrics, weights, and more. Poor experiment management leads to the inability to reproduce results accurately and consistently, and it can throw an entire project off the rails, wasting countless hours of a team\u2019s work.<\/p>\n<p>\u201cEven though companies are prepared to allocate more money and resources to ML programs, they must address some core operational issues first if they want to see a positive return on their investment,\u201d said Mendels. \u201cIf teams are maxed out and struggling with visibility, reproducibility and cost-efficiency today, it will be difficult for them to add more models, experiments and deployments this year, as they expressed the desire to do. Successful ML outcomes depend on people; and with the right tools, teams can avoid burnout.<\/p>\n<p>\u201cDespite the challenges teams and organizations face, there is good news that tools are advancing rapidly to solve for these problems,\u201d added Mendels. \u201cLeading edge companies that have <a href=\"https:\/\/live-cometml.pantheonsite.io\/customers\/uber\/\">implemented modern AI development<\/a> platforms are realizing the benefits, full potential, and value from their machine learning initiatives, which is quite exciting.\u201d<\/p>\n<p>View the complete report: <a href=\"https:\/\/go.comet.ml\/report-machine-learning-practitioners-survey.html\" target=\"_blank\" rel=\"noopener\">2021 ML Practitioner Survey<\/a><br \/>\nView the <a href=\"https:\/\/go.comet.ml\/rs\/912-JJP-445\/images\/Comet_Infographic-2021_ML_Practitioner_Survey.png\" target=\"_blank\" rel=\"noopener\">infographic<\/a><\/p>\n<p><strong>Methodology<\/strong><br \/>\nComet commissioned an online survey from Censuswide of 508 U.S. machine learning practitioners across industries. Respondents answered multiple-choice questions about ML development and factors affecting their teams\u2019 ability to achieve business value using data and machine learning.<\/p>\n<p><strong>About Comet<\/strong><br \/>\nComet\u2019s platform empowers data scientists and machine learning (ML) engineers to accelerate ML development by managing and optimizing the entire ML lifecycle, from experiment tracking to model production monitoring. Trusted by over 150 customers including Ancestry, Cepsa, Etsy, Uber and Zappos, Comet has built a community of tens of thousands of users and academic teams who use the platform free of charge to accelerate research in their fields of study. Comet is headquartered in New York City, with a remote global workforce. To learn more about Comet, visit <a href=\"https:\/\/live-cometml.pantheonsite.io\/\">comet.com<\/a>. To join our community, visit <a href=\"https:\/\/heartbeat.comet.ml\/\">heartbeat.comet.ml<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Comet, provider of the leading development platform for enterprise machine learning (ML) teams, today announced the results of its recent survey of machine learning professionals. Hundreds of enterprise ML practitioners were asked about their experiences and the factors that affected their teams\u2019 ability to deliver the level of business value their organizations expected from ML [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":1448,"template":"","meta":{"customer_name":"","customer_description":"","customer_industry":"","customer_technologies":"","customer_logo":"","subtitle":"Majority of machine learning professionals admit to abandoning 40-60% of their experiments","footnotes":""},"coauthors":[],"class_list":["post-1449","press_release","type-press_release","status-publish","has-post-thumbnail","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v25.9 (Yoast SEO v25.9) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Comet Releases Survey Highlighting AI\u2019s Latest Challenges<\/title>\n<meta name=\"description\" content=\"Results of Comet&#039;s survey of ML professionals show too much friction, too little ML. Most teams abandon 40-60% of experiments \u2014 Comet can fix that.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/comet-releases-new-survey-highlighting-ais-latest-challenges-too-much-friction-too-little-ml\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Comet Releases New Survey Highlighting AI\u2019s Latest Challenges: Too Much Friction, Too Little ML\" \/>\n<meta property=\"og:description\" content=\"Results of Comet&#039;s survey of ML professionals show too much friction, too little ML. Most teams abandon 40-60% of experiments \u2014 Comet can fix that.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/comet-releases-new-survey-highlighting-ais-latest-challenges-too-much-friction-too-little-ml\/\" \/>\n<meta property=\"og:site_name\" content=\"Comet\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cometdotml\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/06\/logo_comet-4.png\" \/>\n\t<meta property=\"og:image:width\" content=\"876\" \/>\n\t<meta property=\"og:image:height\" content=\"504\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@Cometml\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"5 minutes\" \/>\n\t<meta name=\"twitter:label2\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data2\" content=\"engineering@atre.net\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Comet Releases Survey Highlighting AI\u2019s Latest Challenges","description":"Results of Comet's survey of ML professionals show too much friction, too little ML. Most teams abandon 40-60% of experiments \u2014 Comet can fix that.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/comet-releases-new-survey-highlighting-ais-latest-challenges-too-much-friction-too-little-ml\/","og_locale":"en_US","og_type":"article","og_title":"Comet Releases New Survey Highlighting AI\u2019s Latest Challenges: Too Much Friction, Too Little ML","og_description":"Results of Comet's survey of ML professionals show too much friction, too little ML. Most teams abandon 40-60% of experiments \u2014 Comet can fix that.","og_url":"https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/comet-releases-new-survey-highlighting-ais-latest-challenges-too-much-friction-too-little-ml\/","og_site_name":"Comet","article_publisher":"https:\/\/www.facebook.com\/cometdotml","og_image":[{"width":876,"height":504,"url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/06\/logo_comet-4.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_site":"@Cometml","twitter_misc":{"Est. reading time":"5 minutes","Written by":"engineering@atre.net"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/comet-releases-new-survey-highlighting-ais-latest-challenges-too-much-friction-too-little-ml\/","url":"https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/comet-releases-new-survey-highlighting-ais-latest-challenges-too-much-friction-too-little-ml\/","name":"Comet Releases Survey Highlighting AI\u2019s Latest Challenges","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/comet-releases-new-survey-highlighting-ais-latest-challenges-too-much-friction-too-little-ml\/#primaryimage"},"image":{"@id":"https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/comet-releases-new-survey-highlighting-ais-latest-challenges-too-much-friction-too-little-ml\/#primaryimage"},"thumbnailUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/06\/logo_comet-4.png","datePublished":"2022-03-31T00:10:30+00:00","description":"Results of Comet's survey of ML professionals show too much friction, too little ML. Most teams abandon 40-60% of experiments \u2014 Comet can fix that.","breadcrumb":{"@id":"https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/comet-releases-new-survey-highlighting-ais-latest-challenges-too-much-friction-too-little-ml\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/comet-releases-new-survey-highlighting-ais-latest-challenges-too-much-friction-too-little-ml\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/comet-releases-new-survey-highlighting-ais-latest-challenges-too-much-friction-too-little-ml\/#primaryimage","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/06\/logo_comet-4.png","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/06\/logo_comet-4.png","width":876,"height":504,"caption":"Machine Learning Platform | Comet ML"},{"@type":"BreadcrumbList","@id":"https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/comet-releases-new-survey-highlighting-ais-latest-challenges-too-much-friction-too-little-ml\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.comet.com\/site\/"},{"@type":"ListItem","position":2,"name":"Press Releases","item":"https:\/\/www.comet.com\/site\/about-us\/news\/press-releases\/"},{"@type":"ListItem","position":3,"name":"Comet Releases New Survey Highlighting AI\u2019s Latest Challenges: Too Much Friction, Too Little ML"}]},{"@type":"WebSite","@id":"https:\/\/www.comet.com\/site\/#website","url":"https:\/\/www.comet.com\/site\/","name":"Comet","description":"Build Better Models Faster","publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.comet.com\/site\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.comet.com\/site\/#organization","name":"Comet ML, Inc.","alternateName":"Comet","url":"https:\/\/www.comet.com\/site\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","width":310,"height":310,"caption":"Comet ML, Inc."},"image":{"@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/cometdotml","https:\/\/x.com\/Cometml","https:\/\/www.youtube.com\/channel\/UCmN63HKvfXSCS-UwVwmK8Hw"]}]}},"_links":{"self":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/press_release\/1449","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/press_release"}],"about":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/types\/press_release"}],"author":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/users\/1"}],"version-history":[{"count":0,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/press_release\/1449\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media\/1448"}],"wp:attachment":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media?parent=1449"}],"wp:term":[{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/coauthors?post=1449"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}