{"id":2232,"date":"2020-07-31T13:57:53","date_gmt":"2020-07-31T21:57:53","guid":{"rendered":"https:\/\/live-cometml.pantheonsite.io\/blog\/investing-in-ai-unlocking-profitable-machine-learning-with-experiment-management\/"},"modified":"2025-04-24T17:30:42","modified_gmt":"2025-04-24T17:30:42","slug":"investing-in-ai-unlocking-profitable-machine-learning-with-experiment-management","status":"publish","type":"post","link":"https:\/\/www.comet.com\/site\/blog\/investing-in-ai-unlocking-profitable-machine-learning-with-experiment-management\/","title":{"rendered":"Investing in AI: Unlocking Profitable Machine Learning with Experiment Management"},"content":{"rendered":"\n

This post was originally published as a sponsored post<\/a> by Dell Technologies and Intel on CIO.com.<\/em><\/p>\n\n\n\n

We live in an age of rapid AI innovation and progress. Yet even as academics and researchers make astonishing advancements, demonstrating real business value and positive return on investment is challenging.  Developing cutting edge AI applications based on machine learning models integrated with existing business software is a common challenge. This article discusses a few of the core pain points and strategies to address them.<\/p>\n\n\n\n

The first challenge most organizations encounter is the increased complexity of preparing data and dataset management. Companies may be sitting on troves of valuable data that is unstructured and disorganized and therefore unusable for machine learning modeling. The second common challenge is a lack of infrastructure and tooling necessary for data scientists to record data transformation and model development while moving quickly through alternative designs. Machine learning is an inherently iterative science (as opposed to the determinism of software engineering<\/a>). Without tools that natively support design iteration, teams struggle to advance their research. The final pain point is productionization: even if a team develops a good model, how do they deploy it outside the lab in a business setting? Also, how do they monitor its performance and accuracy in production?<\/p>\n\n\n\n

All of these problems together constitute an unfortunate truth: that for many enterprises, AI application development has become a DevOps challenge rather than a data science challenge. Research Scientists with gleaming PhDs are often found wrestling with infrastructure, job schedulers and compute sources rather than building and training valuable models for their unique business cases.<\/p>\n\n\n\n

Moving from DevOps to MLOps<\/strong><\/p>\n\n\n\n

Data science and machine learning model development need not drown in DevOps. Over the last several years, a wide range of tools have been introduced by everyone from startups to new divisions within large technology companies that are built specifically to address these core challenges. (Disclaimer: I work as a Data Scientist at one of these companies, Comet<\/a>).These products and technologies have introduced to the market the nascent notions of \u2018ML best practices\u2019 and \u2018ML tooling.\u2019 Adoption of the right tooling can be the difference between driving a successful machine learning organization and trudging through a broken DevOps quagmire in perpetuity.<\/p>\n\n\n\n

In remainder of this post, I will highlight three broad decisions that AI leaders can make to enable their teams to focus on what they do best. I will focus on ways to address the second pain point mentioned above: experimentation and research, ignoring the very real challenges around data and productionization. At Comet we work with some of the biggest Fortune 100 companies in the world<\/a>, and I can say from direct experience that these tips can immediately speed up research cycles, enable your team to build better models, and drive real business value from your investments in AI.<\/p>\n\n\n\n

Invest in a Digital Laboratory for your Research Team<\/strong><\/p>\n\n\n\n

Biologists, chemists and many other scientists benefit from more than just laboratories to do their best research. Rooms full of beakers, computers, and lab rats alone don’t allow them to record their work, visualize results and iterate quickly onto the next avenue of research. Their labs are also equipped with recording and documentation tools.  In order to provide a digital laboratory for the data scientist, the paradigm of work is shifting from the modularity and sub-tasking of software engineering to logging, visualization and iteration. New tools, both open source and paid, treat every script executed by an engineer or data scientist as an \u2018Experiment,\u2019 with artifacts, metadata, dataset samples, visualizations, models, hyperparameters and more, logged and stored for future comparison. Data Science teams need to know (and be able to recall) the history of where they\u2019ve been to decide where to go next.<\/p>\n\n\n\n

Standardize the Experiment of the Enterprise<\/strong><\/p>\n\n\n\n

Building an expensive laboratory won\u2019t help much if your researchers aren\u2019t versed in how to structure, run, and keep track of experiments. Data Science leaders must ensure their teams can manage a standard set of artifacts, metrics, datasets, hyperparameters, compute sources, and much more that are recorded for each of their experiments. This has beneficial outcomes in two senses. First, the inculcation of these best practices will drive home the need for your data scientists to be focused on much more than their target metric, unlocking holistic research and creative approaches to modeling. Second, a standard experiment format<\/a> enables comparative analysis of experiments over time, something good data scientists cannot afford to do without.<\/p>\n\n\n\n

Adopt an ML-friendly Workflow practice<\/strong><\/p>\n\n\n\n

Unlike software development, where products and features can be broken apart and modularized into subtasks with predictable delivery dates and action items, data scientists don\u2019t know exactly what they\u2019re looking for when they start building a model. Their workflow practices should reflect that. During a keynote address<\/a> at Amazon re:MARS in 2019, Andrew Ng shared that his team at Landing.ai does one-day sprints: write code and build models during the day, run experiments overnight, come back the next day to analyze results and build new models. Teams looking to minimize time to market<\/a> for their models should think about adopting workflow schedules that hew to the nature of a data scientist\u2019s work.<\/p>\n\n\n\n

Conclusion<\/strong> <\/strong><\/p>\n\n\n\n

The tips in this article taken together can be summarized succinctly: data science is not software engineering, so act and invest accordingly. Accepting the iterative, incremental and uncertain nature of your data science team\u2019s projects, and investing in tools to support them, will enable your company, research lab or startup to make good on the seemingly limitless promise of AI. Translating this into business value may be easier than you think.<\/p>\n\n\n\n

Learn More<\/strong><\/p>\n\n\n\n

To learn more about experimental machine learning, management and adopting a digital laboratory for your research team, check out this whitepaper<\/a> co-written by Comet and Dell EMC.<\/p>\n\n\n\n

Thanks to Phil Hummel for his review and edits to improve the blog.<\/p>\n\n\n\n

\n\n\n\n

Want to stay in the loop? Subscribe to the Comet Newsletter<\/a> for weekly insights and perspective on the latest ML news, projects, and more.<\/em><\/h2>\n","protected":false},"excerpt":{"rendered":"

This post was originally published as a sponsored post by Dell Technologies and Intel on CIO.com. We live in an age of rapid AI innovation and progress. Yet even as academics and researchers make astonishing advancements, demonstrating real business value and positive return on investment is challenging.  Developing cutting edge AI applications based on machine […]<\/p>\n","protected":false},"author":1,"featured_media":2233,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"customer_name":"","customer_description":"","customer_industry":"","customer_technologies":"","customer_logo":"","_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[12],"tags":[],"coauthors":[106],"class_list":["post-2232","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-thought-leadership"],"yoast_head":"\nInvesting in AI: Unlocking Profitable Machine Learning with Experiment Management - Comet<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.comet.com\/site\/blog\/investing-in-ai-unlocking-profitable-machine-learning-with-experiment-management\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Investing in AI: Unlocking Profitable Machine Learning with Experiment Management\" \/>\n<meta property=\"og:description\" content=\"This post was originally published as a sponsored post by Dell Technologies and Intel on CIO.com. We live in an age of rapid AI innovation and progress. Yet even as academics and researchers make astonishing advancements, demonstrating real business value and positive return on investment is challenging.  Developing cutting edge AI applications based on machine […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.comet.com\/site\/blog\/investing-in-ai-unlocking-profitable-machine-learning-with-experiment-management\/\" \/>\n<meta property=\"og:site_name\" content=\"Comet\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cometdotml\" \/>\n<meta property=\"article:published_time\" content=\"2020-07-31T21:57:53+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-04-24T17:30:42+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/06\/cio-blog-post.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"675\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Nikolas Laskaris\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@Cometml\" \/>\n<meta name=\"twitter:site\" content=\"@Cometml\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Nikolas Laskaris\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Investing in AI: Unlocking Profitable Machine Learning with Experiment Management - Comet","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.comet.com\/site\/blog\/investing-in-ai-unlocking-profitable-machine-learning-with-experiment-management\/","og_locale":"en_US","og_type":"article","og_title":"Investing in AI: Unlocking Profitable Machine Learning with Experiment Management","og_description":"This post was originally published as a sponsored post by Dell Technologies and Intel on CIO.com. We live in an age of rapid AI innovation and progress. Yet even as academics and researchers make astonishing advancements, demonstrating real business value and positive return on investment is challenging.  Developing cutting edge AI applications based on machine […]","og_url":"https:\/\/www.comet.com\/site\/blog\/investing-in-ai-unlocking-profitable-machine-learning-with-experiment-management\/","og_site_name":"Comet","article_publisher":"https:\/\/www.facebook.com\/cometdotml","article_published_time":"2020-07-31T21:57:53+00:00","article_modified_time":"2025-04-24T17:30:42+00:00","og_image":[{"width":1200,"height":675,"url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/06\/cio-blog-post.jpg","type":"image\/jpeg"}],"author":"Nikolas Laskaris","twitter_card":"summary_large_image","twitter_creator":"@Cometml","twitter_site":"@Cometml","twitter_misc":{"Written by":"Nikolas Laskaris","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.comet.com\/site\/blog\/investing-in-ai-unlocking-profitable-machine-learning-with-experiment-management\/#article","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/blog\/investing-in-ai-unlocking-profitable-machine-learning-with-experiment-management\/"},"author":{"name":"engineering@atre.net","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/550ac35e8e821db8064c5bd1f0a04e6b"},"headline":"Investing in AI: Unlocking Profitable Machine Learning with Experiment Management","datePublished":"2020-07-31T21:57:53+00:00","dateModified":"2025-04-24T17:30:42+00:00","mainEntityOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/investing-in-ai-unlocking-profitable-machine-learning-with-experiment-management\/"},"wordCount":1029,"publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/investing-in-ai-unlocking-profitable-machine-learning-with-experiment-management\/#primaryimage"},"thumbnailUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/06\/cio-blog-post.jpg","articleSection":["Thought Leadership"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.comet.com\/site\/blog\/investing-in-ai-unlocking-profitable-machine-learning-with-experiment-management\/","url":"https:\/\/www.comet.com\/site\/blog\/investing-in-ai-unlocking-profitable-machine-learning-with-experiment-management\/","name":"Investing in AI: Unlocking Profitable Machine Learning with Experiment Management - Comet","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/investing-in-ai-unlocking-profitable-machine-learning-with-experiment-management\/#primaryimage"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/investing-in-ai-unlocking-profitable-machine-learning-with-experiment-management\/#primaryimage"},"thumbnailUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/06\/cio-blog-post.jpg","datePublished":"2020-07-31T21:57:53+00:00","dateModified":"2025-04-24T17:30:42+00:00","breadcrumb":{"@id":"https:\/\/www.comet.com\/site\/blog\/investing-in-ai-unlocking-profitable-machine-learning-with-experiment-management\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.comet.com\/site\/blog\/investing-in-ai-unlocking-profitable-machine-learning-with-experiment-management\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/blog\/investing-in-ai-unlocking-profitable-machine-learning-with-experiment-management\/#primaryimage","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/06\/cio-blog-post.jpg","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/06\/cio-blog-post.jpg","width":1200,"height":675,"caption":"blue lines"},{"@type":"BreadcrumbList","@id":"https:\/\/www.comet.com\/site\/blog\/investing-in-ai-unlocking-profitable-machine-learning-with-experiment-management\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.comet.com\/site\/"},{"@type":"ListItem","position":2,"name":"Investing in AI: Unlocking Profitable Machine Learning with Experiment Management"}]},{"@type":"WebSite","@id":"https:\/\/www.comet.com\/site\/#website","url":"https:\/\/www.comet.com\/site\/","name":"Comet","description":"Build Better Models Faster","publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.comet.com\/site\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.comet.com\/site\/#organization","name":"Comet ML, Inc.","alternateName":"Comet","url":"https:\/\/www.comet.com\/site\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","width":310,"height":310,"caption":"Comet ML, Inc."},"image":{"@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/cometdotml","https:\/\/x.com\/Cometml","https:\/\/www.youtube.com\/channel\/UCmN63HKvfXSCS-UwVwmK8Hw"]},{"@type":"Person","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/550ac35e8e821db8064c5bd1f0a04e6b","name":"engineering@atre.net","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/image\/027c18177377edf459980f0cfb83706c","url":"https:\/\/secure.gravatar.com\/avatar\/d002a459a297e0d1779329318029aee19868c312b3e1f3c9ec9b3e3add2740de?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/d002a459a297e0d1779329318029aee19868c312b3e1f3c9ec9b3e3add2740de?s=96&d=mm&r=g","caption":"engineering@atre.net"},"sameAs":["https:\/\/live-cometml.pantheonsite.io"],"url":"https:\/\/www.comet.com\/site\/blog\/author\/engineeringatre-net\/"}]}},"jetpack_featured_media_url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/06\/cio-blog-post.jpg","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/2232","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/comments?post=2232"}],"version-history":[{"count":1,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/2232\/revisions"}],"predecessor-version":[{"id":15696,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/2232\/revisions\/15696"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media\/2233"}],"wp:attachment":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media?parent=2232"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/categories?post=2232"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/tags?post=2232"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/coauthors?post=2232"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}