{"id":8518,"date":"2024-01-04T08:00:29","date_gmt":"2024-01-04T16:00:29","guid":{"rendered":"https:\/\/live-cometml.pantheonsite.io\/?p=8518"},"modified":"2025-04-24T17:03:37","modified_gmt":"2025-04-24T17:03:37","slug":"introduction-to-langchain-for-including-ai-from-large-language-models-llms-inside-data-applications-and-data-pipelines","status":"publish","type":"post","link":"https:\/\/www.comet.com\/site\/blog\/introduction-to-langchain-for-including-ai-from-large-language-models-llms-inside-data-applications-and-data-pipelines\/","title":{"rendered":"Introduction to LangChain for Including AI from Large Language Models (LLMs) Inside Data Applications and Data Pipelines"},"content":{"rendered":"\n<figure class=\"wp-block-image mb mc md me mf mg ly lz paragraph-image\"><img decoding=\"async\" src=\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*rTF2OUOIqVny3fSB\" alt=\"language logo and blog title &quot;Introduction to LangChain for Including AI from Large Language Models (LLMs) Inside Data Applications and Data Pipelines&quot;\"\/><figcaption class=\"wp-element-caption\">Image by Author<\/figcaption><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp mu mv mw gs mx my mz na nb nc nd ne nf ng nh ni nj nk nl nm fk bj\" id=\"d035\">Large Language Models (LLMs) entered the spotlight with the release of OpenAI\u2019s GPT-3 in 2020. We have seen exploding interest in LLMs and in a broader discipline, Generative AI. There have been great leaps in the LLMs space, from the introduction of Google\u2019s <em class=\"nn\">\u201csentient\u201d<\/em> LaMDA chatbot, BLOOM, the first high-performance and <em class=\"nn\">open-source<\/em> LLM, and the release of next-generation GPT-3.5 models by OpenAI. However, after all these milestones, the release of ChatGPT thrust the LLMs into the spotlight. <a class=\"af no\" href=\"https:\/\/docs.langchain.com\/docs\/\" target=\"_blank\" rel=\"noopener ugc nofollow\"><strong class=\"mt fs\">LangChain<\/strong><\/a> appeared around the same time as the <a class=\"af no\" href=\"https:\/\/openai.com\/chatgpt\" target=\"_blank\" rel=\"noopener ugc nofollow\">ChatGPT<\/a> release and was already packed with excellent tools for building tools based on LLMs. This article will provide an overview of the LangChain library, what it can do, the problems it solves, and its use cases.<\/p>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp mu mv mw gs mx my mz na nb nc nd ne nf ng nh ni nj nk nl nm fk bj\" id=\"efdd\">In the following article, we will look at how we can build applications while leveraging various concepts of LangChain.<\/p>\n\n\n\n<h1 class=\"wp-block-heading np nq fr be nr ns nt gr nu nv nw gu nx ny nz oa ob oc od oe of og oh oi oj ok bj\" id=\"6b4c\">Introduction to LangChain<\/h1>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp ol mv mw gs om my mz na on nc nd ne oo ng nh ni op nk nl nm fk bj\" id=\"32ce\">Langchain is an open-source framework that provides tools, components, and interfaces that simplify the development of LLM-powered applications. It has a collection of APIs that developers can embed in their applications, empowering them to infuse language processing capabilities without building everything from the ground up. LangChain efficiently simplifies the process of crafting LLM-based applications.<\/p>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp mu mv mw gs mx my mz na nb nc nd ne nf ng nh ni nj nk nl nm fk bj\" id=\"e523\">Applications like chatbots, Generative-Question-Answering (GQA), summarization, virtual assistants, and language translation utilities represent LLM-powered applications. Developers leverage LangChain to create tailored language model-based applications that cater to specific needs.<\/p>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp mu mv mw gs mx my mz na nb nc nd ne nf ng nh ni nj nk nl nm fk bj\" id=\"aebf\">Its core idea is that we can <em class=\"nn\">\u201cchain\u201d<\/em> together different components to create more refined LLM apps. Features include:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong class=\"mt fs\">Models<\/strong>: LLMs, chat models, and text embedding models.<\/li>\n\n\n\n<li><strong class=\"mt fs\">Prompt<\/strong>s: Inputs to a model.<\/li>\n\n\n\n<li><strong class=\"mt fs\">Memory<\/strong>: Storing and retrieving data while conversing.<\/li>\n\n\n\n<li><strong class=\"mt fs\">Indexes<\/strong>: An interface for querying large datasets, enabling LLMs to interact with different document types for retrieval purposes.<\/li>\n\n\n\n<li><strong class=\"mt fs\">Chain<\/strong>s: Allows developers to combine components for advanced use cases and integration with other tools.<\/li>\n\n\n\n<li><strong class=\"mt fs\">Agents<\/strong>: Involves LLM decision-making, action execution, observation, and repetition until completion.<\/li>\n\n\n\n<li><strong class=\"mt fs\">Callbacks<\/strong>: Introduce examination and introspection as an additional layer of support.<\/li>\n<\/ul>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp mu mv mw gs mx my mz na nb nc nd ne nf ng nh ni nj nk nl nm fk bj\" id=\"ceac\">The following article explains each of these in more detail with code.<\/p>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp mu mv mw gs mx my mz na nb nc nd ne nf ng nh ni nj nk nl nm fk bj\" id=\"eeea\">LangChain supports using <strong class=\"mt fs\">two programming languages<\/strong>:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong class=\"mt fs\">JavaScrip<\/strong>t: Great for embedding AI into web applications with the node package.<\/li>\n\n\n\n<li><strong class=\"mt fs\">Python<\/strong>: Great for including AI in Python-based software or data pipelines.<\/li>\n<\/ul>\n\n\n\n<h1 class=\"wp-block-heading np nq fr be nr ns nt gr nu nv nw gu nx ny nz oa ob oc od oe of og oh oi oj ok bj\" id=\"c1a9\">What Does LangChain Address?<\/h1>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp ol mv mw gs om my mz na on nc nd ne oo ng nh ni op nk nl nm fk bj\" id=\"5993\">LangChain implements two major workflows for interacting with LLMs: chatting and embedding. These workflows have problems that LangChain aims to address. Let\u2019s discuss some of them:<\/p>\n\n\n\n<h2 class=\"wp-block-heading oy nq fr be nr oz pa pb nu pc pd pe nx na pf pg ph ne pi pj pk ni pl pm pn po bj\" id=\"eb7a\">Memory Constraints On LLMs<\/h2>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp ol mv mw gs om my mz na on nc nd ne oo ng nh ni op nk nl nm fk bj\" id=\"0879\">LLMs generate responses based on previous conversations. However, these models have relatively short memory. For instance, GPT-4 has a memory constraint of 8000 tokens. That means that if the conversation exceeds the memory limit, the responses become inconsistent since it might lose track of the beginning of the conversation.<\/p>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp mu mv mw gs mx my mz na nb nc nd ne nf ng nh ni nj nk nl nm fk bj\" id=\"e1de\">You want your applications, for instance, a chatbot, to remember the entire conversation with a customer. LangChain makes this possible by providing a <a class=\"af no\" href=\"https:\/\/python.langchain.com\/docs\/modules\/memory\/\" target=\"_blank\" rel=\"noopener ugc nofollow\"><strong class=\"mt fs\">chat memory tool<\/strong><\/a> that enables LLMs to reflect on previous interactions.<\/p>\n\n\n\n<h2 class=\"wp-block-heading oy nq fr be nr oz pa pb nu pc pd pe nx na pf pg ph ne pi pj pk ni pl pm pn po bj\" id=\"1569\">Structured Response Formats<\/h2>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp ol mv mw gs om my mz na on nc nd ne oo ng nh ni op nk nl nm fk bj\" id=\"6d27\">When provided with a prompt, you may require a model to consistently generate a response in a specific format (e.g., CSV, JSON, datatime format, etc.) other than text. To handle the output requirement formats, LangChain provides <a class=\"af no\" href=\"https:\/\/python.langchain.com\/docs\/modules\/model_io\/output_parsers\/\" target=\"_blank\" rel=\"noopener ugc nofollow\"><strong class=\"mt fs\">output parser<\/strong><\/a> classes that are responsible for that.<\/p>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp mu mv mw gs mx my mz na nb nc nd ne nf ng nh ni nj nk nl nm fk bj\" id=\"3d57\">You will provide the parser with instructions on how the model\u2019s output should be formatted. When presented with the model\u2019s response, the parser will use these instructions to parse the response onto the specified structure.<\/p>\n\n\n\n<h2 class=\"wp-block-heading oy nq fr be nr oz pa pb nu pc pd pe nx na pf pg ph ne pi pj pk ni pl pm pn po bj\" id=\"dae1\">Prompting<\/h2>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp ol mv mw gs om my mz na on nc nd ne oo ng nh ni op nk nl nm fk bj\" id=\"6938\">LLMs are trained on a simple concept \u2014 you input a text sequence, and the model outputs a sequence of text. In this case, the one crucial variable is the input sequence \u2014 the prompt.<\/p>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp mu mv mw gs mx my mz na nb nc nd ne nf ng nh ni nj nk nl nm fk bj\" id=\"30f8\">With LLMs, prompts are vital. Bad prompts will generate poor responses, while good ones will generate exceptional ones; thus, constructing good ones can be problematic when working with LLMs. While prompting may involve defining the task to complete, it also includes determining the AI\u2019s personality and writing style and including instructions to encourage factual precision.<\/p>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp mu mv mw gs mx my mz na nb nc nd ne nf ng nh ni nj nk nl nm fk bj\" id=\"dca7\">LangChain recognizes the power of prompts and has built great classes just for that. A good prompt may consist of the following components:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong class=\"mt fs\">Instructions<\/strong>: Instruct the model on what to do and how to structure the response.<\/li>\n\n\n\n<li><strong class=\"mt fs\">Context<\/strong>: Additional source of knowledge for the model, which can be manually given, pulled from APIs, retrieved from vector stores, or pulled from other sources.<\/li>\n\n\n\n<li><strong class=\"mt fs\">Query<\/strong>: A user input to the system by a human user.<\/li>\n<\/ul>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp mu mv mw gs mx my mz na nb nc nd ne nf ng nh ni nj nk nl nm fk bj\" id=\"98ae\">For instance, take the following prompt:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><span id=\"9a34\" class=\"pt nq fr pq b bf pu pv l pw px\" data-selectable-paragraph=\"\">prompt = <span class=\"hljs-string\">\"\"\" Answer the question based on the context below. If the information provided cannot answer the question, respond with \"Can't find an answer.\"\nContext: Langchain is an open-source framework that provides tools, components, and interfaces that simplify the development of LLM-powered applications. Its key features include Prompts, Memory, Indexes, Chains, Agents, and callbacks. LangChain is provided in two programming languages: Python and JavaScript.\nQuestion: What are the main features of LangChain, and what programming languages can I implement these components with?\nAnswer:\n\"\"\"<\/span><\/span><\/pre>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp mu mv mw gs mx my mz na nb nc nd ne nf ng nh ni nj nk nl nm fk bj\" id=\"c587\">Well, it\u2019s unlikely that we will hardcode the context and the question in most cases. We would feed them through a <em class=\"nn\">template<\/em>. LangChain makes this possible by providing <a class=\"af no\" href=\"https:\/\/python.langchain.com\/docs\/modules\/model_io\/prompts\/prompt_templates\/\" target=\"_blank\" rel=\"noopener ugc nofollow\"><strong class=\"mt fs\">prompt template classes<\/strong><\/a>. These classes are developed to make prompting with dynamic inputs easier. One such template class is the <code class=\"cw py pz qa pq b\"><a class=\"af no\" href=\"https:\/\/python.langchain.com\/docs\/modules\/model_io\/prompts\/prompt_templates\/#prompttemplate\" target=\"_blank\" rel=\"noopener ugc nofollow\"><strong class=\"mt fs\">PromptTemplate<\/strong><\/a><\/code> class.<\/p>\n\n\n\n<h2 class=\"wp-block-heading oy nq fr be nr oz pa pb nu pc pd pe nx na pf pg ph ne pi pj pk ni pl pm pn po bj\" id=\"4f69\">Switching Between LLM Models<\/h2>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp ol mv mw gs om my mz na on nc nd ne oo ng nh ni op nk nl nm fk bj\" id=\"0bbd\">There are other LLM providers available aside from OpenAI. Building software on one provider or API can lock the software only onto that ecosystem. In the future, you may realize that the software requires more excellent capabilities to enhance your product. Switching to a more capable model can sometimes be a hustle.<\/p>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp mu mv mw gs mx my mz na nb nc nd ne nf ng nh ni nj nk nl nm fk bj\" id=\"df3d\">However, LangChain provides the <a class=\"af no\" href=\"https:\/\/python.langchain.com\/docs\/modules\/model_io\/llms.html\" target=\"_blank\" rel=\"noopener ugc nofollow\">LLM class<\/a> that provides an interface for interacting with many LLMs. This class makes an abstraction easier to swap between LLMs or use multiple LLMs in your software.<\/p>\n\n\n\n<h2 class=\"wp-block-heading oy nq fr be nr oz pa pb nu pc pd pe nx na pf pg ph ne pi pj pk ni pl pm pn po bj\" id=\"0304\">Integration of LLMs Into Pipelines<\/h2>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp ol mv mw gs om my mz na on nc nd ne oo ng nh ni op nk nl nm fk bj\" id=\"3e1a\">LangChain implements <a class=\"af no\" href=\"https:\/\/python.langchain.com\/docs\/modules\/chains\/\" target=\"_blank\" rel=\"noopener ugc nofollow\">chains<\/a> and <a class=\"af no\" href=\"https:\/\/python.langchain.com\/docs\/modules\/agents\/\" target=\"_blank\" rel=\"noopener ugc nofollow\">agents<\/a> that provide pipeline-type workflows. For instance, we may extract data from sources like databases, which we then pass into an LLM and send a processed output to another system.<\/p>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp mu mv mw gs mx my mz na nb nc nd ne nf ng nh ni nj nk nl nm fk bj\" id=\"cbdb\"><strong class=\"mt fs\">Chains<\/strong> are objects that wrap multiple individual components together. A chain integrates an LLM with a prompt, forming modules that can execute operations on our text or other datasets. These chains are designed to process multiple inputs simultaneously. The most commonly used chain is the <a class=\"af no\" href=\"https:\/\/python.langchain.com\/docs\/modules\/chains\/foundational\/llm_chain\" target=\"_blank\" rel=\"noopener ugc nofollow\">LLM Chain<\/a>.<\/p>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp mu mv mw gs mx my mz na nb nc nd ne nf ng nh ni nj nk nl nm fk bj\" id=\"e5c9\"><strong class=\"mt fs\">Agents<\/strong> are more sophisticated, allowing business reasoning to let you decide how the components should interact. Some applications will require a predetermined chain of calls to LLMs\/other tools and potentially an unknown chain that depends on the user\u2019s input. An agent has access to a suite of tools such that, depending on the user input, the agent can then decide which, if any, of these tools to call.<\/p>\n\n\n\n<h2 class=\"wp-block-heading oy nq fr be nr oz pa pb nu pc pd pe nx na pf pg ph ne pi pj pk ni pl pm pn po bj\" id=\"999c\">Passing Data to LLMs<\/h2>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp ol mv mw gs om my mz na on nc nd ne oo ng nh ni op nk nl nm fk bj\" id=\"6446\">LLMs are text-based, so it is not always clear how to pass data into the model. First, you may need to store the data in a particular format, like an SQL table or DataFrame, which lets you control the portions of data sent to the LLM. LangChain implements indexes that provide the functionality to retrieve data in various formats so the LLM can best interact with them.<\/p>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp mu mv mw gs mx my mz na nb nc nd ne nf ng nh ni nj nk nl nm fk bj\" id=\"4c36\">Secondly, you can pass the data to a prompt through <strong class=\"mt fs\">Prompt Stuffing<\/strong><em class=\"nn\">, <\/em>the more straightforward way provided by LangChain\u2019s <a class=\"af no\" href=\"https:\/\/python.langchain.com\/docs\/modules\/data_connection\/indexing\" target=\"_blank\" rel=\"noopener ugc nofollow\">indexing<\/a> category<em class=\"nn\">. <\/em>However, passing the dataset to the prompt as context is simple and efficient but only applicable when you have a small dataset.<\/p>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp mu mv mw gs mx my mz na nb nc nd ne nf ng nh ni nj nk nl nm fk bj\" id=\"d8c6\">Index-related chains have three more techniques apart from Prompt stuffing:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong class=\"mt fs\">Map-reduce<\/strong>: Splits data into chunks, calls the LLM with an initial prompt on each chunk, then runs a different prompt to combine responses from the initial prompt.<\/li>\n\n\n\n<li><strong class=\"mt fs\">Refine<\/strong>: Runs an initial prompt on the first chunk of data, generating some output. Then, for each additional chunk of data, you use a prompt asking the LLM to refine the result based on the new dataset. It can be ideal when the output should converge on a specific output.<\/li>\n\n\n\n<li><strong class=\"mt fs\">Map-Rerank<\/strong>: Runs a prompt for each chunk of data and provides a confidence score for its response, which you can use to rank the outputs. That can be ideal for recommendation-like tasks.<\/li>\n<\/ul>\n\n\n\n<h1 class=\"wp-block-heading np nq fr be nr ns nt gr nu nv nw gu nx ny nz oa ob oc od oe of og oh oi oj ok bj\" id=\"99d7\">Use Cases of LangChain<\/h1>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp ol mv mw gs om my mz na on nc nd ne oo ng nh ni op nk nl nm fk bj\" id=\"4917\">LangChain offers a wide range of applications. It shines in some of the following use cases:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong class=\"mt fs\">API Integration<\/strong>: LangChain can integrate with various APIs, enabling the development of applications that interact with other platforms and services. For instance, you may use an API to retrieve exchange rate data or to interact with a cloud platform. LangChain\u2019s <strong class=\"mt fs\"><em class=\"nn\">chains<\/em><\/strong> and <strong class=\"mt fs\"><em class=\"nn\">agents<\/em><\/strong>features enable you to create data pipelines for this use case.<\/li>\n\n\n\n<li><strong class=\"mt fs\">Chatbot Development<\/strong>: LangChain provides the necessary tools to build chatbots. Chatbots are one of the most popular AI use cases. They need to have a communication style and recall previous interactions\/context during a conversation. LangChain <strong class=\"mt fs\"><em class=\"nn\">prompt templates<\/em><\/strong> can give you control over a chatbot\u2019s responses. Additionally, <strong class=\"mt fs\"><em class=\"nn\">message history<\/em><\/strong> tools ensure conversation consistency by providing the chatbot with adequate memory.<\/li>\n\n\n\n<li><strong class=\"mt fs\">Intelligent Question Answering<\/strong>: We can use LangChain to develop question-answering applications based on specific documents.<\/li>\n\n\n\n<li><strong class=\"mt fs\">Creating Interactive Agents<\/strong>: Developers can create agents that interact with users, make decisions based on their input, and continue their tasks until completion. These agents can be helpful in customer service, data collection, and other interactive applications.<\/li>\n\n\n\n<li><strong class=\"mt fs\">Data Summarization<\/strong>: LangChain can create applications that summarize long documents. This feature can help process lengthy texts, articles, or reports.<\/li>\n\n\n\n<li><strong class=\"mt fs\">Document Retrieval and Clustering:<\/strong> LangChain can simplify retrieval and clustering using embedding models. That can also be important when grouping similar documents or retrieving specific documents based on certain criteria.<\/li>\n<\/ul>\n\n\n\n<h1 class=\"wp-block-heading np nq fr be nr ns nt gr nu nv nw gu nx ny nz oa ob oc od oe of og oh oi oj ok bj\" id=\"8887\">What Are the Limitations of LangChain?<\/h1>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp ol mv mw gs om my mz na on nc nd ne oo ng nh ni op nk nl nm fk bj\" id=\"bdf9\">LangChain offers exceptional capabilities but has some limitations:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong class=\"mt fs\">Constraint to train only on text data<\/strong>: It does not support training on non-textual data. That may hinder its application in domains where data extends beyond text.<\/li>\n\n\n\n<li><strong class=\"mt fs\">Lacks support for multi-task learning<\/strong>: Simultaneously training language models on multiple related tasks improves performance. Without this technique, the efficiency and effectiveness of specific applications may not benefit from leveraging multi-task learning approaches.<\/li>\n\n\n\n<li><strong class=\"mt fs\">Extra steps to deploying LangChain-based models<\/strong>: LangChain does not provide straightforward deployment methods, which may cause problems when deploying such applications in production environments.<\/li>\n<\/ul>\n\n\n\n<h1 class=\"wp-block-heading np nq fr be nr ns nt gr nu nv nw gu nx ny nz oa ob oc od oe of og oh oi oj ok bj\" id=\"19ed\">Final Thoughts<\/h1>\n\n\n\n<p class=\"pw-post-body-paragraph mr ms fr mt b gp ol mv mw gs om my mz na on nc nd ne oo ng nh ni op nk nl nm fk bj\" id=\"24a4\">This article has introduced LangChain in a relatively basic manner. You have learned about LangChain and some of its use cases. Despite its limitations, LangChain is a robust framework that developers can use to harness the capabilities of language models. With its vast benefits and ongoing development, LangChain holds excellent assurance for the future of AI-powered solutions.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Large Language Models (LLMs) entered the spotlight with the release of OpenAI\u2019s GPT-3 in 2020. We have seen exploding interest in LLMs and in a broader discipline, Generative AI. There have been great leaps in the LLMs space, from the introduction of Google\u2019s \u201csentient\u201d LaMDA chatbot, BLOOM, the first high-performance and open-source LLM, and the [&hellip;]<\/p>\n","protected":false},"author":108,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"customer_name":"","customer_description":"","customer_industry":"","customer_technologies":"","customer_logo":"","footnotes":""},"categories":[65],"tags":[],"coauthors":[206],"class_list":["post-8518","post","type-post","status-publish","format-standard","hentry","category-llmops"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v25.9 (Yoast SEO v25.9) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>LangChain for Including AI from LLMs Inside Data Applications<\/title>\n<meta name=\"description\" content=\"Get an overview of the LangChain library, what it can do, the problems it solves, and its use cases in this blog article.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.comet.com\/site\/blog\/introduction-to-langchain-for-including-ai-from-large-language-models-llms-inside-data-applications-and-data-pipelines\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Introduction to LangChain for Including AI from Large Language Models (LLMs) Inside Data Applications and Data Pipelines\" \/>\n<meta property=\"og:description\" content=\"Get an overview of the LangChain library, what it can do, the problems it solves, and its use cases in this blog article.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.comet.com\/site\/blog\/introduction-to-langchain-for-including-ai-from-large-language-models-llms-inside-data-applications-and-data-pipelines\" \/>\n<meta property=\"og:site_name\" content=\"Comet\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cometdotml\" \/>\n<meta property=\"article:published_time\" content=\"2024-01-04T16:00:29+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-04-24T17:03:37+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*rTF2OUOIqVny3fSB\" \/>\n<meta name=\"author\" content=\"Brian Mutea\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@Cometml\" \/>\n<meta name=\"twitter:site\" content=\"@Cometml\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Brian Mutea\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"8 minutes\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"LangChain for Including AI from LLMs Inside Data Applications","description":"Get an overview of the LangChain library, what it can do, the problems it solves, and its use cases in this blog article.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.comet.com\/site\/blog\/introduction-to-langchain-for-including-ai-from-large-language-models-llms-inside-data-applications-and-data-pipelines","og_locale":"en_US","og_type":"article","og_title":"Introduction to LangChain for Including AI from Large Language Models (LLMs) Inside Data Applications and Data Pipelines","og_description":"Get an overview of the LangChain library, what it can do, the problems it solves, and its use cases in this blog article.","og_url":"https:\/\/www.comet.com\/site\/blog\/introduction-to-langchain-for-including-ai-from-large-language-models-llms-inside-data-applications-and-data-pipelines","og_site_name":"Comet","article_publisher":"https:\/\/www.facebook.com\/cometdotml","article_published_time":"2024-01-04T16:00:29+00:00","article_modified_time":"2025-04-24T17:03:37+00:00","og_image":[{"url":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*rTF2OUOIqVny3fSB","type":"","width":"","height":""}],"author":"Brian Mutea","twitter_card":"summary_large_image","twitter_creator":"@Cometml","twitter_site":"@Cometml","twitter_misc":{"Written by":"Brian Mutea","Est. reading time":"8 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.comet.com\/site\/blog\/introduction-to-langchain-for-including-ai-from-large-language-models-llms-inside-data-applications-and-data-pipelines#article","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/blog\/introduction-to-langchain-for-including-ai-from-large-language-models-llms-inside-data-applications-and-data-pipelines\/"},"author":{"name":"Brian Mutea","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/45acdda6535e03a9542e665f23953c3b"},"headline":"Introduction to LangChain for Including AI from Large Language Models (LLMs) Inside Data Applications and Data Pipelines","datePublished":"2024-01-04T16:00:29+00:00","dateModified":"2025-04-24T17:03:37+00:00","mainEntityOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/introduction-to-langchain-for-including-ai-from-large-language-models-llms-inside-data-applications-and-data-pipelines\/"},"wordCount":1677,"publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/introduction-to-langchain-for-including-ai-from-large-language-models-llms-inside-data-applications-and-data-pipelines#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*rTF2OUOIqVny3fSB","articleSection":["LLMOps"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.comet.com\/site\/blog\/introduction-to-langchain-for-including-ai-from-large-language-models-llms-inside-data-applications-and-data-pipelines\/","url":"https:\/\/www.comet.com\/site\/blog\/introduction-to-langchain-for-including-ai-from-large-language-models-llms-inside-data-applications-and-data-pipelines","name":"LangChain for Including AI from LLMs Inside Data Applications","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/introduction-to-langchain-for-including-ai-from-large-language-models-llms-inside-data-applications-and-data-pipelines#primaryimage"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/introduction-to-langchain-for-including-ai-from-large-language-models-llms-inside-data-applications-and-data-pipelines#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*rTF2OUOIqVny3fSB","datePublished":"2024-01-04T16:00:29+00:00","dateModified":"2025-04-24T17:03:37+00:00","description":"Get an overview of the LangChain library, what it can do, the problems it solves, and its use cases in this blog article.","breadcrumb":{"@id":"https:\/\/www.comet.com\/site\/blog\/introduction-to-langchain-for-including-ai-from-large-language-models-llms-inside-data-applications-and-data-pipelines#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.comet.com\/site\/blog\/introduction-to-langchain-for-including-ai-from-large-language-models-llms-inside-data-applications-and-data-pipelines"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/blog\/introduction-to-langchain-for-including-ai-from-large-language-models-llms-inside-data-applications-and-data-pipelines#primaryimage","url":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*rTF2OUOIqVny3fSB","contentUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*rTF2OUOIqVny3fSB"},{"@type":"BreadcrumbList","@id":"https:\/\/www.comet.com\/site\/blog\/introduction-to-langchain-for-including-ai-from-large-language-models-llms-inside-data-applications-and-data-pipelines#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.comet.com\/site\/"},{"@type":"ListItem","position":2,"name":"Introduction to LangChain for Including AI from Large Language Models (LLMs) Inside Data Applications and Data Pipelines"}]},{"@type":"WebSite","@id":"https:\/\/www.comet.com\/site\/#website","url":"https:\/\/www.comet.com\/site\/","name":"Comet","description":"Build Better Models Faster","publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.comet.com\/site\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.comet.com\/site\/#organization","name":"Comet ML, Inc.","alternateName":"Comet","url":"https:\/\/www.comet.com\/site\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","width":310,"height":310,"caption":"Comet ML, Inc."},"image":{"@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/cometdotml","https:\/\/x.com\/Cometml","https:\/\/www.youtube.com\/channel\/UCmN63HKvfXSCS-UwVwmK8Hw"]},{"@type":"Person","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/45acdda6535e03a9542e665f23953c3b","name":"Brian Mutea","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/image\/0008644e1041f4f2e48e3566c59bc055","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/11\/1652705747012-96x96.jpg","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/11\/1652705747012-96x96.jpg","caption":"Brian Mutea"},"url":"https:\/\/www.comet.com\/site\/blog\/author\/brianmuteakgmail-com\/"}]}},"_links":{"self":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/8518","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/users\/108"}],"replies":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/comments?post=8518"}],"version-history":[{"count":1,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/8518\/revisions"}],"predecessor-version":[{"id":15409,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/8518\/revisions\/15409"}],"wp:attachment":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media?parent=8518"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/categories?post=8518"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/tags?post=8518"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/coauthors?post=8518"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}