{"id":9094,"date":"2024-02-03T11:29:47","date_gmt":"2024-02-03T19:29:47","guid":{"rendered":"https:\/\/live-cometml.pantheonsite.io\/?p=9094"},"modified":"2025-04-24T17:03:21","modified_gmt":"2025-04-24T17:03:21","slug":"conversational-ai-with-langchain-and-comet","status":"publish","type":"post","link":"https:\/\/www.comet.com\/site\/blog\/conversational-ai-with-langchain-and-comet\/","title":{"rendered":"Conversational AI with LangChain and\u00a0Comet"},"content":{"rendered":"\n

<\/h3>\n\n\n\n
\"\"\/<\/figure>\n\n\n\n

An Introduction to Conversational AI<\/h3>\n\n\n\n

AI chatbots and other conversational AI offer 24\/7 availability support, minimize errors, save costs, boost sales, and engage customers effectively. Businesses are drawn to chatbots not only for the aforementioned reasons but also due to their user-friendly creation process.<\/p>\n\n\n\n

As per Gartner<\/a>, the primary source of customer service will be AI chatbots for 25% of organizations by 2027. Additionally, Zendesk<\/a> reports that 72% of business leaders prioritize expanding AI and chatbots across the customer experience within 2024.<\/p>\n\n\n\n

Creating a chatbot is now more accessible with many development platforms available. Among many options, LangChain stands out as a powerful library for creating chatbots because it is flexible, scalable, and open-source.<\/p>\n\n\n\n

This article lightly touches on the history and components of chatbots. Later, it explores the s the implementation of chatbots using LangChain and the experiment tracking using Comet<\/a>.<\/p>\n\n\n\n

Chatbots\u200a\u2014\u200aThen and Now<\/h3>\n\n\n\n

Early chatbots could not grasp the context and nuances of language. One of the first chatbots, ELIZA<\/a>, was developed in 1966 to mimic human interaction and had built-in scripts to create the illusion of intelligence.<\/p>\n\n\n\n

Later, chatbots relied on rule-based systems and simpler machine learning approaches. These chatbots had limited capabilities and responded to specific keywords and phrases. This evolution paved the way for the development of conversational AI.<\/p>\n\n\n\n

The recent rise of Large Language Models (LLMs) has been a game changer for the ChatBot industry. These models are trained on extensive data and have been the driving force behind conversational tools like BARD<\/a> and ChatGPT<\/a>. These LLM-based bots have found various applications in various industries and have become the go-to information source for many people.<\/p>\n\n\n\n

However, chatbots are not only restricted to such commercial platforms, and developers can create their custom models using frameworks like LangChain.<\/p>\n\n\n\n

What is LangChain?<\/h3>\n\n\n\n

LangChain is an open-source framework, available in Python and Javascript libraries, that enables users to build applications using LLMs. Fundamentally, LangChain operates as an LLM-centered framework, capable of building chatbot applications, Visual Question-Answering (VQA), summarization, and much more.<\/p>\n\n\n\n

Think of LangChain as a toolbox for Python and Javascript that provides ready-to-use building blocks for working with language models. These building blocks, similar to functions and object classes, are essential components for creating generative AI programs. These components connect seamlessly, allowing you to build applications with less code and without requiring complicated natural language processing (NLP) tasks.<\/p>\n\n\n\n

A distinctive feature of LangChain is its innovative Agents. They operate on the fundamental concept of utilizing a language model to decide on a sequence of actions. Agents leverage a language model as a reasoning engine to determine the actions and their order dynamically.<\/p>\n\n\n\n

Components of Conversational AI<\/h3>\n\n\n\n

The LangChain framework includes various modules, each of which adds to the chatbot\u2019s functionality. This section will explore each component in detail and discuss how it facilitates the creation of a functional chatbot.<\/p>\n\n\n\n

Chat Model<\/h4>\n\n\n\n

Chat models are a category of LLMs specifically trained to output human-like, conversational responses. LLMs and chat models are slightly different as LLMs are pure text completion models, and chat models are fine-tuned for having conversations.<\/p>\n\n\n\n

Chat models utilize a distinct input interface compared to LLMs, accepting a list of chat messages labeled as System, AI, and Human. The AI chat message is then provided as the output from the chat model. LangChain provides a diverse range of chat models<\/a> and also accommodates custom chat models.<\/p>\n\n\n\n

Prompt Template<\/h4>\n\n\n\n

Prompt acts as a set of instructions or input to the language model by a user to guide model response. The purpose of prompting is to help chat models understand the context and generate relevant outputs.<\/p>\n\n\n\n

Prompt templates<\/a> serve as predefined formulas for crafting and reusing prompts across multiple language models. Language models commonly anticipate prompts as a string or a list of conversational messages.<\/p>\n\n\n\n

Since the prompts are a list of messages, we can use `ChatPromptTemplate` for it. Each chat message is associated with content and role, e.g., Human, AI, or a system role.<\/p>\n\n\n\n

Chains<\/h4>\n\n\n\n

A stand-alone LLM is great for basic tasks, but intricate applications require a complex architecture that chains multiple components with the model. LangChain-supported chains include LLMChain<\/a>, RouterChain<\/a>, and SequentialChain<\/a>. The most basic chain is the LLMChain, which combines the LLM, prompt, and optionally an output parser.<\/p>\n\n\n\n

Memory<\/h4>\n\n\n\n

A key component of natural conversation is the ability to reference prior information, which must be stored in memory for later use. In LangChain, the `memory` component solves the problem by simply keeping track of previous conversations.<\/p>\n\n\n\n

Every chain in LangChain defines some core execution that expects certain inputs. Some inputs originate directly from the user, others are retrieved from the memory. When chains are executed, they interact with the memory system twice. First, when it receives the input from the user, it will read from the memory and augment the user inputs.<\/p>\n\n\n\n

Additionally, prior to delivering the output, the chain will record the inputs and outputs of the ongoing execution in the memory.<\/p>\n\n\n\n

Documents<\/h4>\n\n\n\n

Chatbots are most useful to businesses when custom-trained for a specific domain. LangChain loads training data in the form of \u2018Documents\u2019, and these contain text from various sources and their respective metadata. The `Document` class provides various document loader methods to load data from sources, including PDF, HTML, JSON, and CSV.<\/p>\n\n\n\n

Retriever<\/h4>\n\n\n\n

It\u2019s essential for chat models to extract only the information from relevant documents given a query for accurate responses. Retrievers do this job of returning documents by providing a query as input and returning a list of documents.<\/p>\n\n\n\n

Building a Conversation AI With LangChain and Comet<\/h3>\n\n\n\n

The following section will guide you through the explanation and implementation of a retail chatbot. The chatbot will provide users with information about retail products, detailing specifications, prices, and reviews.<\/p>\n\n\n\n

Setup<\/h4>\n\n\n\n

The following chatbot uses OpenAI\u2019s chat model GPT-3.5-turbo <\/strong>and, therefore, includes OpenAI\u2019s API key. Users can get the API key by creating a free account here<\/a>. Additionally, the application will be monitored using the Comet<\/a> platform to keep track of all inputs and outputs. First, we will need to create a Comet account to access its API key.<\/p>\n\n\n\n

Afterwards, we need to install the following libraries using the PIP package manager. Run the following command:<\/p>\n\n\n\n

!pip install langchain openai comet_ml comet-llm textstat tiktoken faiss-gpu```<\/span><\/pre>\n\n\n\n
Chat Models<\/h4>\n\n\n\n
As discussed above, chat models are backed by LLMs such as GPT or Anthropic and are essential for conversational AI. They specialize in having a conversation rather than text completion. For this chatbot, we will be using GPT-3.5. There are many chat models supported in LangChain that can be found here<\/a>.<\/p>\n\n\n\n
Here\u2019s a snippet to initialize a chat model:<\/p>\n\n\n\n
from<\/span> langchain.chat_models import<\/span> ChatOpenAI\n\nllm = ChatOpenAI(temperature = 0.4<\/span>, openai_api_key=\"...\"<\/span>)<\/span><\/pre>\n\n\n\n
Here, the temperature parameters adjust the creativity of the response as it increases with a higher temperature.<\/p>\n\n\n\n
Prompt<\/h4>\n\n\n\n
Prompts guide chatbots in responding to queries. LangChain provides prompt templates that simplify the creation of prompts by combining default messages, user input, and chat history.<\/p>\n\n\n\n
The following is the prompt for the chatbot:<\/p>\n\n\n\n
from<\/span> langchain.prompts import<\/span> (\n   ChatPromptTemplate,\n   HumanMessagePromptTemplate,\n   MessagesPlaceholder,\n   SystemMessagePromptTemplate,\n)\n\n\ninstructions = \"\"\"You are a friendly chatbot capable of answering questions related to products. Users can ask questions about its specifications, prices, and reviews. Be polite and redirect conversation specifically to product information when necessary.\"\"\"<\/span>\n\nhuman = \"\"\" Chat history of the user: {chat_history} New human question: {input} \"\"\"<\/span>\n\nprompt = ChatPromptTemplate(messages=[\n\nSystemMessagePromptTemplate.from_template(instructions), # Guidance<\/span>\n\nHumanMessagePromptTemplate.from_template(human), #User query], <\/span>\n\ninput_variables=['chat_history'<\/span>,'input'<\/span>] # variables)<\/span><\/span><\/pre>\n\n\n\n
Memory<\/h4>\n\n\n\n
Memory is important for conversational AI, as it allows the conversation to be stored and adds it to the current query for reference of the previous conversation. There are many memories supported by LangChain, as discussed above.<\/p>\n\n\n\n
Now, we will use ConversationBufferWindowMemory to track the last four interactions. Here\u2019s how to create it:<\/p>\n\n\n\n
from<\/span> langchain.memory import<\/span> ConversationBufferWindowMemory\n\nmemory = ConversationBufferWindowMemory(memory_key = \"chat_history\"<\/span>,k = 4<\/span>)<\/span><\/pre>\n\n\n\n
The parameter `k` determines how many previous conversations need to go to the chat model.<\/p>\n\n\n\n
Chains<\/h4>\n\n\n\n
Chains are a simple concept of connecting different pieces, like language models, prompts, memory buffers, etc, into a single chain.<\/p>\n\n\n\n
Here\u2019s how the previous components can be connected to make the chatbot:<\/p>\n\n\n\n
from<\/span> langchain.chains import<\/span> ConversationChain\n\n\nconversation = ConversationChain(llm = llm,\n                                 prompt = prompt,\n                                 memory = memory,\n                                 verbose = False<\/span>) # LLMChain conversation<\/span>\n\nconversation.run(\"Hi, my name is Bob.\"<\/span>)\n#### AI: Hey Bob, how can I help you?<\/span>\nconversation.run(\"What is my name?\"<\/span>)\n#### AI: Your name is Bob. Is there anything else I can help you with?<\/span><\/span><\/pre>\n\n\n\n
As we can see, the chatbot is using the previous conversation. `ConversationChain()` takes in multiple parameters such as LLM, prompt and memory to respond to the queries of users, but this chatbot does not have the information of the products.<\/p>\n\n\n\n
Documents<\/h4>\n\n\n\n
Our chatbot requires documents for lookup and answering product-related questions which include reviews, product specifications, and prices.<\/p>\n\n\n\n
So, let\u2019s download the dataset from the data world<\/a>.<\/p>\n\n\n\n
import<\/span> pandas as<\/span> pd\n\ndf = pd.read_csv(\n'https:\/\/query.data.world\/s\/76kyenosebb7rtcruaarjgnfogum66?dws=00000'<\/span>)\n\ndf.to_csv('product.csv'<\/span>,\n          sep=','<\/span>,\n          index=False<\/span>,\n          encoding='utf-8'<\/span>) #Saving as a csv<\/span><\/span><\/pre>\n\n\n\n