In the past year Generative AI and Large Language Models(LLMs) have disrupted the Education Landscape. We are in the paradigm where AI not only helps with immediate learning needs of students but also plan and design a study guide that can be personalized based on individual needs. Conversational AI agents are perfect for solving this problem. In this presentation I am going to speak about building a conversational learning experience at Chegg. Chegg is a centralized hub where students come to get help with writing, science, math, and other educational needs. In order to impact a student’s learning capabilities we are enabling a personalized chat experience powered by an LLM agent. As a user experience this comes in the form of students being able to not just ask questions, but get clarifications on the solution or independent concepts. It’s similar to a tutor helping out with different learning needs. Behind the scenes this experience is powered an LLM agent. An agent is an intelligent system with cognitive abilities to take decisions and make appropriate choices. The first job of the agent is to understand what a student needs when they come to Chegg, they might be wanting quick help when they are stuck on a problem or they could want a study guide for the semester. Once the need of the student is clear the agent has over a decade years of content and student interaction data that it can use and form plans for the conversational learning session. Next the plans are executed by plugging into API such as search and calling traditional Machine Learning models and LLMs from external API and ones fine-tuned on Chegg data. There are several moving parts to building a system that can robustly provide high quality content and scale to millions of students. This requires a robust engineering infrastructure, the agility to adapt to a constantly changing world of LLMs powering the experience and develop a system to monitor and evaluate performance of the conversational AI system. The unique feature of LLMs is they might not give the same result for the same prompt every time, this might cause unexpected behavior when the system is used by millions of students. Building scalable applications with streaming functionality has its own challenges. Fast iterations are extremely important to keep up with the pace of innovation, at the same time creating best practices to ensure accountability and reproducibility is important for experimentation and to create an optimal customer experience. Some of these include prompt versioning, model versioning and monitoring models as they go into production. Another important factor to consider while building an LLM or Generative AI assisted application, does it make sense to build smaller ML models that do classifications, summarizations, NER to reduce the ask from Generative models such that it can scale to larger traffic at lower latency and cost. Is the tradeoff higher development cycles or is it possible to build these models faster using LLM assisted training data? I will address how to answer these questions that come up in the lifecycle of a Generative AI driven application.