{"id":9410,"date":"2024-03-06T06:00:12","date_gmt":"2024-03-06T14:00:12","guid":{"rendered":"https:\/\/live-cometml.pantheonsite.io\/?p=9410"},"modified":"2025-04-24T17:03:01","modified_gmt":"2025-04-24T17:03:01","slug":"text-summarization-modeling-with-scikitllm-and-comet","status":"publish","type":"post","link":"https:\/\/www.comet.com\/site\/blog\/text-summarization-modeling-with-scikitllm-and-comet\/","title":{"rendered":"Text Summarization Modeling with ScikitLLM and\u00a0Comet"},"content":{"rendered":"\n
\"lego
Photo by rishi<\/a> on Unsplash<\/a><\/figcaption><\/figure>\n\n\n\n

 <\/p>\n\n\n\n

Large Language Models (LLMs) are pretty remarkable. These powerful machine-learning models can understand and generate human-like text, creating a natural conversation experience. But they\u2019re not just for chatting\u200a\u2014\u200athey can also be used to deliver information, generate information like text summarizations, and get valuable suggestions.<\/p>\n\n\n\n

Today, we will talk about ScikitLLM or Scikit Large Language Model. ScikitLLM is interesting because it seamlessly integrates LLMs into your traditional Scikit-learn (Sklearn) library. If you\u2019re familiar with machine learning and statistical modeling, you know Sklearn is a powerful tool that provides users with various unsupervised and supervised learning algorithms for building robust machine learning models.<\/p>\n\n\n\n

In this post, we\u2019ll take a deep dive into ScikitLLM and explore how you can use it to build text summarization ML models and monitor them all in Comet.<\/p>\n\n\n\n

What is ScikitLLM?<\/h3>\n\n\n\n

Scikit-LLM, described in the official Scikit-LLM GitHub repository<\/a>, is scikit-learn meets large language models. This means Scikit-LLM brings the power of powerful language models like ChatGPT into scikit-learn for enhanced text analysis tasks.<\/p>\n\n\n\n

One of the most fascinating aspects of this integration is that you can access various ML algorithms and leverage advanced natural language processing. Something else I found interesting was that this library maintains scikit-learn\u2019s workflow. This means the process is still basically the same: You import your libraries, load your dataset, split your data, train with the fit method, and make predictions using the predict method.<\/p>\n\n\n\n

Now, not to bore you with long talk! Let\u2019s get started!<\/p>\n\n\n\n

Getting Started with ScikitLLM<\/h3>\n\n\n\n

To make use of ScikitLLM, we will need to use the pip install command to install ScikitLLM:<\/p>\n\n\n\n

pip install scikit-llm<\/pre>\n\n\n\n
Currently, Scikit-LLM only supports OpenAI, GPT4ALL, Google PaLM 2, and Azure OpenAI.<\/p>\n\n\n\n
We will, however, make use of OpenAI. Thus, you will need to set up an OpenAI account<\/a>. Once done, set up billing<\/a>and generate an OpenAI API token key<\/a> for this project.<\/p>\n\n\n\n
Then head over to your Colab or Jupyter notebook and run this:<\/p>\n\n\n\n
# importing SKLLMConfig<\/span>\nfrom<\/span> skllm.config import<\/span> SKLLMConfig\n# Set your OpenAI API key<\/span>\nSKLLMConfig.set_openai_key(\"*******\"<\/span>)\n# Set your OpenAI organization<\/span>\nSKLLMConfig.set_openai_org(\"**ABC**\"<\/span>)<\/span><\/pre>\n\n\n\n
Note:<\/strong> ******* represents your API token key<\/a>, and **ABC** represents your organization I.D.<\/a><\/p>\n\n\n\n
Text Summarization with ScikitLLM<\/h3>\n\n\n\n
We will use the Starbucks reviews dataset<\/a> from Kaggle for the text summarization modeling. This dataset contains information about reviews, ratings, and location data from various Starbucks stores.<\/p>\n\n\n\n
To summarize the reviews, we will utilize the GPTSummarizer module of the ScikitLLM library and the GPT-3.5-turbo model from OpenAI. In the GPTSummarizer module, we can use the max_word<\/code> parameter to set a flexible limit on the number of words each summary produces. I said flexible because the actual length of the generated summary can be longer than the predetermined limit set. After that, we can use the fit_transform<\/code> method to feed our review to our model. Once done, we can use print<\/code> to see the summarized reviews.<\/p>\n\n\n\n
import<\/span> comet_ml\nimport<\/span> pandas as<\/span> pd\nfrom<\/span> comet_ml import<\/span> Artifact, Experiment\nfrom<\/span> skllm.preprocessing import<\/span> GPTSummarizer\n\n# Load your CSV file into a DataFrame<\/span>\ndf = pd.read_csv('\/content\/reviews_data.csv'<\/span>)\n\n# Select a subset of reviews (e.g., the first 50)<\/span>\nX = df[\"Review\"<\/span>].values[:50<\/span>]\n\n# Initialize the GPTSummarizer<\/span>\nreviews_summarizer = GPTSummarizer(openai_model=\"gpt-3.5-turbo\"<\/span>, max_words=10<\/span>)\n\n# Generate summaries with your model<\/span>\ngenerated_reviews = reviews_summarizer.fit_transform(X)<\/span><\/pre>\n\n\n\n
\"\"\/
The summarized review (Image by Author)<\/figcaption><\/figure>\n\n\n\n
 <\/p>\n\n\n\n
You should try running your model on the entire dataset as well. But the execution of this will take some time. To accomplish this, delete the .values[:50]<\/code> from the code we previously used.<\/p>\n\n\n\n
To evaluate how good our model is, we will then compute the BLEU Score. A sentence is compared to one or more reference sentences to determine the BLEU score. The output score ranges from 0 to 1. An output value of 1 indicates that the candidate sentence exactly matches one of the reference sentences. However, since we lacked a reference summary in our data, we are evaluating the generated summaries compared to themselves.<\/p>\n\n\n\n
# Join the generated reviews into a single string<\/span>\nreviews_text = \"\\n\"<\/span>.join(generated_reviews)\n\nfrom<\/span> nltk.translate.bleu_score import<\/span> corpus_bleu\n\nbleu_score = corpus_bleu([[summary.split()] for<\/span> summary in<\/span> generated_reviews], [summary.split() for<\/span> summary in<\/span> generated_reviews])<\/span><\/pre>\n\n\n\n
Running Our Model In Comet<\/h3>\n\n\n\n
Once done with the modeling, we can log everything in Comet.<\/p>\n\n\n\n
To accomplish that, we must first create a project in Comet. Here, we will record all relevant information, including metrics and created reviews, as true. However, if you don\u2019t already have a Comet account, you must create one<\/a>. You can find the name of your workspace here<\/a>.<\/p>\n\n\n\n
The next step will be to log all our generated reviews from our model as text.<\/p>\n\n\n\n
To log in our artifacts, we create an artifact instance by giving it a name, artifact_type, and specifying the file path with artifact.add()<\/code>. Artifacts here means dataset.<\/p>\n\n\n\n
# Initialize Comet<\/span>\nexperiment = comet_ml.Experiment(\n    project_name=\"Text Summarization\"<\/span>,\n    workspace=\"bennykillua\"<\/span>,\n    api_key=\"YOUR KEY\"<\/span>,\n    auto_metric_logging=True<\/span>,\n    auto_param_logging=True<\/span>,\n    auto_histogram_weight_logging=True<\/span>,\n    auto_histogram_gradient_logging=True<\/span>,\n    auto_histogram_activation_logging=True<\/span>,\n    log_code=True<\/span>\n)\n\n# Log the generated reviews as text<\/span>\nexperiment.log_text(\"Generated Reviews\"<\/span>, reviews_text)\n\ncomet_ml.login(api_key=\"YOUR KEY\"<\/span>)\n\n# Initialize an Artifact<\/span>\nartifact = Artifact(name=\"Reviews\"<\/span>, artifact_type=\"dataset\"<\/span>)\n\n# Specify the path of the artifact (the dataset file)<\/span>\nartifact.add(r\"\/content\/reviews_data.csv\"<\/span>)\n\n# Log the artifact to the experiment (Comet platform)<\/span>\nexperiment.log_artifact(artifact)\n\n# Log the BLEU score<\/span>\nexperiment.log_metric(\"BLEU Score\"<\/span>, bleu_score)\n\n# Log the model<\/span>\nexperiment.log_model(reviews_summarizer, 'model'<\/span>)\n\n# End the experiment<\/span>\nexperiment.end()<\/span><\/pre>\n\n\n\n
\"\"\/
The BLEU score (Image by Author)<\/figcaption><\/figure>\n\n\n\n
 <\/p>\n\n\n\n
Here is the complete code:<\/p>\n\n\n\n
{\n  \"nbformat\": 4,\n  \"nbformat_minor\": 0,\n  \"metadata\": {\n    \"colab\": {\n      \"provenance\": [],\n      \"authorship_tag\": \"ABX9TyNQMK9Afs279mqdTEWTKfir\",\n      \"include_colab_link\": true\n    },\n    \"kernelspec\": {\n      \"name\": \"python3\",\n      \"display_name\": \"Python 3\"\n    },\n    \"language_info\": {\n      \"name\": \"python\"\n    }\n  },\n  \"cells\": [\n    {\n      \"cell_type\": \"markdown\",\n      \"metadata\": {\n        \"id\": \"view-in-github\",\n        \"colab_type\": \"text\"\n      },\n      \"source\": [\n        \"<a href=\\\"https:\/\/colab.research.google.com\/gist\/Bennykillua\/db4e3673029d0b933c97bcfe141fdff2\/scikit-llm-sklearn-meets-large-language-models.ipynb\\\" target=\\\"_parent\\\"><img src=\\\"https:\/\/colab.research.google.com\/assets\/colab-badge.svg\\\" alt=\\\"Open In Colab\\\"\/><\/a>\"\n      ]\n    },\n    {\n      \"cell_type\": \"markdown\",\n      \"source\": [\n        \"# Text Summarization Modeling with ScikitLLM and Comet ML\\n\",\n        \"\\n\",\n        \"## Install Scikit-LLM\\n\",\n        \"\\n\",\n        \"Scikit-LLM integrate powerful language models like ChatGPT into scikit-learn for enhanced text analysis tasks.\\n\",\n        \"\\n\",\n        \"Check out the [Scikit-LLM offical docs](https:\/\/github.com\/iryna-kondr\/scikit-llm)\\n\"\n      ],\n      \"metadata\": {\n        \"id\": \"9Kp3v7WGLnn2\"\n      }\n    },\n    {\n      \"cell_type\": \"code\",\n      \"execution_count\": 2,\n      \"metadata\": {\n        \"id\": \"GWogANz_MQJG\",\n        \"colab\": {\n          \"base_uri\": \"https:\/\/localhost:8080\/\"\n        },\n        \"outputId\": \"b9a9de35-d2b1-458d-bf12-a2d10ef5e8e4\"\n      },\n      \"outputs\": [\n        {\n          \"output_type\": \"stream\",\n          \"name\": \"stdout\",\n          \"text\": [\n            \"Requirement already satisfied: scikit-llm in \/usr\/local\/lib\/python3.10\/dist-packages (0.4.1)\\n\",\n            \"Requirement already satisfied: scikit-learn>=1.1.0 in \/usr\/local\/lib\/python3.10\/dist-packages (from scikit-llm) (1.2.2)\\n\",\n            \"Requirement already satisfied: pandas>=1.5.0 in \/usr\/local\/lib\/python3.10\/dist-packages (from scikit-llm) (1.5.3)\\n\",\n            \"Requirement already satisfied: openai>=0.27.9 in \/usr\/local\/lib\/python3.10\/dist-packages (from scikit-llm) (0.28.0)\\n\",\n            \"Requirement already satisfied: tqdm>=4.60.0 in \/usr\/local\/lib\/python3.10\/dist-packages (from scikit-llm) (4.66.1)\\n\",\n            \"Requirement already satisfied: google-cloud-aiplatform>=1.27.0 in \/usr\/local\/lib\/python3.10\/dist-packages (from scikit-llm) (1.33.1)\\n\",\n            \"Requirement already satisfied: google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0 in \/usr\/local\/lib\/python3.10\/dist-packages (from google-cloud-aiplatform>=1.27.0->scikit-llm) (2.11.1)\\n\",\n            \"Requirement already satisfied: proto-plus<2.0.0dev,>=1.22.0 in \/usr\/local\/lib\/python3.10\/dist-packages (from google-cloud-aiplatform>=1.27.0->scikit-llm) (1.22.3)\\n\",\n            \"Requirement already satisfied: protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5 in \/usr\/local\/lib\/python3.10\/dist-packages (from google-cloud-aiplatform>=1.27.0->scikit-llm) (3.20.3)\\n\",\n            \"Requirement already satisfied: packaging>=14.3 in \/usr\/local\/lib\/python3.10\/dist-packages (from google-cloud-aiplatform>=1.27.0->scikit-llm) (23.1)\\n\",\n            \"Requirement already satisfied: google-cloud-storage<3.0.0dev,>=1.32.0 in \/usr\/local\/lib\/python3.10\/dist-packages (from google-cloud-aiplatform>=1.27.0->scikit-llm) (2.8.0)\\n\",\n            \"Requirement already satisfied: google-cloud-bigquery<4.0.0dev,>=1.15.0 in \/usr\/local\/lib\/python3.10\/dist-packages (from google-cloud-aiplatform>=1.27.0->scikit-llm) (3.10.0)\\n\",\n            \"Requirement already satisfied: google-cloud-resource-manager<3.0.0dev,>=1.3.3 in \/usr\/local\/lib\/python3.10\/dist-packages (from google-cloud-aiplatform>=1.27.0->scikit-llm) (1.10.4)\\n\",\n            \"Requirement already satisfied: shapely<2.0.0 in \/usr\/local\/lib\/python3.10\/dist-packages (from google-cloud-aiplatform>=1.27.0->scikit-llm) (1.8.5.post1)\\n\",\n            \"Requirement already satisfied: requests>=2.20 in \/usr\/local\/lib\/python3.10\/dist-packages (from openai>=0.27.9->scikit-llm) (2.31.0)\\n\",\n            \"Requirement already satisfied: aiohttp in \/usr\/local\/lib\/python3.10\/dist-packages (from openai>=0.27.9->scikit-llm) (3.8.5)\\n\",\n            \"Requirement already satisfied: python-dateutil>=2.8.1 in \/usr\/local\/lib\/python3.10\/dist-packages (from pandas>=1.5.0->scikit-llm) (2.8.2)\\n\",\n            \"Requirement already satisfied: pytz>=2020.1 in \/usr\/local\/lib\/python3.10\/dist-packages (from pandas>=1.5.0->scikit-llm) (2023.3.post1)\\n\",\n            \"Requirement already satisfied: numpy>=1.21.0 in \/usr\/local\/lib\/python3.10\/dist-packages (from pandas>=1.5.0->scikit-llm) (1.23.5)\\n\",\n            \"Requirement already satisfied: scipy>=1.3.2 in \/usr\/local\/lib\/python3.10\/dist-packages (from scikit-learn>=1.1.0->scikit-llm) (1.11.2)\\n\",\n            \"Requirement already satisfied: joblib>=1.1.1 in \/usr\/local\/lib\/python3.10\/dist-packages (from scikit-learn>=1.1.0->scikit-llm) (1.3.2)\\n\",\n            \"Requirement already satisfied: threadpoolctl>=2.0.0 in \/usr\/local\/lib\/python3.10\/dist-packages (from scikit-learn>=1.1.0->scikit-llm) (3.2.0)\\n\",\n            \"Requirement already satisfied: googleapis-common-protos<2.0.dev0,>=1.56.2 in \/usr\/local\/lib\/python3.10\/dist-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform>=1.27.0->scikit-llm) (1.60.0)\\n\",\n            \"Requirement already satisfied: google-auth<3.0.dev0,>=2.14.1 in \/usr\/local\/lib\/python3.10\/dist-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform>=1.27.0->scikit-llm) (2.17.3)\\n\",\n            \"Requirement already satisfied: grpcio<2.0dev,>=1.33.2 in \/usr\/local\/lib\/python3.10\/dist-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform>=1.27.0->scikit-llm) (1.57.0)\\n\",\n            \"Requirement already satisfied: grpcio-status<2.0.dev0,>=1.33.2 in \/usr\/local\/lib\/python3.10\/dist-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform>=1.27.0->scikit-llm) (1.48.2)\\n\",\n            \"Requirement already satisfied: google-cloud-core<3.0.0dev,>=1.6.0 in \/usr\/local\/lib\/python3.10\/dist-packages (from google-cloud-bigquery<4.0.0dev,>=1.15.0->google-cloud-aiplatform>=1.27.0->scikit-llm) (2.3.3)\\n\",\n            \"Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in \/usr\/local\/lib\/python3.10\/dist-packages (from google-cloud-bigquery<4.0.0dev,>=1.15.0->google-cloud-aiplatform>=1.27.0->scikit-llm) (2.6.0)\\n\",\n            \"Requirement already satisfied: grpc-google-iam-v1<1.0.0dev,>=0.12.4 in \/usr\/local\/lib\/python3.10\/dist-packages (from google-cloud-resource-manager<3.0.0dev,>=1.3.3->google-cloud-aiplatform>=1.27.0->scikit-llm) (0.12.6)\\n\",\n            \"Requirement already satisfied: six>=1.5 in \/usr\/local\/lib\/python3.10\/dist-packages (from python-dateutil>=2.8.1->pandas>=1.5.0->scikit-llm) (1.16.0)\\n\",\n            \"Requirement already satisfied: charset-normalizer<4,>=2 in \/usr\/local\/lib\/python3.10\/dist-packages (from requests>=2.20->openai>=0.27.9->scikit-llm) (3.2.0)\\n\",\n            \"Requirement already satisfied: idna<4,>=2.5 in \/usr\/local\/lib\/python3.10\/dist-packages (from requests>=2.20->openai>=0.27.9->scikit-llm) (3.4)\\n\",\n            \"Requirement already satisfied: urllib3<3,>=1.21.1 in \/usr\/local\/lib\/python3.10\/dist-packages (from requests>=2.20->openai>=0.27.9->scikit-llm) (2.0.4)\\n\",\n            \"Requirement already satisfied: certifi>=2017.4.17 in \/usr\/local\/lib\/python3.10\/dist-packages (from requests>=2.20->openai>=0.27.9->scikit-llm) (2023.7.22)\\n\",\n            \"Requirement already satisfied: attrs>=17.3.0 in \/usr\/local\/lib\/python3.10\/dist-packages (from aiohttp->openai>=0.27.9->scikit-llm) (23.1.0)\\n\",\n            \"Requirement already satisfied: multidict<7.0,>=4.5 in \/usr\/local\/lib\/python3.10\/dist-packages (from aiohttp->openai>=0.27.9->scikit-llm) (6.0.4)\\n\",\n            \"Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in \/usr\/local\/lib\/python3.10\/dist-packages (from aiohttp->openai>=0.27.9->scikit-llm) (4.0.3)\\n\",\n            \"Requirement already satisfied: yarl<2.0,>=1.0 in \/usr\/local\/lib\/python3.10\/dist-packages (from aiohttp->openai>=0.27.9->scikit-llm) (1.9.2)\\n\",\n            \"Requirement already satisfied: frozenlist>=1.1.1 in \/usr\/local\/lib\/python3.10\/dist-packages (from aiohttp->openai>=0.27.9->scikit-llm) (1.4.0)\\n\",\n            \"Requirement already satisfied: aiosignal>=1.1.2 in \/usr\/local\/lib\/python3.10\/dist-packages (from aiohttp->openai>=0.27.9->scikit-llm) (1.3.1)\\n\",\n            \"Requirement already satisfied: cachetools<6.0,>=2.0.0 in \/usr\/local\/lib\/python3.10\/dist-packages (from google-auth<3.0.dev0,>=2.14.1->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform>=1.27.0->scikit-llm) (5.3.1)\\n\",\n            \"Requirement already satisfied: pyasn1-modules>=0.2.1 in \/usr\/local\/lib\/python3.10\/dist-packages (from google-auth<3.0.dev0,>=2.14.1->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform>=1.27.0->scikit-llm) (0.3.0)\\n\",\n            \"Requirement already satisfied: rsa<5,>=3.1.4 in \/usr\/local\/lib\/python3.10\/dist-packages (from google-auth<3.0.dev0,>=2.14.1->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform>=1.27.0->scikit-llm) (4.9)\\n\",\n            \"Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in \/usr\/local\/lib\/python3.10\/dist-packages (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4.0.0dev,>=1.15.0->google-cloud-aiplatform>=1.27.0->scikit-llm) (1.5.0)\\n\",\n            \"Requirement already satisfied: pyasn1<0.6.0,>=0.4.6 in \/usr\/local\/lib\/python3.10\/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3.0.dev0,>=2.14.1->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform>=1.27.0->scikit-llm) (0.5.0)\\n\"\n          ]\n        }\n      ],\n      \"source\": [\n        \"pip install scikit-llm\"\n      ]\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [\n        \"# importing SKLLMConfig\\n\",\n        \"from skllm.config import SKLLMConfig\\n\",\n        \"\\n\",\n        \"# Set your OpenAI API key\\n\",\n        \"SKLLMConfig.set_openai_key(\\\"*******\\\")\\n\",\n        \"\\n\",\n        \"# Set your OpenAI organization\\n\",\n        \"\\n\",\n        \"SKLLMConfig.set_openai_org(\\\"**ABC**\\\")\"\n      ],\n      \"metadata\": {\n        \"id\": \"g9D6yVFo4rdq\"\n      },\n      \"execution_count\": 25,\n      \"outputs\": []\n    },\n    {\n      \"cell_type\": \"markdown\",\n      \"source\": [\n        \"### Mounting our drive\"\n      ],\n      \"metadata\": {\n        \"id\": \"KPe07ry0MKMV\"\n      }\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [\n        \"from google.colab import drive\\n\",\n        \"drive.mount('\/content\/drive')\"\n      ],\n      \"metadata\": {\n        \"colab\": {\n          \"base_uri\": \"https:\/\/localhost:8080\/\"\n        },\n        \"id\": \"bLwZw66Hrazf\",\n        \"outputId\": \"4789c4d3-8324-4a48-af5c-ba99e5dfc6b2\"\n      },\n      \"execution_count\": 4,\n      \"outputs\": [\n        {\n          \"output_type\": \"stream\",\n          \"name\": \"stdout\",\n          \"text\": [\n            \"Drive already mounted at \/content\/drive; to attempt to forcibly remount, call drive.mount(\\\"\/content\/drive\\\", force_remount=True).\\n\"\n          ]\n        }\n      ]\n    },\n    {\n      \"cell_type\": \"markdown\",\n      \"source\": [\n        \"### Our dataset\\n\",\n        \"\\n\",\n        \"We will make use of the [Starbucks reviews dataset](https:\/\/www.kaggle.com\/datasets\/harshalhonde\/starbucks-reviews-dataset). This dataset contains information about reviews, rating, and location data from various Starbucks outlets.\"\n      ],\n      \"metadata\": {\n        \"id\": \"ix1Ws69mMQlZ\"\n      }\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [\n        \"import pandas as pd\\n\",\n        \"\\n\",\n        \"df = pd.read_csv('\/content\/reviews_data.csv')\"\n      ],\n      \"metadata\": {\n        \"id\": \"Krok0k6zMeVt\"\n      },\n      \"execution_count\": 5,\n      \"outputs\": []\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [\n        \"df.head(5)\"\n      ],\n      \"metadata\": {\n        \"colab\": {\n          \"base_uri\": \"https:\/\/localhost:8080\/\",\n          \"height\": 293\n        },\n        \"id\": \"FITj6vTn8I1V\",\n        \"outputId\": \"dd688b2e-7ce6-4fe9-c936-37b1a83843a5\"\n      },\n      \"execution_count\": 6,\n      \"outputs\": [\n        {\n          \"output_type\": \"execute_result\",\n          \"data\": {\n            \"text\/plain\": [\n              \"       name           location                     Date  Rating  \\\\\\n\",\n              \"0     Helen  Wichita Falls, TX  Reviewed Sept. 13, 2023     5.0   \\n\",\n              \"1  Courtney         Apopka, FL   Reviewed July 16, 2023     5.0   \\n\",\n              \"2  Daynelle  Cranberry Twp, PA    Reviewed July 5, 2023     5.0   \\n\",\n              \"3    Taylor        Seattle, WA    Reviewed May 26, 2023     5.0   \\n\",\n              \"4   Tenessa        Gresham, OR   Reviewed Jan. 22, 2023     5.0   \\n\",\n              \"\\n\",\n              \"                                              Review  \\\\\\n\",\n              \"0  Amber and LaDonna at the Starbucks on Southwes...   \\n\",\n              \"1  ** at the Starbucks by the fire station on 436...   \\n\",\n              \"2  I just wanted to go out of my way to recognize...   \\n\",\n              \"3  Me and my friend were at Starbucks and my card...   \\n\",\n              \"4  I\u2019m on this kick of drinking 5 cups of warm wa...   \\n\",\n              \"\\n\",\n              \"                                         Image_Links  \\n\",\n              \"0                                      ['No Images']  \\n\",\n              \"1                                      ['No Images']  \\n\",\n              \"2  ['https:\/\/media.consumeraffairs.com\/files\/cach...  \\n\",\n              \"3                                      ['No Images']  \\n\",\n              \"4  ['https:\/\/media.consumeraffairs.com\/files\/cach...  \"\n            ],\n            \"text\/html\": [\n              \"\\n\",\n              \"  <div id=\\\"df-bd6fdd44-67e9-4e5b-9132-30166cf4137d\\\" class=\\\"colab-df-container\\\">\\n\",\n              \"    <div>\\n\",\n              \"<style scoped>\\n\",\n              \"    .dataframe tbody tr th:only-of-type {\\n\",\n              \"        vertical-align: middle;\\n\",\n              \"    }\\n\",\n              \"\\n\",\n              \"    .dataframe tbody tr th {\\n\",\n              \"        vertical-align: top;\\n\",\n              \"    }\\n\",\n              \"\\n\",\n              \"    .dataframe thead th {\\n\",\n              \"        text-align: right;\\n\",\n              \"    }\\n\",\n              \"<\/style>\\n\",\n              \"<table border=\\\"1\\\" class=\\\"dataframe\\\">\\n\",\n              \"  <thead>\\n\",\n              \"    <tr style=\\\"text-align: right;\\\">\\n\",\n              \"      <th><\/th>\\n\",\n              \"      <th>name<\/th>\\n\",\n              \"      <th>location<\/th>\\n\",\n              \"      <th>Date<\/th>\\n\",\n              \"      <th>Rating<\/th>\\n\",\n              \"      <th>Review<\/th>\\n\",\n              \"      <th>Image_Links<\/th>\\n\",\n              \"    <\/tr>\\n\",\n              \"  <\/thead>\\n\",\n              \"  <tbody>\\n\",\n              \"    <tr>\\n\",\n              \"      <th>0<\/th>\\n\",\n              \"      <td>Helen<\/td>\\n\",\n              \"      <td>Wichita Falls, TX<\/td>\\n\",\n              \"      <td>Reviewed Sept. 13, 2023<\/td>\\n\",\n              \"      <td>5.0<\/td>\\n\",\n              \"      <td>Amber and LaDonna at the Starbucks on Southwes...<\/td>\\n\",\n              \"      <td>['No Images']<\/td>\\n\",\n              \"    <\/tr>\\n\",\n              \"    <tr>\\n\",\n              \"      <th>1<\/th>\\n\",\n              \"      <td>Courtney<\/td>\\n\",\n              \"      <td>Apopka, FL<\/td>\\n\",\n              \"      <td>Reviewed July 16, 2023<\/td>\\n\",\n              \"      <td>5.0<\/td>\\n\",\n              \"      <td>** at the Starbucks by the fire station on 436...<\/td>\\n\",\n              \"      <td>['No Images']<\/td>\\n\",\n              \"    <\/tr>\\n\",\n              \"    <tr>\\n\",\n              \"      <th>2<\/th>\\n\",\n              \"      <td>Daynelle<\/td>\\n\",\n              \"      <td>Cranberry Twp, PA<\/td>\\n\",\n              \"      <td>Reviewed July 5, 2023<\/td>\\n\",\n              \"      <td>5.0<\/td>\\n\",\n              \"      <td>I just wanted to go out of my way to recognize...<\/td>\\n\",\n              \"      <td>['https:\/\/media.consumeraffairs.com\/files\/cach...<\/td>\\n\",\n              \"    <\/tr>\\n\",\n              \"    <tr>\\n\",\n              \"      <th>3<\/th>\\n\",\n              \"      <td>Taylor<\/td>\\n\",\n              \"      <td>Seattle, WA<\/td>\\n\",\n              \"      <td>Reviewed May 26, 2023<\/td>\\n\",\n              \"      <td>5.0<\/td>\\n\",\n              \"      <td>Me and my friend were at Starbucks and my card...<\/td>\\n\",\n              \"      <td>['No Images']<\/td>\\n\",\n              \"    <\/tr>\\n\",\n              \"    <tr>\\n\",\n              \"      <th>4<\/th>\\n\",\n              \"      <td>Tenessa<\/td>\\n\",\n              \"      <td>Gresham, OR<\/td>\\n\",\n              \"      <td>Reviewed Jan. 22, 2023<\/td>\\n\",\n              \"      <td>5.0<\/td>\\n\",\n              \"      <td>I\u2019m on this kick of drinking 5 cups of warm wa...<\/td>\\n\",\n              \"      <td>['https:\/\/media.consumeraffairs.com\/files\/cach...<\/td>\\n\",\n              \"    <\/tr>\\n\",\n              \"  <\/tbody>\\n\",\n              \"<\/table>\\n\",\n              \"<\/div>\\n\",\n              \"    <div class=\\\"colab-df-buttons\\\">\\n\",\n              \"\\n\",\n              \"  <div class=\\\"colab-df-container\\\">\\n\",\n              \"    <button class=\\\"colab-df-convert\\\" onclick=\\\"convertToInteractive('df-bd6fdd44-67e9-4e5b-9132-30166cf4137d')\\\"\\n\",\n              \"            title=\\\"Convert this dataframe to an interactive table.\\\"\\n\",\n              \"            style=\\\"display:none;\\\">\\n\",\n              \"\\n\",\n              \"  <svg xmlns=\\\"http:\/\/www.w3.org\/2000\/svg\\\" height=\\\"24px\\\" viewBox=\\\"0 -960 960 960\\\">\\n\",\n              \"    <path d=\\\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\\\"\/>\\n\",\n              \"  <\/svg>\\n\",\n              \"    <\/button>\\n\",\n              \"\\n\",\n              \"  <style>\\n\",\n              \"    .colab-df-container {\\n\",\n              \"      display:flex;\\n\",\n              \"      gap: 12px;\\n\",\n              \"    }\\n\",\n              \"\\n\",\n              \"    .colab-df-convert {\\n\",\n              \"      background-color: #E8F0FE;\\n\",\n              \"      border: none;\\n\",\n              \"      border-radius: 50%;\\n\",\n              \"      cursor: pointer;\\n\",\n              \"      display: none;\\n\",\n              \"      fill: #1967D2;\\n\",\n              \"      height: 32px;\\n\",\n              \"      padding: 0 0 0 0;\\n\",\n              \"      width: 32px;\\n\",\n              \"    }\\n\",\n              \"\\n\",\n              \"    .colab-df-convert:hover {\\n\",\n              \"      background-color: #E2EBFA;\\n\",\n              \"      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\\n\",\n              \"      fill: #174EA6;\\n\",\n              \"    }\\n\",\n              \"\\n\",\n              \"    .colab-df-buttons div {\\n\",\n              \"      margin-bottom: 4px;\\n\",\n              \"    }\\n\",\n              \"\\n\",\n              \"    [theme=dark] .colab-df-convert {\\n\",\n              \"      background-color: #3B4455;\\n\",\n              \"      fill: #D2E3FC;\\n\",\n              \"    }\\n\",\n              \"\\n\",\n              \"    [theme=dark] .colab-df-convert:hover {\\n\",\n              \"      background-color: #434B5C;\\n\",\n              \"      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\\n\",\n              \"      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\\n\",\n              \"      fill: #FFFFFF;\\n\",\n              \"    }\\n\",\n              \"  <\/style>\\n\",\n              \"\\n\",\n              \"    <script>\\n\",\n              \"      const buttonEl =\\n\",\n              \"        document.querySelector('#df-bd6fdd44-67e9-4e5b-9132-30166cf4137d button.colab-df-convert');\\n\",\n              \"      buttonEl.style.display =\\n\",\n              \"        google.colab.kernel.accessAllowed ? 'block' : 'none';\\n\",\n              \"\\n\",\n              \"      async function convertToInteractive(key) {\\n\",\n              \"        const element = document.querySelector('#df-bd6fdd44-67e9-4e5b-9132-30166cf4137d');\\n\",\n              \"        const dataTable =\\n\",\n              \"          await google.colab.kernel.invokeFunction('convertToInteractive',\\n\",\n              \"                                                    [key], {});\\n\",\n              \"        if (!dataTable) return;\\n\",\n              \"\\n\",\n              \"        const docLinkHtml = 'Like what you see? Visit the ' +\\n\",\n              \"          '<a target=\\\"_blank\\\" href=https:\/\/colab.research.google.com\/notebooks\/data_table.ipynb>data table notebook<\/a>'\\n\",\n              \"          + ' to learn more about interactive tables.';\\n\",\n              \"        element.innerHTML = '';\\n\",\n              \"        dataTable['output_type'] = 'display_data';\\n\",\n              \"        await google.colab.output.renderOutput(dataTable, element);\\n\",\n              \"        const docLink = document.createElement('div');\\n\",\n              \"        docLink.innerHTML = docLinkHtml;\\n\",\n              \"        element.appendChild(docLink);\\n\",\n              \"      }\\n\",\n              \"    <\/script>\\n\",\n              \"  <\/div>\\n\",\n              \"\\n\",\n              \"\\n\",\n              \"<div id=\\\"df-54455e1a-8aed-4f88-b643-16b4094707e1\\\">\\n\",\n              \"  <button class=\\\"colab-df-quickchart\\\" onclick=\\\"quickchart('df-54455e1a-8aed-4f88-b643-16b4094707e1')\\\"\\n\",\n              \"            title=\\\"Suggest charts.\\\"\\n\",\n              \"            style=\\\"display:none;\\\">\\n\",\n              \"\\n\",\n              \"<svg xmlns=\\\"http:\/\/www.w3.org\/2000\/svg\\\" height=\\\"24px\\\"viewBox=\\\"0 0 24 24\\\"\\n\",\n              \"     width=\\\"24px\\\">\\n\",\n              \"    <g>\\n\",\n              \"        <path d=\\\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\\\"\/>\\n\",\n              \"    <\/g>\\n\",\n              \"<\/svg>\\n\",\n              \"  <\/button>\\n\",\n              \"\\n\",\n              \"<style>\\n\",\n              \"  .colab-df-quickchart {\\n\",\n              \"      --bg-color: #E8F0FE;\\n\",\n              \"      --fill-color: #1967D2;\\n\",\n              \"      --hover-bg-color: #E2EBFA;\\n\",\n              \"      --hover-fill-color: #174EA6;\\n\",\n              \"      --disabled-fill-color: #AAA;\\n\",\n              \"      --disabled-bg-color: #DDD;\\n\",\n              \"  }\\n\",\n              \"\\n\",\n              \"  [theme=dark] .colab-df-quickchart {\\n\",\n              \"      --bg-color: #3B4455;\\n\",\n              \"      --fill-color: #D2E3FC;\\n\",\n              \"      --hover-bg-color: #434B5C;\\n\",\n              \"      --hover-fill-color: #FFFFFF;\\n\",\n              \"      --disabled-bg-color: #3B4455;\\n\",\n              \"      --disabled-fill-color: #666;\\n\",\n              \"  }\\n\",\n              \"\\n\",\n              \"  .colab-df-quickchart {\\n\",\n              \"    background-color: var(--bg-color);\\n\",\n              \"    border: none;\\n\",\n              \"    border-radius: 50%;\\n\",\n              \"    cursor: pointer;\\n\",\n              \"    display: none;\\n\",\n              \"    fill: var(--fill-color);\\n\",\n              \"    height: 32px;\\n\",\n              \"    padding: 0;\\n\",\n              \"    width: 32px;\\n\",\n              \"  }\\n\",\n              \"\\n\",\n              \"  .colab-df-quickchart:hover {\\n\",\n              \"    background-color: var(--hover-bg-color);\\n\",\n              \"    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\\n\",\n              \"    fill: var(--button-hover-fill-color);\\n\",\n              \"  }\\n\",\n              \"\\n\",\n              \"  .colab-df-quickchart-complete:disabled,\\n\",\n              \"  .colab-df-quickchart-complete:disabled:hover {\\n\",\n              \"    background-color: var(--disabled-bg-color);\\n\",\n              \"    fill: var(--disabled-fill-color);\\n\",\n              \"    box-shadow: none;\\n\",\n              \"  }\\n\",\n              \"\\n\",\n              \"  .colab-df-spinner {\\n\",\n              \"    border: 2px solid var(--fill-color);\\n\",\n              \"    border-color: transparent;\\n\",\n              \"    border-bottom-color: var(--fill-color);\\n\",\n              \"    animation:\\n\",\n              \"      spin 1s steps(1) infinite;\\n\",\n              \"  }\\n\",\n              \"\\n\",\n              \"  @keyframes spin {\\n\",\n              \"    0% {\\n\",\n              \"      border-color: transparent;\\n\",\n              \"      border-bottom-color: var(--fill-color);\\n\",\n              \"      border-left-color: var(--fill-color);\\n\",\n              \"    }\\n\",\n              \"    20% {\\n\",\n              \"      border-color: transparent;\\n\",\n              \"      border-left-color: var(--fill-color);\\n\",\n              \"      border-top-color: var(--fill-color);\\n\",\n              \"    }\\n\",\n              \"    30% {\\n\",\n              \"      border-color: transparent;\\n\",\n              \"      border-left-color: var(--fill-color);\\n\",\n              \"      border-top-color: var(--fill-color);\\n\",\n              \"      border-right-color: var(--fill-color);\\n\",\n              \"    }\\n\",\n              \"    40% {\\n\",\n              \"      border-color: transparent;\\n\",\n              \"      border-right-color: var(--fill-color);\\n\",\n              \"      border-top-color: var(--fill-color);\\n\",\n              \"    }\\n\",\n              \"    60% {\\n\",\n              \"      border-color: transparent;\\n\",\n              \"      border-right-color: var(--fill-color);\\n\",\n              \"    }\\n\",\n              \"    80% {\\n\",\n              \"      border-color: transparent;\\n\",\n              \"      border-right-color: var(--fill-color);\\n\",\n              \"      border-bottom-color: var(--fill-color);\\n\",\n              \"    }\\n\",\n              \"    90% {\\n\",\n              \"      border-color: transparent;\\n\",\n              \"      border-bottom-color: var(--fill-color);\\n\",\n              \"    }\\n\",\n              \"  }\\n\",\n              \"<\/style>\\n\",\n              \"\\n\",\n              \"  <script>\\n\",\n              \"    async function quickchart(key) {\\n\",\n              \"      const quickchartButtonEl =\\n\",\n              \"        document.querySelector('#' + key + ' button');\\n\",\n              \"      quickchartButtonEl.disabled = true;  \/\/ To prevent multiple clicks.\\n\",\n              \"      quickchartButtonEl.classList.add('colab-df-spinner');\\n\",\n              \"      try {\\n\",\n              \"        const charts = await google.colab.kernel.invokeFunction(\\n\",\n              \"            'suggestCharts', [key], {});\\n\",\n              \"      } catch (error) {\\n\",\n              \"        console.error('Error during call to suggestCharts:', error);\\n\",\n              \"      }\\n\",\n              \"      quickchartButtonEl.classList.remove('colab-df-spinner');\\n\",\n              \"      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\\n\",\n              \"    }\\n\",\n              \"    (() => {\\n\",\n              \"      let quickchartButtonEl =\\n\",\n              \"        document.querySelector('#df-54455e1a-8aed-4f88-b643-16b4094707e1 button');\\n\",\n              \"      quickchartButtonEl.style.display =\\n\",\n              \"        google.colab.kernel.accessAllowed ? 'block' : 'none';\\n\",\n              \"    })();\\n\",\n              \"  <\/script>\\n\",\n              \"<\/div>\\n\",\n              \"    <\/div>\\n\",\n              \"  <\/div>\\n\"\n            ]\n          },\n          \"metadata\": {},\n          \"execution_count\": 6\n        }\n      ]\n    },\n    {\n      \"cell_type\": \"markdown\",\n      \"source\": [\n        \"## Text Summarization\"\n      ],\n      \"metadata\": {\n        \"id\": \"aKjN6YB878_z\"\n      }\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [\n        \"!pip install comet_ml\"\n      ],\n      \"metadata\": {\n        \"colab\": {\n          \"base_uri\": \"https:\/\/localhost:8080\/\"\n        },\n        \"id\": \"zWEwZQdp8mYH\",\n        \"outputId\": \"322d7d38-ba74-4131-c475-c123d3a5c6d0\"\n      },\n      \"execution_count\": 8,\n      \"outputs\": [\n        {\n          \"output_type\": \"stream\",\n          \"name\": \"stdout\",\n          \"text\": [\n            \"Requirement already satisfied: comet_ml in \/usr\/local\/lib\/python3.10\/dist-packages (3.33.10)\\n\",\n            \"Requirement already satisfied: jsonschema!=3.1.0,>=2.6.0 in \/usr\/local\/lib\/python3.10\/dist-packages (from comet_ml) (4.19.0)\\n\",\n            \"Requirement already satisfied: psutil>=5.6.3 in \/usr\/local\/lib\/python3.10\/dist-packages (from comet_ml) (5.9.5)\\n\",\n            \"Requirement already satisfied: python-box<7.0.0 in \/usr\/local\/lib\/python3.10\/dist-packages (from comet_ml) (6.1.0)\\n\",\n            \"Requirement already satisfied: requests-toolbelt>=0.8.0 in \/usr\/local\/lib\/python3.10\/dist-packages (from comet_ml) (1.0.0)\\n\",\n            \"Requirement already satisfied: requests>=2.18.4 in \/usr\/local\/lib\/python3.10\/dist-packages (from comet_ml) (2.31.0)\\n\",\n            \"Requirement already satisfied: semantic-version>=2.8.0 in \/usr\/local\/lib\/python3.10\/dist-packages (from comet_ml) (2.10.0)\\n\",\n            \"Requirement already satisfied: sentry-sdk>=1.1.0 in \/usr\/local\/lib\/python3.10\/dist-packages (from comet_ml) (1.31.0)\\n\",\n            \"Requirement already satisfied: simplejson in \/usr\/local\/lib\/python3.10\/dist-packages (from comet_ml) (3.19.1)\\n\",\n            \"Requirement already satisfied: six in \/usr\/local\/lib\/python3.10\/dist-packages (from comet_ml) (1.16.0)\\n\",\n            \"Requirement already satisfied: urllib3>=1.21.1 in \/usr\/local\/lib\/python3.10\/dist-packages (from comet_ml) (2.0.4)\\n\",\n            \"Requirement already satisfied: websocket-client<1.4.0,>=0.55.0 in \/usr\/local\/lib\/python3.10\/dist-packages (from comet_ml) (1.3.3)\\n\",\n            \"Requirement already satisfied: wrapt>=1.11.2 in \/usr\/local\/lib\/python3.10\/dist-packages (from comet_ml) (1.15.0)\\n\",\n            \"Requirement already satisfied: wurlitzer>=1.0.2 in \/usr\/local\/lib\/python3.10\/dist-packages (from comet_ml) (3.0.3)\\n\",\n            \"Requirement already satisfied: everett[ini]<3.2.0,>=1.0.1 in \/usr\/local\/lib\/python3.10\/dist-packages (from comet_ml) (3.1.0)\\n\",\n            \"Requirement already satisfied: dulwich!=0.20.33,>=0.20.6 in \/usr\/local\/lib\/python3.10\/dist-packages (from comet_ml) (0.21.6)\\n\",\n            \"Requirement already satisfied: rich>=13.3.2 in \/usr\/local\/lib\/python3.10\/dist-packages (from comet_ml) (13.5.2)\\n\",\n            \"Requirement already satisfied: configobj in \/usr\/local\/lib\/python3.10\/dist-packages (from everett[ini]<3.2.0,>=1.0.1->comet_ml) (5.0.8)\\n\",\n            \"Requirement already satisfied: attrs>=22.2.0 in \/usr\/local\/lib\/python3.10\/dist-packages (from jsonschema!=3.1.0,>=2.6.0->comet_ml) (23.1.0)\\n\",\n            \"Requirement already satisfied: jsonschema-specifications>=2023.03.6 in \/usr\/local\/lib\/python3.10\/dist-packages (from jsonschema!=3.1.0,>=2.6.0->comet_ml) (2023.7.1)\\n\",\n            \"Requirement already satisfied: referencing>=0.28.4 in \/usr\/local\/lib\/python3.10\/dist-packages (from jsonschema!=3.1.0,>=2.6.0->comet_ml) (0.30.2)\\n\",\n            \"Requirement already satisfied: rpds-py>=0.7.1 in \/usr\/local\/lib\/python3.10\/dist-packages (from jsonschema!=3.1.0,>=2.6.0->comet_ml) (0.10.2)\\n\",\n            \"Requirement already satisfied: charset-normalizer<4,>=2 in \/usr\/local\/lib\/python3.10\/dist-packages (from requests>=2.18.4->comet_ml) (3.2.0)\\n\",\n            \"Requirement already satisfied: idna<4,>=2.5 in \/usr\/local\/lib\/python3.10\/dist-packages (from requests>=2.18.4->comet_ml) (3.4)\\n\",\n            \"Requirement already satisfied: certifi>=2017.4.17 in \/usr\/local\/lib\/python3.10\/dist-packages (from requests>=2.18.4->comet_ml) (2023.7.22)\\n\",\n            \"Requirement already satisfied: markdown-it-py>=2.2.0 in \/usr\/local\/lib\/python3.10\/dist-packages (from rich>=13.3.2->comet_ml) (3.0.0)\\n\",\n            \"Requirement already satisfied: pygments<3.0.0,>=2.13.0 in \/usr\/local\/lib\/python3.10\/dist-packages (from rich>=13.3.2->comet_ml) (2.16.1)\\n\",\n            \"Requirement already satisfied: mdurl~=0.1 in \/usr\/local\/lib\/python3.10\/dist-packages (from markdown-it-py>=2.2.0->rich>=13.3.2->comet_ml) (0.1.2)\\n\"\n          ]\n        }\n      ]\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [\n        \"import comet_ml\\n\",\n        \"import pandas as pd\\n\",\n        \"from comet_ml import Artifact, Experiment\\n\",\n        \"from skllm.preprocessing import GPTSummarizer\\n\",\n        \"\\n\",\n        \"# Load your CSV file into a DataFrame\\n\",\n        \"df = pd.read_csv('\/content\/reviews_data.csv')\\n\",\n        \"\\n\",\n        \"# Select a subset of reviews (e.g., the first 50)\\n\",\n        \"X = df[\\\"Review\\\"].values[:50]\\n\",\n        \"\\n\",\n        \"# Initialize the GPTSummarizer\\n\",\n        \"reviews_summarizer = GPTSummarizer(openai_model=\\\"gpt-3.5-turbo\\\", max_words=10)\\n\",\n        \"\\n\",\n        \"# Generate summaries with your model\\n\",\n        \"generated_reviews = reviews_summarizer.fit_transform(X)\\n\"\n      ],\n      \"metadata\": {\n        \"colab\": {\n          \"base_uri\": \"https:\/\/localhost:8080\/\"\n        },\n        \"id\": \"B14j45F2zJJm\",\n        \"outputId\": \"80d72ab1-2ddc-42a4-baf1-316a6b41375b\"\n      },\n      \"execution_count\": 12,\n      \"outputs\": [\n        {\n          \"output_type\": \"stream\",\n          \"name\": \"stderr\",\n          \"text\": [\n            \"100%|\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 50\/50 [00:47<00:00,  1.05it\/s]\\n\"\n          ]\n        }\n      ]\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [\n        \"print(generated_reviews)\"\n      ],\n      \"metadata\": {\n        \"id\": \"Gxah5ShjJbYu\",\n        \"colab\": {\n          \"base_uri\": \"https:\/\/localhost:8080\/\"\n        },\n        \"outputId\": \"c19a2ba4-7c62-4192-ed23-a28569bedeb8\"\n      },\n      \"execution_count\": 18,\n      \"outputs\": [\n        {\n          \"output_type\": \"stream\",\n          \"name\": \"stdout\",\n          \"text\": [\n            \"['Starbucks employees at Southwest Parkway are friendly and efficient.'\\n\",\n            \" 'Starbucks barista in Altamonte Springs made perfect drink, great customer service.'\\n\",\n            \" 'Starbucks employee Billy made order quickly and with a smile.'\\n\",\n            \" 'Kind worker at Starbucks paid for drinks when card failed.'\\n\",\n            \" 'Drinking 5 cups of warm water, Starbucks gives free hot water.'\\n\",\n            \" 'Order mistakes, uncomfortable encounter, possible racism against stepmom.'\\n\",\n            \" 'Starbucks coffee tastes like dirt from a garbage dump.'\\n\",\n            \" 'Starbucks launched new fall items but quickly ran out.'\\n\",\n            \" 'Ordered wrong drink size, upset about paying $5 for it.'\\n\",\n            \" 'Unpleasant experience at Smythe St. Superstore, considering selling elsewhere.'\\n\",\n            \" 'Starbucks canceled my reward cards, played games, and avoided responding.'\\n\",\n            \" 'Starbucks often runs out of strawberries for Strawberry Refreshers.'\\n\",\n            \" 'Customer service insulted me, closed account due to low quality.'\\n\",\n            \" 'Starbucks failed to honor birthday reward, causing frustration for customer.'\\n\",\n            \" 'No purchase credit or stars received despite reporting the issue.'\\n\",\n            \" \\\"Mother sent $15 gift card, Walgreens won't replace, Starbucks unhelpful.\\\"\\n\",\n            \" 'Coffee spilled due to faulty lid, causing damage and inconvenience.'\\n\",\n            \" \\\"Long wait time and understaffed, won't return to this location.\\\"\\n\",\n            \" \\\"Starbucks ignores customer's request for assistance with rewards points.\\\"\\n\",\n            \" 'Starbucks Como Lake: bad service, long wait, confused orders.'\\n\",\n            \" 'Starbucks sold disappointing breakfast sandwiches that did not match the picture.'\\n\",\n            \" 'Unhappy customer wants refund for overcharged Starbucks drink.'\\n\",\n            \" 'Received invalid Starbucks gift cards twice, suspecting a scam.'\\n\",\n            \" 'Starbucks fails to address lid issue despite customer complaints.'\\n\",\n            \" \\\"Starbucks' Covid restrictions and coffee policy are criticized.\\\"\\n\",\n            \" 'Bad service and bad coffee at town east Starbucks.'\\n\",\n            \" 'Starbucks has small tables, making it uncomfortable for customers.'\\n\",\n            \" 'Chemical odors made me gag, Starbucks spelled my name wrong.'\\n\",\n            \" \\\"Starbucks' unhealthy drinks and increased loyalty points made me quit.\\\"\\n\",\n            \" 'Starbucks served a drink with a cleaning rag in it.'\\n\",\n            \" 'Starbucks refuses to resolve account blockage and refund substantial balance.'\\n\",\n            \" 'Starbucks is ungrateful to employees, rewards program is unfair.'\\n\",\n            \" 'Starbucks increases stars needed for free drink, prompting dissatisfaction.'\\n\",\n            \" 'Starbucks provided poor customer service, resulting in a 10 cent refund.'\\n\",\n            \" 'Starbucks inside Food City has poor logistics and limited options.'\\n\",\n            \" 'Coffee not good, many other options available.'\\n\",\n            \" \\\"Company's deceitful ways, weak coffee, pressured tipping, lost customer.\\\"\\n\",\n            \" 'Lakeport Commons in Sioux City always out of items, unreliable mobile ordering.'\\n\",\n            \" 'Starbucks app failed to refund money for broken drink.'\\n\",\n            \" \\\"Company doesn't help customers with mobile account issues, poor customer service.\\\"\\n\",\n            \" 'Person has a bad experience at Starbucks drive-thru in Portland.'\\n\",\n            \" 'Bad service at Starbucks with an old man serving coffee.'\\n\",\n            \" 'Starbucks closed outlets, lost loyal customer after 20 years.'\\n\",\n            \" 'Starbucks ad for free coffee on Veterans Day was a lie.'\\n\",\n            \" \\\"Starbucks refuses diabetic's request for unsweetened beverage during holidays.\\\"\\n\",\n            \" 'Starbucks refused free coffee to veterans on Veterans Day.'\\n\",\n            \" 'Frequent lid issues at Green Bay Starbucks, spilled drink on floor.'\\n\",\n            \" 'Unprofessional and rude manager at Tilton Starbucks, unacceptable customer service.'\\n\",\n            \" 'Starbucks on Sunset is great, but not the new one on Elms Springs.'\\n\",\n            \" 'Starbucks removes electrical outlets, angers loyal customers.']\\n\"\n          ]\n        }\n      ]\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [\n        \"# Join the generated reviews into a single string\\n\",\n        \"reviews_text = \\\"\\\\n\\\".join(generated_reviews)\\n\",\n        \"\\n\",\n        \"from nltk.translate.bleu_score import corpus_bleu\\n\",\n        \"\\n\",\n        \"bleu_score = corpus_bleu([[summary.split()] for summary in generated_reviews], [summary.split() for summary in generated_reviews])\"\n      ],\n      \"metadata\": {\n        \"id\": \"UqUzgppRB3xr\"\n      },\n      \"execution_count\": 21,\n      \"outputs\": []\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [\n        \"# Initialize Comet\\n\",\n        \"experiment = comet_ml.Experiment(\\n\",\n        \"    project_name=\\\"Text Summarization\\\",\\n\",\n        \"    workspace=\\\"bennykillua\\\",\\n\",\n        \"    api_key=\\\"YOUR KEY\\\",\\n\",\n        \"    auto_metric_logging=True,\\n\",\n        \"    auto_param_logging=True,\\n\",\n        \"    auto_histogram_weight_logging=True,\\n\",\n        \"    auto_histogram_gradient_logging=True,\\n\",\n        \"    auto_histogram_activation_logging=True,\\n\",\n        \"    log_code=True\\n\",\n        \")\\n\",\n        \"\\n\",\n        \"# Log the generated reviews as text\\n\",\n        \"experiment.log_text(\\\"Generated Reviews\\\", reviews_text)\\n\",\n        \"\\n\",\n        \"comet_ml.login(api_key=\\\"YOUR KEY\\\")\\n\",\n        \"\\n\",\n        \"# Initialize an Artifact\\n\",\n        \"artifact = Artifact(name=\\\"Reviews\\\", artifact_type=\\\"dataset\\\")\\n\",\n        \"\\n\",\n        \"# Specify the path of the artifact (the dataset file)\\n\",\n        \"artifact.add(r\\\"\/content\/reviews_data.csv\\\")\\n\",\n        \"\\n\",\n        \"# Log the artifact to the experiment (Comet platform)\\n\",\n        \"experiment.log_artifact(artifact)\\n\",\n        \"\\n\",\n        \"# Log the BLEU score\\n\",\n        \"experiment.log_metric(\\\"BLEU Score\\\", bleu_score)\\n\",\n        \"\\n\",\n        \"# Log the model\\n\",\n        \"experiment.log_model(reviews_summarizer, 'model')\\n\",\n        \"\\n\",\n        \"# End the experiment\\n\",\n        \"experiment.end()\"\n      ],\n      \"metadata\": {\n        \"id\": \"FS8aVQ3VJ2fB\"\n      },\n      \"execution_count\": null,\n      \"outputs\": []\n    },\n    {\n      \"cell_type\": \"markdown\",\n      \"source\": [\n        \"# You can view the logged model on the [Comet platform](https:\/\/www.comet.com\/bennykillua\/text-summarization\/view\/new\/panels)\"\n      ],\n      \"metadata\": {\n        \"id\": \"RF_-2K-H1XWx\"\n      }\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [],\n      \"metadata\": {\n        \"id\": \"Y0rrgFRLLYaj\"\n      },\n      \"execution_count\": null,\n      \"outputs\": []\n    }\n  ]\n}<\/pre>\n\n\n\n
Conclusion<\/h3>\n\n\n\n
Integrating Large Language Models with scikit-learn through the SKLLM library allows us to leverage advanced language understanding for various machine learning algorithms. Furthermore, by leveraging Comet, you have a user-friendly interface to track and optimize your hyperparameters and collaborate with other data scientists.<\/p>\n\n\n\n
P.S.<\/strong> If you prefer to learn by code, check out this Github gist<\/a>, which hosts the code snippets. Also, do check out the logged model on the Comet platform<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"
  Large Language Models (LLMs) are pretty remarkable. These powerful machine-learning models can understand and generate human-like text, creating a natural conversation experience. But they\u2019re not just for chatting\u200a\u2014\u200athey can also be used to deliver information, generate information like text summarizations, and get valuable suggestions. Today, we will talk about ScikitLLM or Scikit Large Language […]<\/p>\n","protected":false},"author":90,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"customer_name":"","customer_description":"","customer_industry":"","customer_technologies":"","customer_logo":"","footnotes":""},"categories":[65,9,7],"tags":[],"coauthors":[187],"class_list":["post-9410","post","type-post","status-publish","format-standard","hentry","category-llmops","category-product","category-tutorials"],"yoast_head":"\nText Summarization Modeling with ScikitLLM and\u00a0Comet<\/title>\n<meta name=\"description\" content=\"Dive into ScikitLLM and explore how you can use it to build text summarization ML models and monitor them all in Comet.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.comet.com\/site\/blog\/text-summarization-modeling-with-scikitllm-and-comet\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Text Summarization Modeling with ScikitLLM and\u00a0Comet\" \/>\n<meta property=\"og:description\" content=\"Dive into ScikitLLM and explore how you can use it to build text summarization ML models and monitor them all in Comet.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.comet.com\/site\/blog\/text-summarization-modeling-with-scikitllm-and-comet\" \/>\n<meta property=\"og:site_name\" content=\"Comet\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cometdotml\" \/>\n<meta property=\"article:published_time\" content=\"2024-03-06T14:00:12+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-04-24T17:03:01+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/cdn-images-1.medium.com\/max\/800\/1*M_vKJM1CnGSjt-WBa62akg.jpeg\" \/>\n<meta name=\"author\" content=\"Benny Ifeanyi Iheagwara\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@Cometml\" \/>\n<meta name=\"twitter:site\" content=\"@Cometml\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Benny Ifeanyi Iheagwara\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"17 minutes\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Text Summarization Modeling with ScikitLLM and\u00a0Comet","description":"Dive into ScikitLLM and explore how you can use it to build text summarization ML models and monitor them all in Comet.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.comet.com\/site\/blog\/text-summarization-modeling-with-scikitllm-and-comet","og_locale":"en_US","og_type":"article","og_title":"Text Summarization Modeling with ScikitLLM and\u00a0Comet","og_description":"Dive into ScikitLLM and explore how you can use it to build text summarization ML models and monitor them all in Comet.","og_url":"https:\/\/www.comet.com\/site\/blog\/text-summarization-modeling-with-scikitllm-and-comet","og_site_name":"Comet","article_publisher":"https:\/\/www.facebook.com\/cometdotml","article_published_time":"2024-03-06T14:00:12+00:00","article_modified_time":"2025-04-24T17:03:01+00:00","og_image":[{"url":"https:\/\/cdn-images-1.medium.com\/max\/800\/1*M_vKJM1CnGSjt-WBa62akg.jpeg","type":"","width":"","height":""}],"author":"Benny Ifeanyi Iheagwara","twitter_card":"summary_large_image","twitter_creator":"@Cometml","twitter_site":"@Cometml","twitter_misc":{"Written by":"Benny Ifeanyi Iheagwara","Est. reading time":"17 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.comet.com\/site\/blog\/text-summarization-modeling-with-scikitllm-and-comet#article","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/blog\/text-summarization-modeling-with-scikitllm-and-comet\/"},"author":{"name":"Benny Ifeanyi Iheagwara","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/71e200e652e19c10311a4dacf3891cb1"},"headline":"Text Summarization Modeling with ScikitLLM and\u00a0Comet","datePublished":"2024-03-06T14:00:12+00:00","dateModified":"2025-04-24T17:03:01+00:00","mainEntityOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/text-summarization-modeling-with-scikitllm-and-comet\/"},"wordCount":786,"publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/text-summarization-modeling-with-scikitllm-and-comet#primaryimage"},"thumbnailUrl":"https:\/\/cdn-images-1.medium.com\/max\/800\/1*M_vKJM1CnGSjt-WBa62akg.jpeg","articleSection":["LLMOps","Product","Tutorials"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.comet.com\/site\/blog\/text-summarization-modeling-with-scikitllm-and-comet\/","url":"https:\/\/www.comet.com\/site\/blog\/text-summarization-modeling-with-scikitllm-and-comet","name":"Text Summarization Modeling with ScikitLLM and\u00a0Comet","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/text-summarization-modeling-with-scikitllm-and-comet#primaryimage"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/text-summarization-modeling-with-scikitllm-and-comet#primaryimage"},"thumbnailUrl":"https:\/\/cdn-images-1.medium.com\/max\/800\/1*M_vKJM1CnGSjt-WBa62akg.jpeg","datePublished":"2024-03-06T14:00:12+00:00","dateModified":"2025-04-24T17:03:01+00:00","description":"Dive into ScikitLLM and explore how you can use it to build text summarization ML models and monitor them all in Comet.","breadcrumb":{"@id":"https:\/\/www.comet.com\/site\/blog\/text-summarization-modeling-with-scikitllm-and-comet#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.comet.com\/site\/blog\/text-summarization-modeling-with-scikitllm-and-comet"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/blog\/text-summarization-modeling-with-scikitllm-and-comet#primaryimage","url":"https:\/\/cdn-images-1.medium.com\/max\/800\/1*M_vKJM1CnGSjt-WBa62akg.jpeg","contentUrl":"https:\/\/cdn-images-1.medium.com\/max\/800\/1*M_vKJM1CnGSjt-WBa62akg.jpeg"},{"@type":"BreadcrumbList","@id":"https:\/\/www.comet.com\/site\/blog\/text-summarization-modeling-with-scikitllm-and-comet#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.comet.com\/site\/"},{"@type":"ListItem","position":2,"name":"Text Summarization Modeling with ScikitLLM and\u00a0Comet"}]},{"@type":"WebSite","@id":"https:\/\/www.comet.com\/site\/#website","url":"https:\/\/www.comet.com\/site\/","name":"Comet","description":"Build Better Models Faster","publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.comet.com\/site\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.comet.com\/site\/#organization","name":"Comet ML, Inc.","alternateName":"Comet","url":"https:\/\/www.comet.com\/site\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","width":310,"height":310,"caption":"Comet ML, Inc."},"image":{"@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/cometdotml","https:\/\/x.com\/Cometml","https:\/\/www.youtube.com\/channel\/UCmN63HKvfXSCS-UwVwmK8Hw"]},{"@type":"Person","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/71e200e652e19c10311a4dacf3891cb1","name":"Benny Ifeanyi Iheagwara","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/image\/f1d1f429361deaae05413115cd5914d4","url":"https:\/\/secure.gravatar.com\/avatar\/0b222798a274a8dee5e2394af804c5e76ba2fa8e5708ef6a35c5549a7d2794f2?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/0b222798a274a8dee5e2394af804c5e76ba2fa8e5708ef6a35c5549a7d2794f2?s=96&d=mm&r=g","caption":"Benny Ifeanyi Iheagwara"},"url":"https:\/\/www.comet.com\/site\/blog\/author\/iheifeanyigmail-com\/"}]}},"_links":{"self":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/9410","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/users\/90"}],"replies":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/comments?post=9410"}],"version-history":[{"count":1,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/9410\/revisions"}],"predecessor-version":[{"id":15377,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/9410\/revisions\/15377"}],"wp:attachment":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media?parent=9410"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/categories?post=9410"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/tags?post=9410"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/coauthors?post=9410"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
You can view the logged model on the Comet platform<\/a>.<\/p>\n\n\n\n