Integrate a Text Summarizer to your App to Analyze User Reviews
TL;DR:
- Use LLMs to summarize user reviews and analyze their overall sentiment.
This article's title and TL;DR have been generated with Cohere.
Get started with text generation.
Natural language processing (NLP) is invaluable, whether you're moderating member-posted content, quickly summarizing news articles, or providing auto-completion and auto-correction capabilities. In Python, NLP is mainly carried out using the Natural Language Toolkit (NLTK), which helps developers easily tap into Python's NLP capabilities.
Cohere provides a Python software development kit (SDK) that you can add to your Python project with a simple pip install command. You can use Cohere's NLP capabilities, like summarization and classification, for a wide range of use cases, from eCommerce systems and social media platforms to team communication and news apps.
In this tutorial, you'll use Cohere's Python SDK to add NLP functionality to an existing Python application. Then, your app will use Cohere's text summarizer endpoint to automatically summarize user reviews and analyze their overall sentiment and toxicity levels.
You'll start by setting the sentiment of a review manually, after which the Cohere Platform will extract the first sentence and use it as the summary. Cohere will learn from this example and begin to classify text automatically, reducing the need for your input. Finally, Cohere will automatically analyze the overall sentiment of each review, determine whether or not it is suitable for saving based on toxicity, and then write a summary.
Prerequisites
Before you begin, ensure that you have:
- Working knowledge of Python and an up-to-date installation
- Working knowledge of the Django framework and an up-to-date installation
- Working knowledge of HTML and CSS
- A Cohere account to access an API key
See the full project code on GitHub.
The Initial Django App
First, let's look at the initial Django app (see full build instructions).
This is the home screen.
Under each product, there's a button that, when clicked, takes the user to a page where they can view the product's reviews and add a new review.
When the user clicks the Add a review button, the system displays the following page.
There is a radio button for each sentiment and a form element where the user enters their review.
The following image shows the project's directory structure. Make a note of the directory's file names and locations.
Adding the Views
You can locate the app's routes in the demoshop/urls.py. After firing, they render the respective views:
from django.urls import path
from . import views
urlpatterns = [
path('', views.index, name='index'),
path('product/<prod_id>/', views.product),
path('<prod_id>/add', views.add_review),
path('process', views.process_request),
]
The routes are registered in the coherenlpreviewsdemo/urls.py file in the urlpatterns array using this line.
path('reviews/', include('demoshop.urls')),
This denotes that our app will be running at localhost:8000/reviews address.
Views are added in demoshop/views.py.
from django.shortcuts import render
from .models import review
def index(request):
return render(request, 'demoshop/index.html')
def product(request, prod_id):
all_reviews = review.objects.filter(product_id=prod_id).order_by('-id')
image_to_display = ''
if (prod_id == "0001"):
image_to_display = 'images/daniel-korpai-hbTKIbuMmBI-unsplash.jpg'
elif (prod_id == "0002"):
image_to_display = 'images/alvan-nee-vhvH46ASxH0-unsplash.jpg'
elif (prod_id == "0003"):
image_to_display = 'images/ivan-shilov-KiAYZZjpjkQ-unsplash.jpg'
elif (prod_id == "0004"):
image_to_display = 'images/david-svihovec-HM-Y497t5CU-unsplash.jpg'
elif (prod_id == "0005"):
image_to_display = 'images/kiran-ck-UkO7K1CPFS8-unsplash.jpg'
elif (prod_id == "0006"):
image_to_display = 'images/daniel-korpai-hbTKIbuMmBI-unsplash.jpg'
context = {'all_reviews': all_reviews, 'prod_id': prod_id, 'prod_img': image_to_display}
return render(request, 'demoshop/product_page.html', context)
def add_review(request, prod_id):
context = {"prod_id": prod_id}
return render(request, 'demoshop/review_form.html', context)
def process_request(request):
review_text = request.POST['review'].strip()
sentiment = request.POST['sentiment'].strip()
prod_id = request.POST['prod_id'].strip()
summary = review_text.split(".")[0].strip()
rev = review.create(prod_id, sentiment, summary, review_text)
rev.save()
return render(request, 'demoshop/index.html')
The index method above displays the home page, product displays the product page, and add_review displays the review entry form. The last method, process_request, processes the POST request by tapping the user's form entries and removing trailing spaces using the strip() method.
The if statement in the product function ensures the appropriate image is rendered based on the product's ID as passed in from the route.
Adding the Model
The code In the demoshop/models.py file is used to add the review model. It will store a review using five fields: the summary (rev_summary), the reviewed product's ID (product_id), sentiment (rev_sentiment), full review (full_review), and the review id, which Django autogenerates. In the review class, the create method creates an instance of the class. This comes in handy when saving data to the database.
from django.db import models
class review(models.Model):
product_id = models.CharField(max_length=200)
rev_sentiment = models.CharField(max_length=8)
rev_summary = models.TextField(default="some summary", null=False)
full_review = models.TextField(default="some review summary", null=False)
@classmethod
def create(cls, prod_id, sentiment, summary, review_text):
review_instance = cls(product_id=prod_id,
rev_sentiment=sentiment,
rev_summary=summary,
full_review=review_text)
return review_instance
The images are stored in the demoshop/static/images in agreement with Django's static files semantics.
For the UI, use HTML files in the demoshop/templates/demoshop folder. Bootstrap 5 is used for styling. The files are for building user interfaces, which you can create or modify as preferred.
Those are the essential files for the project. Make the necessary migrations and try running the app using the python manage.py runserver command.
Adding Cohere to the Project
In this final section, you'll add Cohere's NLP toolkit or dependency to the project. The project will first determine whether or not a review is appropriate to save to the database by using the classify class to check for toxic language. After the toxicity check, it will determine the sentiment using the classify class again. Lastly, it will summarize the review using the generate class.
Install Cohere using the command below.
pip install cohere
Then, open the demoshop/views.py file and import the required modules.
import cohere
from cohere.classify import Example
api_key = '' #Enter your API key here
co = cohere.Client(api_key)
Next, update the process_request method to reflect this.
def process_request(request):
review_text = request.POST['review'].strip()
prod_id = request.POST['prod_id'].strip()
toxicity_response = co.classify(
model='large',
inputs=[review_text],
examples=[Example("Ugliest delivery guys", "Toxic"), Example("What a poor company", "Toxic"),
Example("I will give my review later", "Non-Toxic"),
Example("You are a fantastic company", "Non-Toxic"), Example("Scammer watches", "Toxic"),
Example("WHAT A BUNCH OF JERKS", "Toxic"), Example("I love that!", "Non-Toxic"),
Example("Very expensive but it\'s a good one", "Non-Toxic"), Example("Beautiful", "Non-Toxic"),
Example("Is this company haunted by ghosts?", "Toxic")])
toxicity_classification = format(toxicity_response.classifications[0].prediction)
sentiment_response = co.classify(
model='large',
inputs=[review_text],
examples=[Example("Great for beginners or more experienced users. Bought as a gift and she loves it", "Positive"),
Example(
"This product so far has not disappointed. My children love to use it and I like being able to monitor what content they see with ease.",
"Positive"), Example(
"Didn't have some of the features I was looking for. Returned it the next day. May be good for others",
"Negative"), Example(
"I bought this around Black Friday for that amount hoping it would be awesome... it failed so hard. i tried multiple different micro SD cards none of which were recognized and YES i formatted them with every format i could think of ... Fat32, NTFS, Fat, Xfat",
"Negative"),
Example("Totally worth the money. It has all the apps I use on a daily basis!", "Positive"), Example(
"Functionality and ease of use are outstanding right out of the box. Super clear screen and weight is minimal. A joy to use.",
"Positive"), Example(
"The price is low because AmShop uses their ads to subsidize the price. The actual watch works well. The biggest negative is that you can only install apps from the AmShop store.",
"Neutral"),
Example("Every time I get home, my brother shows me something new on the watch. I am jealous!",
"Neutral"), Example("I’ll give a review later on.", "Neutral"),
Example("Good sound quality", "Positive"), Example("Bad sound quality", "Negative"),
Example("A nice watch", "Positive")])
sentiment_classification = format(sentiment_response.classifications[0].prediction)
summary_response = co.generate(
model='large',
prompt='Review: This inexpensive tablet does everything I want it to do. I can read my books and magazines and keep tabs on my kids and grandkids on Facebook. Very easy to use and crystal clear pictures and video.\n\nSummary: Inexpensive but delivers.\n--\nReview: Great entry-level tablet for kids or anyone that will not be heavily using a tablet. The 16 GB one is too small but can add an SD card. Beware only comes with 6 GB of the 16 for your apps. Comes with a bunch of preloaded Amazon apps.\n\nSummary: Great for kids but has a smaller space.\n--\nReview: I was looking for something to read on and this fit the bill for the right price. Great screen clarity, good adjustment too. Works well and the battery life is great. Size is nice too. Big enough for 1 hand if needed.\n\nSummary: Nice for reading, good battery life and well-sized.\n--\n\nReview: {}\n\nSummary:'.format(review_text),
max_tokens=50,
temperature=0.8,
k=0,
p=1,
frequency_penalty=0,
presence_penalty=0,
stop_sequences=["--"],
return_likelihoods='NONE')
summary_generated = format(summary_response.generations[0].text)
if (toxicity_classification == "Non-Toxic"):
sentiment = sentiment_classification
summary = summary_generated
rev = review.create(prod_id, sentiment, summary, review_text)
rev.save()
else:
print("Too vague")
In the classify methods, set the user's entry in the prompt and set the model type and the test values in the examples array. The examples array takes in the test values in the Example class constructor. The generate class receives several parameters, but let's focus on prompt.
This parameter differs from classify as the value to be checked is added in the prompt and not an inputs array. Part of the test values for the generate class were taken from this Kaggle dataset. Using the generate class, Cohere does the summarization using the examples given. It learns the trends from the examples given and suitably summarizes a new prompt. Learn more about summarization in this Cohere guide.
Modify the demoshop/templates/demoshop/review_form.html by removing the snippet below to remove the radio buttons.
<label class="form-label" >Sentiment</label>
<div class="d-flex justify-content-center">
<div class="form-check">
<input style="margin: 3%"
class="form-check-input"
type="radio"
name="sentiment"
checked
value="Positive"
/>
<label class="form-check-label">
Positive
</label>
</div>
<div class="form-check">
<input style="margin: 3%"
class="form-check-input"
type="radio"
name="sentiment"
value="Negative"
/>
<label class="form-check-label">
Negative
</label>
</div>
<div class="form-check">
<input style="margin: 3%"
class="form-check-input"
type="radio"
name="sentiment"
value="Neutral"
/>
<label class="form-check-label">
Neutral
</label>
</div>
</div>
To make the application more accurate, feed in more data and try adjusting the parameters passed into the classes, especially generate.
Here is a working demo GIF.
Final Thoughts
NLP has countless applications, and the Cohere platform makes it easier to add NLP capabilities to your products. You only need an account and a test API Key. As the article demonstrates, you can easily add NLP to your Django applications using Cohere, and use the text summarizer endpoint to analyze user reviews as shown.
When you're ready, learn more about working with NLP in Python with Cohere.