If you are a developer looking to add language processing capabilities to your project, this guide is for you. We’ll give you a quick tour of what’s possible with language AI using Cohere’s Large Language Model (LLM) API. Our guide is essentially your “Hello, World!” of language AI, and since this is all about language processing, we’ll do this by exploring the phrase Hello, World! itself!
We’ll cover three groups of tasks that you will typically work on when dealing with language data, including:
- Generating text
- Classifying text
- Analyzing text
These map nicely to three Cohere endpoints — Generate, Classify, and Embed — which we will cover in the examples. We’ll cover the first one in Part 1 and the other two in Part 2. At the end of Part 2, we’ll summarize these endpoints in a table for easy reference.
This guide assumes little or no background in machine learning or NLP. The examples are shown using Python, although Cohere also provides SDKs for Node.js, Go, and Ruby. We include only key snippets of the code used, but you can run the full version if you wish. It is hosted on Google Colaboratory, so you won’t need to go through additional setup or installation.
Before we proceed, let’s use a toolbox metaphor to try and make sense of the commonly used terms and how they compare to each other:
- Natural Language Processing (NLP): The toolkit for using computers to process, understand, and generate text.
- Language AI: A set of modern tools within that toolkit which leverage machine learning models. Some examples include text generation, classification, summarization, paraphrasing, and entity extraction.
- Large Language Models (LLM): A general-purpose type of language AI that eliminates the need for different models for different tasks. It has also achieved state-of-the-art performance in some NLP tasks. These are the kind of models you interact with using the Cohere platform.
The first step is to install the Cohere Python SDK.
$ pip install cohere
Next, sign up for a Cohere account and create an API key, which you can generate from one of the following interfaces:
Once that is done, you can set up the Cohere client as follows.
import cohere co = cohere.Client(api_key)
1 - Generating text
There are two broad themes in language processing — language generation and language understanding. This part covers the former, which is the capability that you can access by calling Cohere’s Generate endpoint.
Cohere’s language AI endpoints produce different types of outputs, but there is a similar pattern in how you work with them. This can be broken into three steps:
- Prepare input
- Define model settings
- Get output
The Cohere Generate endpoint generates text given an input, called a “prompt.”
The prompt provides a context for the text that we want the model to generate. To illustrate this, let’s start with a simple prompt as the input.
prompt = "Hello World is a program that"
Define model settings
Next, we’ll define some model settings. There are a number of settings that you can modify with the Generate endpoint, but we’ll start with four:
model— the model size, which ranges from
xlarge. Generally smaller models are faster, while larger models will perform better. We’ll use the default which is
prompt— the input string to be used.
max_tokens— the number of tokens to be generated. Translating tokens into words, one word contains approximately 2-3 tokens.
temperature— a number between 0 and 5. Lower temperature will cause the model to output text that is more predictable, while higher temperature means that the output will be more creative. There is no one right setting as it depends on your task and it requires some experimentation. In most cases, somewhere between 0 and 1 works fine, and for our case, we’ll use 0.4.
Finally, we’ll generate the output, giving us the
generations response. Putting everything together, we get the code below.
prompt = "Hello World is a program that" response = co.generate( model='xlarge', prompt=prompt, max_tokens=75, temperature=0.4) output = response.generations.text
Here’s a sample output returned:
prints "Hello World" to the screen. To write a Hello World program, you need to create a new file and save it as HelloWorld.py. Then, you can write the following code into the file: print("Hello World!") Save the file and run it using Python. If you don't have Python installed,
The output is not bad, but it could be better. If we were writing a blog post for example, the tone of the generated output wouldn’t fit very well. Also, there is no natural ending to the generated text, which looks like it could continue for quite some time.
We need to find a way to make the output tighter and closer to how we want it to be, which is where we leverage prompt engineering.
Create a better prompt
Prompt engineering is a fascinating topic. It is about figuring out the optimal way to prompt a model for a particular task, so we can shape the output to be how we want it to be. There are many creative prompts that have proven effective out there, and the whole topic is an active area of research. Having said that, the basic format that generally works well is as follows:
- A short description about the overall context
- A few examples of prompts and completions; usually two to three examples are sufficient but for more challenging tasks, you will need more
- A short sequence of characters or “stop sequence” to give the model a hint to create a complete passage and then stop
Let’s say we are writing a blog about Hello, World! Specifically, we want to write an introductory paragraph about Learning to Code with Hello, World!
For this, we can create a prompt consisting of:
- A short description about what this “program” is about
- A couple of examples of the blog title and its first paragraph
- A stop sequence, which we use “--”
The prompt is as follows:
prompt = """ This program will generate the first paragraph of a blog post given a blog title. -- Blog Title: Best Activities in Toronto First Paragraph: Looking for fun things to do in Toronto? When it comes to exploring Canada's largest city, there's an ever-evolving set of activities to choose from. Whether you're looking to visit a local museum or sample the city's varied cuisine, there is plenty to fill any itinerary. In this blog post, I'll share some of my favorite recommendations -- Blog Title: Mastering Dynamic Programming First Paragraph: In this piece, we'll help you understand the fundamentals of dynamic programming, and when to apply this optimization technique. We'll break down bottom-up and top-down approaches to solve dynamic programming problems. -- Blog Title: Learning to Code with Hello, World! First Paragraph: """
Using the new prompt and adding stop sequence, our generation code now looks like this:
response = co.generate( model='xlarge', prompt=prompt, max_tokens=75, temperature=0.4, stop_sequences=["--"]) output = response.generations.text
Here’s a sample output returned:
Coding is a fun and exciting way to learn the basics of computer science. In this article, we'll review the fundamentals of programming, including variables, functions, conditional statements, and loops. We'll also discuss how to use Python to write code that prints "Hello, World!" --
This looks much better! Because we have created a couple of examples with the tone of a blog post, the model captured that context and was then able to generate something with a similar tone. There is also a natural ending to the piece, which you notice ends with the stop sequence.
Automate the process
In actual applications, you will likely need to produce these text generations on an ongoing basis, given different inputs. Let’s simulate that with our example.
First, we create a list of new topics, so we can iterate on them and get the paragraphs generated. We then make some tweaks to the earlier prompt: we create a base prompt containing the examples, and then we append it to the current prompt, which is the new topic. These steps are shown below.
The list of topics:
topics = ["How to Grow in Your Career", "What Makes a Great Software Developer", "Ideas for a Relaxing Weekend"]
The base prompt:
prompt = """ This program will generate the first paragraph of a blog post given a blog title. -- Blog Title: Best Activities in Toronto First Paragraph: Looking for fun things to do in Toronto? When it comes to exploring Canada's largest city, there's an ever-evolving set of activities to choose from. Whether you're looking to visit a local museum or sample the city's varied cuisine, there is plenty to fill any itinerary. In this blog post, I'll share some of my favorite recommendations -- Blog Title: Mastering Dynamic Programming First Paragraph: In this piece, we'll help you understand the fundamentals of dynamic programming, and when to apply this optimization technique. We'll break down bottom-up and top-down approaches to solve dynamic programming problems. -- Blog Title:"""
Set up the model:
def generate_text(base_prompt, current_prompt): response = co.generate( model='xlarge', prompt = base_prompt + current_prompt, max_tokens=75, temperature=0.4, stop_sequences=["--"]) generation = response.generations.text return generation
Iterate on the topics:
for topic in topics: current_prompt = " " + topic + "\n" + "First Paragraph:" para = generate_text(base_prompt, current_prompt)
And here’s a sample output returned:
Topic: How to Grow in Your Career First Paragraph: If you've been working in the same position for a while, you may be wondering how to grow in your career. In this article, we'll discuss how to advance your career and take your skills to the next level. We'll cover how to get promoted, how to get a raise, and how to find a new job. ---------- Topic: The Habits of Great Software Developers First Paragraph: What makes a great software developer? What separates them from the rest? What are the habits of great software developers? In this post, I will share with you the habits of great software developers. ---------- Topic: Ideas for a Relaxing Weekend First Paragraph: If you're looking for ideas for a relaxing weekend, there are plenty of options available. Whether you're hoping to spend some time with friends or family, or you're looking for a quiet weekend at home, you can find a relaxing weekend activity that fits your needs. Here are some ideas for a relaxing weekend. ----------
We have covered one example, but the ways to construct a prompt are only limited by your creativity. It is also highly dependent on the task at hand. Prompt engineering can be used not only in text completion, but also to do any form of text generation, for example:
- Summarizing text
- Rewriting text
- Extracting key information from text
There are more model settings that we have not covered, some of which are worth exploring:
num_generations— instead of generating one output per call, you can generate up to a maximum of 5
return_likelihoods— enable this to also receive the likelihood of the tokens generated
- and more, which you can find in the API reference
We have taken a quick tour of text generation, but there is so much more to explore. Here are some additional resources:
- A guide to prompt engineering
- Controlling generation outputs
- Some use case ideas with text generation
- The Generate API reference
In Part 2, we’ll complete our tour by looking at how we can use Language AI to classify and analyze text.In the meantime, to try out the Generate endpoint for yourself, sign up for a Cohere account now.