Create a ChatGPT Powered Web App to Chat With Historical Philosophers, Part 1: OpenAI API and Prompt Engineering
Introduction
This is Part 1 of a 2 part tutorial on creating an app that leverages the ChatGPT PythonAPI and deploying it to the web using Streamlit.
This tutorial will guide you on creating a web app powered by ChatGPT, which enables users to chat with historical philosophers on any topic. The app selects a philosopher who has written about the question’s topic and responds from their perspective while providing references to their work.
Large Language Models (LLMs) like ChatGPT are versatile tools that can be applied to various tasks, from coding to chatting to formatting documents. However, their greatest strength lies in language understanding, be it English, Python, or SQL.
Philosophy is particularly suitable for LLMs because it revolves around language. It is a human attempt to articulate the world and utilize words to comprehend the enigmatic and bewildering aspects of our surroundings and experiences. For a more technical and mathematical take on this endeavor, we can turn to the field of “Data Science.”
In this tutorial, I will demonstrate how to utilize the OpenAI API, Streamlit, and effective prompt engineering to specialize an LLM for a specific task with minimal code. We will direct ChatGPT to research, summarize, and present the viewpoints of different philosophers on profound questions raised by users.
Requirements
- OpenAI API Key and Python Package: To get started, sign up for an OpenAI API key using this link. Note that this service is not free, so expect a small fee. During the development of my app, I spent $0.77 on API calls. OpenAI allows you to set usage caps, but I haven’t made my app public to avoid exceeding them. To install the OpenAI API Python package, use the command
pip install openaiin your terminal. - Python IDE: This tutorial employs Python, so you’ll need a Python environment to run the code. While it’s technically possible to replicate this and have a functional app using only GitHub and a Streamlit account (as Streamlit can read directly from GitHub and you can edit documents on the GitHub website), I recommend using a Python environment in a user-friendly IDE like VS Code. This way, you can test your app locally before deploying it to the web.
- Streamlit: Streamlit is a simplified Python package and web platform for building web applications. It makes it easy to insert text and widgets using an object-oriented approach. Install Streamlit into your Python environment with
pip install streamlitin your terminal. Once you've developed your app, you can run it locally from the terminal usingstreamlit run app.py.
What’s even more exciting is that after developing and pushing your app to a GitHub repository, the Streamlit website can create a live web-based app for you for free! I love that with Streamlit I can develop and test my app locally until it meets my requirements. Then, I simply push it to GitHub and deploy it with a few clicks on the Streamlit site. If I need to make changes, I can easily push the updates to the same repository. Streamlit automatically detects the changes and updates the live app. Development and troubleshooting are remarkably straightforward! - GitHub Account: While optional if you don’t plan to deploy your app to the web, I recommend having a GitHub account if you’re here for code tutorials. If you wish to deploy your app using the Streamlit site, you’ll need a GitHub account and a repository for your code.
The ChatGPT API
The code in this app is divided into two parts. “app.py” focuses on using Streamlit to create an interactive app that interacts with the model, while “AIAgent.py” contains the AIAgent class responsible for interacting with the ChatGPT model. This part of the tutorial will focus on the AIAgent.
OpenAI provides an excellent guide for using their API, which includes prompt engineering tips, new application ideas, and user-friendly API documentation. You can find their general guides, tips, and tricks here. However, to get started, we are interested in the actual API references found here.
In this demonstration, we only need two API functions: setting the secret API key and passing a prompt, or more specifically a list of prompts that represent the history of a conversation.
Cost
As mentioned earlier, you’ll need an API key from OpenAI to make queries to the model from your code. Obtaining the key is free, but usage incurs a fee. You’ll need to provide a credit card, and you’ll be charged a very small amount per ‘token’. At the time of writing, the cost is $0.003/1000 tokens to use GPT-3.5.
Tokens
But what is a token? It’s similar to a word but not exactly the same. Tokens are semantic word parts used by the model to parse text. OpenAI suggests that 1 token is roughly equivalent to 3/4 of a word. Therefore, a 1000-word prompt would consume approximately 1250 tokens.
Remember that you’ll be charged for tokens both in the prompts and the model’s reply.
Once you have your API key, you can add it to the API by assigning it to the openai.api_key attribute.
import openai
openai.api_key = <your key>If you’re committing your code to GitHub, ensure that your API key is NOT visible in your code or any files within your repository. Never make your API keys publicly viewable, as you’ve linked your credit card to them!
One approach to avoid exposing your API key is to set it as an environment variable. Here is a simple and straightforward guide on how to do that. Once your key is stored as an environment variable, you can access it in your code like this:
import os
import openai
openai.api_key = os.getenv("OPENAI_API_KEY")When you’re ready to deploy to Streamlit, you’ll need to add your key to the “Streamlit Secrets” for your app. I’ll explain how to do that in Part 2.
Completions and ChatCompletions
Why does OpenAI call questions to the model ‘completions’?
ChatGPT is, in fact, a text completion model sometimes referred to as ‘Autocomplete on Steroids.’ Large Language Models like ChatGPT are prediction engines that forecast the next token. They use your prompt as context and generate tokens in response one at a time. After each new token, the model evaluates the context prompt and the text generated so far, and predicts the next token. The model repeats this process until the <end> token is predicted. For an informative introduction to this topic, check out this Dataiku article by Kurt Muehmel.
Once you’ve added your API key to the openai.api_key attribute, you're ready to try a completion.
There are two classes to choose from when requesting completions: openai.Completion and openai.ChatCompletion.
But what’s the difference?
The Completion class is suitable for a single prompt without any follow-up. You only need to specify the model to access and the prompt itself.
response = openai.Completion.create(
model="gpt-3.5",
prompt="What is the meaning of life?"
)On the other hand, the ChatCompletion class can maintain a conversation by stringing together multiple prompts. The model 'remembers' the previous conversation and considers it as part of its context for each new prompt.
However, the ‘memory’ is not stored on OpenAI’s side; it’s your responsibility! You need to maintain a list of all the messages exchanged and provide the entire list to the API for each new prompt.
Note: You’ll be charged for each token in the conversation for both messages and responses in subsequent completions. For example, if your conversation spans five turns, you’ll be charged for the first prompt five times, once for each time it was sent back to the model as part of the conversation history.
While the Completion class requires only a single string prompt, the ChatCompletion class accepts a list of dictionaries using the messages argument.
response = openai.ChatCompletion.create(
model="gpt-3.5-turbo",
messages=[
{"role": "user", "content": "Hello!"},
{"role": "assistant", "content": "Hello! How can I assist you today?"},
{"role": "user", "content": "I'd like to know about the meaning of life."}
]
)Each dictionary in the messages list identifies the speaker and their content using the "role" and "content" keys, respectively. The "role" key indicates whether it's the user or the model speaking, while the "content" key holds the actual prompt or reply. This way, your code can maintain a local memory for the model.
It’s important to note that the actual model running on OpenAI servers does not remember your conversation. It lacks its own memory and is neither conscious nor self-aware. This distinction is crucial given our philosophical theme. The model’s memory is an illusion that you create for your users or, at best, an additional component to the actual model.
Both openai.Completion.create and openai.ChatCompletion.create offer additional arguments to fine-tune the responses or limit their size, but for a quick start, these are the only ones we need.
System Message and Prompt Engineering
The true magic behind the Chat with a Philosopher app lies in the system prompt and prompt engineering strategies.
The system prompt allows us to include a hidden “system” message as the first message, which primes the model to respond in a specific manner. It adds an element of mystery to the app, making it seem like magic. Let’s take a closer look at the system message that transforms ChatGPT into a philosopher:
system_message = """For each query, consider writings by philosophers that have
addressed that question and choose one.
Respond to the query from the point of view of that philosopher
Finally, use those referenced sources to form a response to the
query that the chosen philosopher would support.
Your response should be written from the point of view of that
philosopher and be sure your response is supported by original sources
of text authored by that philosopher.
You should reference those sources in your response.
For example:
Query: What is the meaning of life?
Response: Hi, my name is Albert Camus. In The Myth of Sysyphus,
I wrote about how life is meaningless from
an objective viewpoint, but that we can create our own meaning for our
lives. In my works of fiction, I describe situations where people
must make their own meaning in during times of crisis, doubt and
confusion.
Ultimately, life is what you make of it and it means what it means
to you. One of my most famous quotes is:
"The meaning of life is whatever you are doing that keeps you
from killing yourself."""
message = [{'role': 'system', 'content':system_message}]Implementing several prompt engineering strategies, the system message is the result of multiple iterations. It’s important to note that you may not achieve the perfect prompt on the first attempt. Continue tweaking it until you obtain the desired responses.
Let’s discuss the prompt engineering strategies employed:
- Explicitly guide the model’s thought process:
“For each query, consider writings by philosophers that have addressed that question and choose one”
- We want the model to conduct research and identify philosophers who have written about the given topic. This prevents the model from inventing information and attributing it to the wrong philosopher.
“Respond to the query from the point of view of that philosopher”
- This step adds an immersive experience as the model pretends to be the philosopher while responding.
“Finally, use those referenced sources to form a response to the query that the chosen philosopher would support”
- We emphasize the importance of grounding the response in references and writings that the philosopher has authored to help prevent ‘Hallucinations’, or incorrect information that the model has invented on its own.
2. Define the expected response:
“Your response should be written from the point of view of that philosopher and be sure your response is supported by original sources of text authored by that philosopher. You should reference those sources in your response.”
- We instruct the model to ensure that the response reflects the philosopher’s perspective and is supported by references to their original works.
3. Utilize few-shot learning by providing an example:
Query: “What is the meaning of life?”
Response: “Hi, my name is Albert Camus. In The Myth of Sysyphus, I wrote about how life is meaningless from an objective viewpoint, but that we can create our own meaning for our lives. In my works of fiction, I describe situations where people must make their own meaning during times of crisis, doubt, and confusion. Ultimately, life is what you make of it and it means what it means to you. One of my most famous quotes is: ‘The meaning of life is whatever you are doing that keeps you from killing yourself.’”
In this example, we not only tell the model what we expect but also show it through the provided response. By assuming the role of Albert Camus, we include textual references and even incorporate a quote.
Prompt engineering is a powerful technique that improves the model’s performance. By providing explicit instructions, assuming roles, and using references, we enhance the accuracy and quality of the responses.
Compare the above example response to an actual response the model provided:
Notice that while the model is responding to the same question as the example we gave it, it answers as a different philosopher with a different point of view.
The model:
- Starts with a greeting playing the role of Aristotle
- Immediately provides a text reference to Nicomachean Ethics and a summary of its relevant content.
- Provides a second reference with more information.
- Summarizes the response.
It doesn’t include a quote, but I also did not ask for that in my instructions outside of the example. If we wanted this to be more consistent, we could add the requirement explicitly before the example reponse.
Putting the Pieces Together and Sending the prompt
Now that we have our system prompt, it’s time to add a user prompt. The AIAgent class maintains a list of messages, acting as a 'memory' for the conversation. The system message is placed first and is not visible to the user. The remaining messages in the list alternate between the roles of "user" and "assistant".
When a query is made, the user prompt is added to the list. The entire conversation history is then sent to the model, and the model generates a response, which is subsequently added to the conversation history.
import os
import openai
# Set the OpenAI API key
openai.api_key = os.getenv("OPENAI_API_KEY")
class AIAgent():
def __init__(self, model="gpt-3.5-turbo"):
"""Initialize the AI Agent. Set the system message,
initial prompt prefix, and initial response"""
self.model=model
self.system_message = """For each query, consider writings by philosophers that have addressed that question and choose one.
Respond to the query from the point of view of that philosopher
Finally, use those referenced sources to form a response to the query that the chosen philosopher would support.
Your response should be written from the point of view of that philosopher and be sure your response is supported by original sources
of text authored by that philosopher. You should reference those sources in your response.
For example:
Query: What is the meaning of life?
Response: Hi, my name is Albert Camus. In The Myth of Sisyphus, I wrote about how life is meaningless from
an objective viewpoint, but that we can create our own meaning for our lives. In my works of fiction, I describe
situations where people must make their own meaning in during times of crisis, doubt and confusion.
Ultimately, life is what you make of it and it means what it means to you. One of my most famous quotes is:
"The meaning of life is whatever you are doing that keeps you from killing yourself."""
# Initialize the prompt prefix
self.prefix = ''
# Add system message to message history
self.history = [{'role': 'system', 'content':self.system_message}]
# Intialize pre-prompt output
self.response = 'Philosophers are waiting patiently, possibly smoking a cigar or pipe'
def add_message(self, text, role):
"""Add a message to the conversation history"""
message = {'role':role, 'content':text}
self.history.append(message)
def query(self, prompt, temperature=.1):
"""Query the remote model and add messages to the hisotry"""
# Add user prompt to history
self.add_message(self.prefix + prompt, 'user')
# Query the model through the API
result = openai.ChatCompletion.create(
model=self.model,
messages=self.history,
temperature=temperature, # this is the degree of randomness of the model's output
)
self.response = result.choices[0].message["content"]
# Add reply to message history
self.add_message(self.response, 'assistant')
# set prompt prefix to ensure same philosopher answers each time.
self.prefix = 'Please continue to respond as the previous philosopher: '
def clear_history(self):
"""Reset the history to its initial state.
Also reset the prefix and current response"""
self.history = [{'role':'system', 'content':self.system_message}]
self.prefix = ''
self.response = 'Philosophers are waiting patiently, possibly smoking a cigar or pipe'The response object contains a lot of information about the transaction, but we are interested in the actual content of the response:
self.response = result.choices[0].message["content"]This is what we want to show the user and add to the message history.
Prompt Wrappers
The last thing to notice is that prompts can be wrapped in additional instructions. In this case we add “Please continue to respond as the previous philosopher: ” to the beginning of each prompt after the first.
self.prefix = 'Please continue to respond as the previous philosopher: 'self.add_message(self.prefix + prompt, 'user')This ensures that the model is consistent and the user can continue to converse with the same philosopher until they start a new conversation.
Summary
In this section, you gained insights into using the OpenAI API to engage with the ChatGPT-3.5 model. You explored the Completion and ChatCompletion classes, understanding their distinctions and how to leverage conversation history in the form of a list of dictionaries when interacting with the model through the API.
Additionally, you discovered essential prompt engineering strategies aimed at ensuring the model produces the desired responses. By explicitly guiding the model’s thought process and defining the expected format of the responses, you can influence the model’s behavior and enhance the accuracy and relevance of its outputs.
In Part 2 you’ll learn how to transform your code into a web-app for others to interact with.
Links
Link to GitHub repository for this project
Link to Chat with a Philosopher Site
References:
OpenAI Documentation References: https://platform.openai.com/docs/introduction
