Skip to content

Instantly share code, notes, and snippets.

@caleb-kaiser

caleb-kaiser/1-prompt-engineering-overview-part2.ipynb

Last active January 2, 2024 22:17
Show Gist options
  • Save caleb-kaiser/d396b3f29c0845d02f96417dbc7260da to your computer and use it in GitHub Desktop.
Save caleb-kaiser/d396b3f29c0845d02f96417dbc7260da to your computer and use it in GitHub Desktop.
Download ZIP
1-prompt-engineering-overview-part2.ipynb
Display the source blob
Display the rendered blob
Raw
1-prompt-engineering-overview-part2.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "pJUgpXEq75iG"
},
"source": [
"# Overview of Prompt Engineering Techniques & Best Practices"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "IxBHRaGL75iJ"
},
"source": [
"## Part 2: Prompt Engineering Techniques\n",
"\n",
"In this section, we cover the best prompt engineering techniques and how to apply them."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "KUb6m_OT75iK"
},
"source": [
"---"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "U9m-55tj75iK"
},
"source": [
"We first load the necessary libraries:"
]
},
{
"cell_type": "code",
"source": [
"! pip install openai==0.28 langchain google-search-results chromadb tiktoken --quiet"
],
"metadata": {
"id": "omekMCDE7_86"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "k9oO2J6675iL"
},
"outputs": [],
"source": [
"import openai\n",
"import os\n",
"import IPython\n",
"\n",
"# API configuration\n",
"openai.api_key = \"YOUR-API-KEY\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "FVCiytVV75iM"
},
"outputs": [],
"source": [
"# completion function\n",
"def get_completion(messages, model=\"gpt-3.5-turbo\", temperature=0, max_tokens=300):\n",
" response = openai.ChatCompletion.create(\n",
" model=model,\n",
" messages=messages,\n",
" temperature=temperature,\n",
" max_tokens=max_tokens,\n",
" )\n",
" return response.choices[0].message[\"content\"]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4JdLUeuU75iM"
},
"source": [
"### Few-shot In-Context Learning\n",
"\n",
"Below we provide an example of few-shot prompting with demonstrations:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "4IxjDiXQ75iN",
"outputId": "a1a9ac7f-1da1-4b6c-fea5-022f58d17946"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"non-offensive\n"
]
}
],
"source": [
"prompt = \"\"\"Your task is to classify an input text (delimited by ```) as either offensive or non-offensive.\n",
"\n",
"Use the following examples to help with steering your responses:\n",
"\n",
"Text: I love you\n",
"Output: non-offensive\n",
"\n",
"Text: I dislike all those people working at the company\n",
"Output: offensive\n",
"\n",
"Text: I think this feature is not ideal\n",
"Output: non-offensive\n",
"\n",
"Text: Those people are so stupid\n",
"Output: offensive\n",
"\n",
"Text: {user_input}\n",
"Output:\n",
"\"\"\"\n",
"\n",
"message = [\n",
" {\n",
" \"role\": \"user\",\n",
" \"content\": prompt.format(user_input=\"```I respectfully disagree with your opinion.```\")\n",
" }\n",
"]\n",
"\n",
"response = get_completion(message)\n",
"print(response)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "SWARCtrw75iO"
},
"source": [
"### Chain-of-Though (CoT) Prompting\n",
"\n",
"Below is an example of CoT applied. Specifically, we aim to build a movie recommendation system using CoT.\n",
"\n",
"Let's first define a set of movies:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "h252IuDV75iO"
},
"outputs": [],
"source": [
"movies = \"\"\"\n",
"The Enigma Code\n",
"Category: Historical Drama\n",
"Rating: 8.3/10\n",
"Description: Set during World War II, this gripping historical drama follows the life of Alan Turing, a brilliant mathematician tasked with cracking the Enigma code used by the Nazis. His efforts contribute significantly to the Allies' victory.\n",
"Actors: Benedict Cumberbatch, Keira Knightley, Matthew Goode\n",
"Language: English\n",
"Release Date: March 15, 2014\n",
"Award Winner: Academy Award for Best Adapted Screenplay\n",
"\n",
"Shadows of the Samurai\n",
"Category: Action/Adventure\n",
"Rating: 7.9/10\n",
"Description: In feudal Japan, a skilled samurai seeks vengeance against the corrupt warlord who murdered his master. With his swordsmanship and determination, he embarks on a dangerous journey to restore justice.\n",
"Actors: Ken Watanabe, Tadanobu Asano, Rinko Kikuchi\n",
"Language: Japanese\n",
"Release Date: November 7, 2017\n",
"Award Winner: None\n",
"\n",
"Mind Games\n",
"Category: Psychological Thriller\n",
"Rating: 8.1/10\n",
"Description: A renowned psychologist becomes entangled in a twisted game of cat and mouse with a patient who harbors dark secrets. As their sessions progress, the lines between reality and deception blur, leading to a mind-bending climax.\n",
"Actors: Leonardo DiCaprio, Natalie Portman, Michael Fassbender\n",
"Language: English\n",
"Release Date: August 22, 2019\n",
"Award Winner: None\n",
"\n",
"La Casa del Tango\n",
"Category: Musical/Drama\n",
"Rating: 8.7/10\n",
"Description: In the vibrant world of Buenos Aires, a passionate tango dancer finds love and inspiration amidst the backdrop of political unrest. This musical drama explores the power of dance and the pursuit of dreams.\n",
"Actors: Antonio Banderas, Penélope Cruz, Javier Bardem\n",
"Language: Spanish\n",
"Release Date: June 5, 2020\n",
"Award Winner: Golden Globe for Best Foreign Language Film\n",
"\n",
"Timeless Love\n",
"Category: Romance/Fantasy\n",
"Rating: 7.5/10\n",
"Description: A magical encounter transports a modern-day writer back in time to Victorian England, where she falls in love with a charming aristocrat. As they navigate the complexities of time, their love is put to the ultimate test.\n",
"Actors: Rachel McAdams, Tom Hiddleston, Emma Thompson\n",
"Language: English\n",
"Release Date: February 14, 2022\n",
"Award Winner: None\n",
"\n",
"The Pursuit of Justice\n",
"Category: Legal Drama\n",
"Rating: 8.4/10\n",
"Description: Inspired by true events, this gripping legal drama follows a determined lawyer's fight against a powerful pharmaceutical company responsible for a life-threatening drug. The courtroom battle becomes a quest for justice and redemption.\n",
"Actors: Denzel Washington, Viola Davis, Michael B. Jordan\n",
"Language: English\n",
"Release Date: October 10, 2022\n",
"Award Winner: None\n",
"\n",
"The Forgotten Island\n",
"Category: Adventure/Mystery\n",
"Rating: 7.6/10\n",
"Description: A group of explorers stumbles upon a mysterious island believed to be uninhabited. As they uncover the island's secrets, they encounter deadly challenges and unravel an ancient civilization's enigma.\n",
"Actors: Chris Pratt, Bryce Dallas Howard, Tom Holland\n",
"Language: English\n",
"Release Date: July 2, 2023\n",
"Award Winner: None\n",
"\n",
"The Silent Witness\n",
"Category: Crime/Thriller\n",
"Rating: 8.2/10\n",
"Description: A talented forensic pathologist becomes entangled in a high-stakes murder investigation when she discovers crucial evidence that points to a powerful criminal network. With her life on the line, she must outsmart the perpetrators.\n",
"Actors: Emily Blunt, Jake Gyllenhaal, Mark Ruffalo\n",
"Language: English\n",
"Release Date: November 18, 2023\n",
"Award Winner: None\n",
"\n",
"A Tale of Two Worlds\n",
"Category: Fantasy/Adventure\n",
"Rating: 7.8/10\n",
"Description: When a young orphan discovers a magical portal to a parallel universe, she embarks on a thrilling adventure to save both realms from an impending disaster. Along the way, she learns about the power of friendship and self-belief.\n",
"Actors: Millie Bobby Brown, Tom Holland, Helena Bonham Carter\n",
"Language: English\n",
"Release Date: April 5, 2024\n",
"Award Winner: None\n",
"\n",
"A Symphony of Souls\n",
"Category: Music/Drama\n",
"Rating: 9.0/10\n",
"Description: Set against the backdrop of a renowned symphony orchestra, this emotionally charged drama explores the lives and intertwining stories of its members. Through the power of music, they find solace, love, and redemption.\n",
"Actors: Meryl Streep, Tom Hanks, Cate Blanchett\n",
"Language: English\n",
"Release Date: December 25, 2024\n",
"Award Winner: None\n",
"\"\"\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Xq9J_ESe75iP"
},
"source": [
"We then take those movies and list the different steps the model should perform. Notice that the after detailing the steps, we ask the model to providing the reasoning steps and the final response. This is a format you can use to elicit reasoning in LLMs."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ZxmNzJ7775iQ",
"outputId": "7d7905da-ea78-4fd5-fc93-21b6cf91274e"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Step 1: The user is asking about movies.\n",
"\n",
"Step 2: The user is specifically asking for drama movies.\n",
"\n",
"Step 3: We have several drama movies available:\n",
"\n",
"1. The Enigma Code - Category: Historical Drama, Rating: 8.3/10, Release Date: March 15, 2014\n",
"2. La Casa del Tango - Category: Musical/Drama, Rating: 8.7/10, Release Date: June 5, 2020\n",
"3. The Pursuit of Justice - Category: Legal Drama, Rating: 8.4/10, Release Date: October 10, 2022\n",
"4. The Silent Witness - Category: Crime/Thriller, Rating: 8.2/10, Release Date: November 18, 2023\n",
"5. A Symphony of Souls - Category: Music/Drama, Rating: 9.0/10, Release Date: December 25, 2024\n",
"\n",
"Step 4: Based on your request for drama movies, I recommend the following options:\n",
"- The Enigma Code: Set during World War II, this gripping historical drama follows the life of Alan Turing, a brilliant mathematician tasked with cracking the Enigma code used by the Nazis. His efforts contribute significantly to the Allies' victory. Starring Benedict Cumberbatch, Keira Knightley, and Matthew Goode.\n",
"- La Casa del Tango: In the vibrant world of Buenos Aires, a passionate tango dancer finds love and inspiration amidst the backdrop of political unrest. This musical drama explores the power of dance and the pursuit of dreams. Starring Antonio Banderas, Penélope Cruz, and Javier Bardem.\n",
"- The Pursuit of Justice: Inspired by true events, this gripping legal drama follows a determined lawyer's fight against a powerful pharmaceutical company responsible for a life-threatening drug. The courtroom battle becomes a quest for justice and redemption. Starring Denzel Washington, Viola Davis, and Michael B. Jordan.\n",
"- The Silent Witness: A talented forensic pathologist becomes entangled in a high-stakes murder investigation when she discovers crucial evidence that points to a powerful criminal network. With her life on the line, she must outsmart the perpetrators. Starring Emily Blunt, Jake Gyllenhaal, and Mark Ruffalo.\n",
"- A Symphony of Souls: Set against the backdrop of a renowned symphony orchestra, this emotionally charged drama explores the lives and intertwining stories of its\n"
]
}
],
"source": [
"# the system message contains the logic (step by step) for the system to follow\n",
"system_message = \"\"\"\n",
"You task is to make movie recommendations based on a user request (delimited by ```).\n",
"\n",
"Step 1: Check if the user is asking about movies. If the user is not asking about movies, just respond \"Please ask something about movies!\".\n",
"\n",
"Step 2: If the user is asking for a movie recommendation, check if they have any specific requests or interests.\n",
"\n",
"Step 3: Check if there are any movie/s we can recommend from the following: {movies}\n",
"\n",
"Step 4: Prepare a response to the user with the movie recommendation/s. The recommendation have to be about movies that are available in the list above. The response needs to have a friendly and helpful tone.\n",
"\n",
"Return a response with the following reasoning steps and final output to the user:\n",
"Step 1: <Step 1 reasoning>\n",
"Step 2: <Step 2 reasoning>\n",
"Step 3: <Step 3 reasoning>\n",
"Step 4: <final response>\n",
"\"\"\"\n",
"\n",
"messages = [\n",
" {\n",
" \"role\": \"system\",\n",
" \"content\": system_message.format(movies=movies)\n",
" },\n",
" {\n",
" \"role\": \"user\",\n",
" \"content\": \"```Do you have any drama movies?```\"\n",
" }\n",
"]\n",
"\n",
"movie_recommendation_response = get_completion(messages, temperature=0, max_tokens=500)\n",
"\n",
"print(movie_recommendation_response)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "9PO6l0hZ75iQ"
},
"source": [
"### Prompt Chaining\n",
"\n",
"The example below demonstrates how to chain separate prompts to achieve a specific task. The previous prompt list all the reasoning steps. In the following prompt, we ask the model to extract only the final response to the user:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "j8w10Dwy75iQ",
"outputId": "2770e41c-aa84-4a3b-8d30-2d59a1d2b16b"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Based on your request for drama movies, I recommend the following options:\n",
"- The Enigma Code: Set during World War II, this gripping historical drama follows the life of Alan Turing, a brilliant mathematician tasked with cracking the Enigma code used by the Nazis. His efforts contribute significantly to the Allies' victory. Starring Benedict Cumberbatch, Keira Knightley, and Matthew Goode.\n",
"- La Casa del Tango: In the vibrant world of Buenos Aires, a passionate tango dancer finds love and inspiration amidst the backdrop of political unrest. This musical drama explores the power of dance and the pursuit of dreams. Starring Antonio Banderas, Penélope Cruz, and Javier Bardem.\n",
"- The Pursuit of Justice: Inspired by true events, this gripping legal drama follows a determined lawyer's fight against a powerful pharmaceutical company responsible for a life-threatening drug. The courtroom battle becomes a quest for justice and redemption. Starring Denzel Washington, Viola Davis, and Michael B. Jordan.\n",
"- The Silent Witness: A talented forensic pathologist becomes entangled in a high-stakes murder investigation when she discovers crucial evidence that points to a powerful criminal network. With her life on the line, she must outsmart the perpetrators. Starring Emily Blunt, Jake Gyllenhaal, and Mark Ruffalo.\n",
"- A Symphony of Souls: Set against the backdrop of a renowned symphony orchestra, this emotionally charged drama explores the lives and intertwining stories of its characters. St\n"
]
}
],
"source": [
"# Prompt 1: step by step reasoning (provided above)\n",
"# Prompt 2: extract only the final response we will send to the user\n",
"\n",
"system_message_2 = \"\"\"\n",
"You will be given a list of steps that a model has responded with. Your task is to extract only the full response in Step 4 from the following text: {movie_recommendation_response}\n",
"\n",
"Step 4:\n",
"\"\"\"\n",
"\n",
"messages = [\n",
" {\n",
" \"role\": \"system\",\n",
" \"content\": system_message_2.format(movie_recommendation_response=movie_recommendation_response)\n",
" }\n",
"]\n",
"\n",
"final_response = get_completion(messages, temperature=0)\n",
"\n",
"print(final_response)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "imAdMD4375iR"
},
"source": [
"### ReAct\n",
"\n",
"The code below shows an example of combining LLMs with external tools to achieve a task. In particular, the example uses the ReAct framework to prompt and guide the model to the result by leveraging the LLM and a search engine. We will be using SerpAPI in the below example. As mentioned in the course introduction, you will need to register for a free account with SerpAPI to complete this part of the exercise. You can [register here](https://serpapi.com/)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "htgE9zNN75iR"
},
"outputs": [],
"source": [
"from langchain.agents import load_tools\n",
"from langchain.agents import AgentType\n",
"from langchain.llms import OpenAI\n",
"from langchain.agents import initialize_agent\n",
"\n",
"# for LangChain\n",
"os.environ[\"OPENAI_API_KEY\"] = openai.api_key # needs key from OpenAI\n",
"os.environ[\"SERPAPI_API_KEY\"] = \"SERPAPI_API_KEY\" # needs key for serpapi\n",
"\n",
"# model\n",
"llm = OpenAI(temperature=0)\n",
"\n",
"# tools\n",
"tools = load_tools([\"serpapi\", \"llm-math\"], llm=llm)\n",
"agent = initialize_agent(tools, llm , agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "eUyQ1DSH75iR",
"outputId": "d0ddc4e3-5948-4439-d24c-dd06cfc8be5f"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3m I need to find out what happened in the 2023 NBA finals\n",
"Action: Search\n",
"Action Input: 2023 NBA finals\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3mLed by Nikola Jokic's 28 points (12-16 FG), 16 rebounds, and four assists, the Nuggets defeated the Heat in Game 5, 94-89. Michael Porter Jr. (16 points ...\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: The Denver Nuggets won the 2023 NBA finals.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'The Denver Nuggets won the 2023 NBA finals.'"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# run the agent\n",
"agent.run(\"Which team won the 2023 NBA finals?\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "bNN3743a75iS"
},
"source": [
"### Data Augmentation / RAG\n",
"\n",
"The example below demonstrates how to apply data augmented generation."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "-R4DyoBH75iS"
},
"source": [
"We first load the necessary LangChain modules:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "nf0ViFQH75iS"
},
"outputs": [],
"source": [
"from langchain.embeddings.openai import OpenAIEmbeddings\n",
"from langchain.embeddings.cohere import CohereEmbeddings\n",
"from langchain.text_splitter import CharacterTextSplitter\n",
"from langchain.vectorstores.elastic_vector_search import ElasticVectorSearch\n",
"from langchain.vectorstores import Chroma\n",
"from langchain.docstore.document import Document\n",
"from langchain.prompts import PromptTemplate\n",
"\n",
"from langchain.chains.qa_with_sources import load_qa_with_sources_chain\n",
"from langchain.llms import OpenAI"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Q0euIhOy75iS"
},
"source": [
"Next, we split the loaded text file into chunks and embed the chunks using OpenAI Embeddings."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "zjqiX2_-75iS"
},
"outputs": [],
"source": [
"# load the data from GitHub\n",
"import urllib\n",
"f = urllib.request.urlopen(\"https://raw.githubusercontent.com/comet-ml/comet-llmops/main/data/kar-gpt.txt\")\n",
"text_data = f.read().decode(\"utf-8\")\n",
"\n",
"# split text into chunks\n",
"text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0, separator=\" \")\n",
"texts = text_splitter.split_text(text_data)\n",
"\n",
"# embeddings obtained from OpenAI\n",
"embeddings = OpenAIEmbeddings()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "sAdBa3Fc75iT"
},
"source": [
"The next step is to store the embeddings into Chroma, a dedicated vector storage. And finally, we perform a search by querying Chroma."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "QcigNs2e75iT",
"outputId": "641ae522-3e78-420e-8c25-a484fd9779bf"
},
"outputs": [
{
"data": {
"text/plain": [
"{'output_text': ' This course is about understanding and appreciating how chat GPT works, and how to develop a transformer neural network. It requires proficiency in Python and some basic understanding of calculus and statistics.\\nSOURCES: 1, 7, 107, 108'}"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# test search\n",
"docsearch = Chroma.from_texts(texts, embeddings, metadatas=[{\"source\": str(i)} for i in range(len(texts))])\n",
"\n",
"query = \"What is the course about?\"\n",
"docs = docsearch.similarity_search(query)\n",
"\n",
"chain = load_qa_with_sources_chain(OpenAI(temperature=0), chain_type=\"stuff\")\n",
"chain({\"input_documents\": docs, \"question\": query}, return_only_outputs=True)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "comet",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.17"
},
"orig_nbformat": 4,
"colab": {
"provenance": [],
"include_colab_link": true
}
},
"nbformat": 4,
"nbformat_minor": 0
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment