{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# 4.5 - Advanced - LLM APIs 2\n", "\n", "COMET Team
*Jonathan Graves, Alex Haddon* \n", "2024-08-07\n", "\n", "## What are LLMs?\n", "\n", "Large Language Models (LLMs) are advanced machine learning models\n", "designed to understand and generate human-like text based on the data\n", "they have been trained on. Examples of popular LLMs include GPT-3.5 from\n", "OpenAI and open-source models such as `ollama` or `huggingface`.\n", "\n", "## Applications of LLMs\n", "\n", "Large language models have a wide range of applications across various\n", "domains. In natural language understanding (NLU), they excel in tasks\n", "like text classification, named entity recognition, and language\n", "translation, enabling efficient content categorization and multilingual\n", "communication. LLMs are also powerful tools for text generation,\n", "facilitating the creation of articles, creative writing, and\n", "summarization of lengthy documents. Additionally, they enhance\n", "conversational agents and virtual assistants, providing human-like\n", "interactions and support. Furthermore, LLMs play a crucial role in\n", "knowledge extraction, sentiment analysis, and automated coding, making\n", "them invaluable in fields like customer support, market analysis,\n", "software development, and beyond. In fact, what you are reading right\n", "now was created using an LLM!\n", "\n", "Here is a [cool\n", "video](https://www.youtube.com/watch?v=5sLYAQS9sWQ&ab_channel=IBMTechnology)\n", "made by IBM that explains a little more about how LLMs work.\n", "\n", "# Setting Up the Environment\n", "\n", "Head to [ollama.com](https://ollama.com/) and download ollama locally.\n", "Then, in your terminal, run the code `ollama pull llama3` and wait for\n", "it to install\n", "\n", "## Installing Required Libraries\n", "\n", "Make sure to install the ollama library if you haven’t already; in your\n", "terminal use the command `pip install ollama`. There will be various\n", "other packages you will be prompted to install later in this notebook.\n", "\n", "# Using an LLM (e.g., llama3)\n", "\n", "## Connecting to the LLM API\n", "\n", "Define a function to query the model by specifying the correct model as\n", "well as the prompt we want to pass to the model.\n", "\n", "NOTE: Make sure that you have the ollama application open and running\n", "locally before you try and make an API call or else you will get an\n", "error likely stating your connection has been “refused”." ], "id": "169d9ee0-eb0e-4ffe-8798-6c9b129c2f6b" }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "!pip install ollama\n", "!pip install pandas" ], "id": "cell-2" }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import pandas as pd\n", "import ollama\n", "from advanced_llm_apis2_tests import Tests" ], "id": "cell-3" }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "\n", "response = ollama.chat(\n", " model='llama3', # specify the model \n", " messages=[{'role': 'user', 'content': 'In fewer than 50 words, why is the sky blue?'}]) # insert the desired prompt\n", "\n", "print(response)" ], "id": "cell-4" }, { "cell_type": "markdown", "metadata": {}, "source": [ "The output of our API call to `ollama` comes in the\n", "[JSON](https://www.json.org/json-en.html) form which stands for\n", "JavaScript Object Notation. Essentially the output is split into a\n", "series of pairs consisting of a field name, colon, and then the value.\n", "For example, the output of our API call has `'model':'llama3'` as one of\n", "the entries meaning that the model we used to generate the response is\n", "llama3. If we want just the response to be the output we can specify\n", "that in our print statement using the code below:" ], "id": "00322bb3-38a5-416c-856d-9d63b236e147" }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Only show the response from llama3\n", "print(response['message']['content'])" ], "id": "cell-6" }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now you try! Fill in the code skeleton with the correct code.\n", "\n", "HINT: *In your prompt specify that you don’t want a long response.\n", "Without that, ollama can take a very long time, especially if your\n", "machine is slower, as it is running locally rather than connecting to\n", "external servers.*" ], "id": "3fb5b8ef-2a9f-4ff0-a7a1-07084d5196d6" }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#response = ollama.chat(\n", "# model= ...,\n", "# messages=[{'role': 'user', 'content': ...}])\n", "\n", "# print(...)" ], "id": "cell-8" }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Self-Tests and Exercises\n", "\n", "## Multiple Choice Questions\n", "\n", "Here are a few questions you can use to check your understanding. Run\n", "the cell below before attempting the multiple choice questions.\n", "\n", "### Question 1\n", "\n", "The output in JSON form uses the dictionary data type. What key (or\n", "sequence of keys) in the dictionary holds the output of the model? - A)\n", "\\[‘model’\\] - B) \\[‘message’\\] - C) \\[‘message’\\]\\[‘content’\\] - D)\n", "\\[‘content’\\] - E) \\[‘content’\\]\\[‘message’\\] - F)\n", "\\[‘model’\\]\\[‘message’\\]\n", "\n", "*Enter your answer below as a a string with one of A,B,C,D,E,F ie. “A”*" ], "id": "1fe0195c-80bb-4e65-b461-8c3afd268354" }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "answer1 = #your answer here\n", "\n", "Tests.test1(answer1)" ], "id": "cell-11" }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Question 2\n", "\n", "Out of the options below, which best describes what an LLM (Large\n", "Language Model) is?\n", "\n", "- 1. A specialized algorithm for analyzing large datasets and\n", " generating insights.\n", "- 1. A type of neural network that excels in generating human-like\n", " text based on extensive training data.\n", "- 1. A tool designed for processing and translating spoken language\n", " into text.\n", "- 1. A machine learning model primarily used for image and object\n", " recognition.\n", "\n", "*Enter your answer below as a a string with one of A,B,C,D ie. “A”*" ], "id": "d4568ad3-8401-47c2-b64c-3fd5d3ad4e0d" }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "answer2 = #your answer here \n", "\n", "Tests.test2(answer2)" ], "id": "cell-13" }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Worked Example: Natural language processing and sentiment Analysis\n", "\n", "**Natural language processing** is a field of artificial intelligence\n", "that focuses on the interaction between computers and human languages.\n", "It aims to enable machines to understand, interpret, generate, and\n", "respond to human language in a way that is both meaningful and useful.\n", "One category of natural language processing is sentiment analysis: the\n", "NLP technique used to determine the emotional tone or sentiment\n", "expressed in a piece of text. It aims to classify text as positive,\n", "negative, neutral, or sometimes more granular emotional categories\n", "(e.g., anger, joy, sadness). Sentiment analysis, also referred to as\n", "“opinion mining”, is widely used for analyzing opinions, feedback, and\n", "emotions in social media posts, reviews, surveys, and other types of\n", "text data.\n", "\n", "## Problem Statement\n", "\n", "One real world application of what we learned above is when we have a\n", "pdf that we want our LLM to be able to answer questions about. This is a\n", "process called “fine tuning” where we train the LLM to answer our\n", "prompts under the context of the contents of our pdf or more broadly the\n", "information that we give to it. In this example, we will fine tune our\n", "LLM using The Gobal Risks Report 2024 from the World Economic Forum.\n", "After doing so, we will ask the LLM to give us some contextual based\n", "answers to questions we prompt the LLM with.\n", "\n", "## Solution Using the LLM\n", "\n", "Follow the steps below to get a comprehensive analysis using an LLM.\n", "\n", "#### Step 1: Installing Required Python Libraries\n", "\n", "We did this above when ran the command `pip install pandas` and\n", "`import pandas as pd` We will install pandas which will help us convert\n", "our survey responses into a machine readable data frame. Pandas is a\n", "popular Python library used for data manipulation and analysis. It\n", "provides powerful data structures like DataFrames and Series, which\n", "allow users to work with labeled and relational data intuitively. With\n", "pandas, you can easily read, clean, transform, and analyze data from\n", "various formats such as CSV, Excel, SQL databases, and more.\n", "\n", "#### Step 2: Read the CSV File\n", "\n", "First, we will read the `survey_responses_election.csv` file into a\n", "pandas DataFrame to load the survey responses." ], "id": "51f9354d-e26c-4577-9763-ef245260f9a5" }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Read the CSV file with the survey responses\n", "df = pd.read_csv('survey_responses_election.csv')\n", "\n", "# Display the first few rows of the DataFrame to verify the content\n", "print(df.head())" ], "id": "cell-17" }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Step 3: Prepare the Responses for Sentiment Analysis\n", "\n", "Next, we’ll convert the survey responses from the DataFrame into a list\n", "format that can be passed to the LLM for sentiment analysis." ], "id": "59dabb1d-51e4-4c8a-b6b4-a32d6941f919" }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Convert the 'Survey Response' column into a list\n", "responses = df['Survey Response'].tolist()\n", "\n", "# Print the list of responses to verify\n", "print(responses[:5]) # Display the first 5 responses" ], "id": "cell-19" }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Step 4: Perform Sentiment Analysis Using an LLM\n", "\n", "Now that we have the responses in a list, we will use an LLM model (like\n", "llama3 from ollama) to perform sentiment analysis. The LLM will analyze\n", "the sentiments expressed in the survey responses." ], "id": "bd97db22-d84d-489a-bb6d-313a1555b87e" }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Define a prompt for sentiment analysis\n", "response = ollama.chat(\n", " model='llama3', # specify the model\n", " messages=[{'role': 'user', 'content': f\"Analyze the sentiment of the following survey responses:\\n{responses}\"}]\n", ")\n", "\n", "# Print the model's output (the sentiment analysis) formatted the output for better readability\n", "print(\"Sentiment Analysis Results:\")\n", "print(response['message']['content'])" ], "id": "cell-21" }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Conclusion\n", "\n", "## Recap of What Was Learned\n", "\n", "- We re-introduced the concept of Large Language Models (LLMs) and\n", " their applications.\n", "- We set up the environment and connected to the Ollama API.\n", "- We explored how to use LLMs with example prompts and responses.\n", "- We created our own embeddings from which we could make api calls to\n", " the Ollama API with the additional context of the given pdf.\n", "\n", "For more information about word embeddings and retrieval-augmented\n", "generation (RAG) see our other applicable notebooks." ], "id": "f4a26f09-ec00-49de-bf4f-c019fe926fe6" } ], "nbformat": 4, "nbformat_minor": 5, "metadata": { "kernelspec": { "name": "python3", "display_name": "Python 3 (ipykernel)", "language": "python" }, "language_info": { "name": "python", "codemirror_mode": { "name": "ipython", "version": "3" }, "file_extension": ".py", "mimetype": "text/x-python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.7" } } }