{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# 4.5 - Advanced - LLM APIs 2\n",
        "\n",
        "COMET Team <br> *Jonathan Graves, Alex Haddon*  \n",
        "2024-08-07\n",
        "\n",
        "## What are LLMs?\n",
        "\n",
        "Large Language Models (LLMs) are advanced machine learning models\n",
        "designed to understand and generate human-like text based on the data\n",
        "they have been trained on. Examples of popular LLMs include GPT-3.5 from\n",
        "OpenAI and open-source models such as `ollama` or `huggingface`.\n",
        "\n",
        "## Applications of LLMs\n",
        "\n",
        "Large language models have a wide range of applications across various\n",
        "domains. In natural language understanding (NLU), they excel in tasks\n",
        "like text classification, named entity recognition, and language\n",
        "translation, enabling efficient content categorization and multilingual\n",
        "communication. LLMs are also powerful tools for text generation,\n",
        "facilitating the creation of articles, creative writing, and\n",
        "summarization of lengthy documents. Additionally, they enhance\n",
        "conversational agents and virtual assistants, providing human-like\n",
        "interactions and support. Furthermore, LLMs play a crucial role in\n",
        "knowledge extraction, sentiment analysis, and automated coding, making\n",
        "them invaluable in fields like customer support, market analysis,\n",
        "software development, and beyond. In fact, what you are reading right\n",
        "now was created using an LLM!\n",
        "\n",
        "Here is a [cool\n",
        "video](https://www.youtube.com/watch?v=5sLYAQS9sWQ&ab_channel=IBMTechnology)\n",
        "made by IBM that explains a little more about how LLMs work.\n",
        "\n",
        "# Setting Up the Environment\n",
        "\n",
        "Head to [ollama.com](https://ollama.com/) and download ollama locally.\n",
        "Then, in your terminal, run the code `ollama pull llama3` and wait for\n",
        "it to install\n",
        "\n",
        "## Installing Required Libraries\n",
        "\n",
        "Make sure to install the ollama library if you haven’t already; in your\n",
        "terminal use the command `pip install ollama`. There will be various\n",
        "other packages you will be prompted to install later in this notebook.\n",
        "\n",
        "# Using an LLM (e.g., llama3)\n",
        "\n",
        "## Connecting to the LLM API\n",
        "\n",
        "Define a function to query the model by specifying the correct model as\n",
        "well as the prompt we want to pass to the model.\n",
        "\n",
        "NOTE: Make sure that you have the ollama application open and running\n",
        "locally before you try and make an API call or else you will get an\n",
        "error likely stating your connection has been “refused”."
      ],
      "id": "6dd76f5e-ac17-4d8c-b9c7-d8e8fdb82716"
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Import the required library (ollama)\n",
        "import ollama\n",
        "\n",
        "\n",
        "response = ollama.chat(\n",
        "    model='llama3',  # specify the model \n",
        "    messages=[{'role': 'user', 'content': 'In fewer than 50 words, why is the sky blue?'}]) # insert the desired prompt\n",
        "\n",
        "print(response)"
      ],
      "id": "711a2cb7"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "The output of our API call to `ollama` comes in the\n",
        "[JSON](https://www.json.org/json-en.html) form which stands for\n",
        "JavaScript Object Notation. Essentially the output is split into a\n",
        "series of pairs consisting of a field name, colon, and then the value.\n",
        "For example, the output of our API call has `'model':'llama3'` as one of\n",
        "the entries meaning that the model we used to generate the response is\n",
        "llama3. If we want just the response to be the output we can specify\n",
        "that in our print statement using the code below:"
      ],
      "id": "19360bfb-da1a-4aab-93f8-94d89048a6df"
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Only show the response from llama3\n",
        "print(response['message']['content'])"
      ],
      "id": "1020f8bb"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Now you try! Fill in the code skeleton with the correct code.\n",
        "\n",
        "HINT: *In your prompt specify that you don’t want a long response.\n",
        "Without that, ollama can take a very long time, especially if your\n",
        "machine is slower, as it is running locally rather than connecting to\n",
        "external servers.*"
      ],
      "id": "93aab987-da16-437b-8d86-0c937e92b017"
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {},
      "outputs": [],
      "source": [
        "#response = ollama.chat(\n",
        "#     model= ...,\n",
        "#     messages=[{'role': 'user', 'content': ...}])\n",
        "\n",
        "# print(...)"
      ],
      "id": "a1e4da89"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Self-Tests and Exercises\n",
        "\n",
        "## Multiple Choice Questions\n",
        "\n",
        "Here are a few questions you can use to check your understanding. Run\n",
        "the cell below before attempting the multiple choice questions."
      ],
      "id": "94671aaf-9bb1-4efa-b4e5-b36a1e07d149"
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {},
      "outputs": [],
      "source": [
        "from advanced_llm_apis2_tests.py import Tests"
      ],
      "id": "52b4927c"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Question 1\n",
        "\n",
        "The output in JSON form uses the dictionary data type. What key (or\n",
        "sequence of keys) in the dictionary holds the output of the model? - A)\n",
        "\\[‘model’\\] - B) \\[‘message’\\] - C) \\[‘message’\\]\\[‘content’\\] - D)\n",
        "\\[‘content’\\] - E) \\[‘content’\\]\\[‘message’\\] - F)\n",
        "\\[‘model’\\]\\[‘message’\\]\n",
        "\n",
        "*Enter your answer below as a a string with one of A,B,C,D,E,F ie. “A”*"
      ],
      "id": "7fb55a94-c16f-45de-8c96-2f5e2cda10e1"
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {},
      "outputs": [],
      "source": [
        "answer1 = #your answer here \n",
        "\n",
        "Tests.test1(answer1)"
      ],
      "id": "03c83871"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Question 2\n",
        "\n",
        "Out of the options below, which best describes what an LLM (Large\n",
        "Language Model) is?\n",
        "\n",
        "-   1.  A specialized algorithm for analyzing large datasets and\n",
        "        generating insights.\n",
        "-   1.  A type of neural network that excels in generating human-like\n",
        "        text based on extensive training data.\n",
        "-   1.  A tool designed for processing and translating spoken language\n",
        "        into text.\n",
        "-   1.  A machine learning model primarily used for image and object\n",
        "        recognition.\n",
        "\n",
        "*Enter your answer below as a a string with one of A,B,C,D ie. “A”*"
      ],
      "id": "6ea5fafb-dbdf-40e2-a7ed-02ed7087e170"
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {},
      "outputs": [],
      "source": [
        "answer2 = #your answer here \n",
        "\n",
        "Tests.test2(answer2)"
      ],
      "id": "d7328cc5"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Worked Example: Retrieving Data from a PDF\n",
        "\n",
        "## Problem Statement\n",
        "\n",
        "One real world application of what we learned above is when we have a\n",
        "pdf that we want our LLM to be able to answer questions about. This is a\n",
        "process called “fine tuning” where we train the LLM to answer our\n",
        "prompts under the context of the contents of our pdf or more broadly the\n",
        "information that we give to it. In this example, we will fine tune our\n",
        "LLM using The Gobal Risks Report 2024 from the World Economic Forum.\n",
        "After doing so, we will ask the LLM to give us some contextual based\n",
        "answers to questions we prompt the LLM with.\n",
        "\n",
        "## Solution Using the LLM\n",
        "\n",
        "Follow the steps below to get a comprehensive analysis using an LLM.\n",
        "\n",
        "#### Step 1\n",
        "\n",
        "First we will need to install various packages that will allow the PDF\n",
        "to be read and interpreted by the LLM. If you are interested in how this\n",
        "process works, feel free to open up the functions file as well as check\n",
        "out other lessons on word embeddings and creating RAGs."
      ],
      "id": "3a04901a-c15e-4701-b601-8695cae781e7"
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {},
      "outputs": [],
      "source": [
        "%pip install --upgrade --q unstructured langchain\n",
        "%pip install --upgrade --q \"unstructured[all-docs]\"\n",
        "%pip install --q langchain_community langchain_text_splitters chromadb\n",
        "!brew install libmagic\n",
        "!brew install poppler\n",
        "!brew install tesseract"
      ],
      "id": "78f2b79b"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "#### Step 2\n",
        "\n",
        "Now we will import the pdf using the `load_file(...)` function."
      ],
      "id": "d3f04c43-3b7f-43f7-ba1d-dda9d4c3a03e"
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {},
      "outputs": [],
      "source": [
        "from v2_functions import load_file\n",
        "\n",
        "path = \"WEF_The_Global_Risks_Report_2024.pdf\"\n",
        "\n",
        "data = load_file(path)\n",
        "\n",
        "# Check if data was loaded successfully\n",
        "if data:\n",
        "    # Preview the first page content\n",
        "    print(data[0].page_content)\n",
        "else:\n",
        "    print(\"Failed to load the PDF file.\")"
      ],
      "id": "c8fa5c75"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "#### Step 3\n",
        "\n",
        "We will now embed to text into vectors and create a database from which\n",
        "the LLM will pull information from. We are essentially “translating” the\n",
        "contents of the pdf into a readable text for the LLM to use. Here, we\n",
        "use the function `process_and_store_pdf(...)`."
      ],
      "id": "7be437f6-3696-4ac3-90a4-f7cc081e8c80"
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {},
      "outputs": [],
      "source": [
        "from v2_functions import process_and_store_pdf\n",
        "\n",
        "!ollama pull nomic-embed-text\n",
        "\n",
        "vector_database = process_and_store_pdf(data)"
      ],
      "id": "93a2a4f6"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "#### Step 4\n",
        "\n",
        "Now, we need to specify that we will be using the `llama3` LLM model and\n",
        "then use the function `setup_retriever_and_chain(..., ...)` which will\n",
        "make calls to the API to answer our prompts. Then, we will try answering\n",
        "a few prompts using the LLM!"
      ],
      "id": "55b6d97e-108d-43dc-a9bd-659ac9cbfdef"
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {},
      "outputs": [],
      "source": [
        "from v2_functions import setup_retriever_and_chain\n",
        "\n",
        "# specify LLM model\n",
        "model = \"llama3\"\n",
        "\n",
        "# retrieval function; see the functions file for more information\n",
        "chain = setup_retriever_and_chain(vector_database, model)\n",
        "\n",
        "# specify prompt\n",
        "prompt = \"Provide a summary of the report.\"\n",
        "\n",
        "# After, try a few of the prompts below:\n",
        "# - \"What does the report say about climate change?\"\n",
        "# - \"What does the report say about misinformation?\"\n",
        "# - \"Summarize the section on conflict.\"\n",
        "\n",
        "# create a function to put it all together\n",
        "def local_model(chain, prompt):\n",
        "    response = chain.invoke(prompt)\n",
        "    print(response)\n",
        "\n",
        "# making a function call; adjust the prompt as desired\n",
        "local_model(chain, prompt)"
      ],
      "id": "1b508ed5"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "#### Step 6 (Optional)\n",
        "\n",
        "If we wanted to restart with a different pdf, we would need to delete\n",
        "all the existing embeddings we made and you can do so by running the\n",
        "code below."
      ],
      "id": "08611f39-5480-4d01-b103-204897c3fdcd"
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Reset the embeddings with this line of code\n",
        "# vector_database.delete_collection()"
      ],
      "id": "32ac9ec9"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Conclusion\n",
        "\n",
        "## Recap of What Was Learned\n",
        "\n",
        "-   We re-introduced the concept of Large Language Models (LLMs) and\n",
        "    their applications.\n",
        "-   We set up the environment and connected to the Ollama API.\n",
        "-   We explored how to use LLMs with example prompts and responses.\n",
        "-   We created our own embeddings from which we could make api calls to\n",
        "    the Ollama API with the additional context of the given pdf.\n",
        "\n",
        "For more information about word embeddings and retrieval-augmented\n",
        "generation (RAG) see our other applicable notebooks."
      ],
      "id": "40aed54d-7b3a-449d-b340-a67bcc5a0e2f"
    }
  ],
  "nbformat": 4,
  "nbformat_minor": 5,
  "metadata": {
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "path": "/usr/local/share/jupyter/kernels/python3"
    },
    "language_info": {
      "name": "python",
      "codemirror_mode": {
        "name": "ipython",
        "version": "3"
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.12.3"
    }
  }
}