4.5 - Advanced - LLM APIs 2

This notebook illustrates how to call different Large Language Models (LLMs) using their API, for the purposes of data analysis or computational use.

What are LLMs?

Large Language Models (LLMs) are advanced machine learning models designed to understand and generate human-like text based on the data they have been trained on. Examples of popular LLMs include GPT-3.5 from OpenAI and open-source models such as ollama or huggingface.

Applications of LLMs

Large language models have a wide range of applications across various domains. In natural language understanding (NLU), they excel in tasks like text classification, named entity recognition, and language translation, enabling efficient content categorization and multilingual communication. LLMs are also powerful tools for text generation, facilitating the creation of articles, creative writing, and summarization of lengthy documents. Additionally, they enhance conversational agents and virtual assistants, providing human-like interactions and support. Furthermore, LLMs play a crucial role in knowledge extraction, sentiment analysis, and automated coding, making them invaluable in fields like customer support, market analysis, software development, and beyond. In fact, what you are reading right now was created using an LLM!

Here is a cool video made by IBM that explains a little more about how LLMs work.

Setting Up the Environment

Head to ollama.com and download ollama locally. Then, in your terminal, run the code ollama pull llama3 and wait for it to install

Installing Required Libraries

Make sure to install the ollama library if you haven’t already; in your terminal use the command pip install ollama. There will be various other packages you will be prompted to install later in this notebook.

Using an LLM (e.g., llama3)

Connecting to the LLM API

Define a function to query the model by specifying the correct model as well as the prompt we want to pass to the model.

NOTE: Make sure that you have the ollama application open and running locally before you try and make an API call or else you will get an error likely stating your connection has been “refused”.

!pip install ollama
!pip install pandas

import pandas as pd
import ollama
from advanced_llm_apis2_tests import Tests

response = ollama.chat(
    model='llama3',  # specify the model 
    messages=[{'role': 'user', 'content': 'In fewer than 50 words, why is the sky blue?'}]) # insert the desired prompt

print(response)

The output of our API call to ollama comes in the JSON form which stands for JavaScript Object Notation. Essentially the output is split into a series of pairs consisting of a field name, colon, and then the value. For example, the output of our API call has 'model':'llama3' as one of the entries meaning that the model we used to generate the response is llama3. If we want just the response to be the output we can specify that in our print statement using the code below:

# Only show the response from llama3
print(response['message']['content'])

Now you try! Fill in the code skeleton with the correct code.

HINT: In your prompt specify that you don’t want a long response. Without that, ollama can take a very long time, especially if your machine is slower, as it is running locally rather than connecting to external servers.

#response = ollama.chat(
#     model= ...,
#     messages=[{'role': 'user', 'content': ...}])

# print(...)

Self-Tests and Exercises

Multiple Choice Questions

Here are a few questions you can use to check your understanding. Run the cell below before attempting the multiple choice questions.

Question 1

The output in JSON form uses the dictionary data type. What key (or sequence of keys) in the dictionary holds the output of the model? - A) [‘model’] - B) [‘message’] - C) [‘message’][‘content’] - D) [‘content’] - E) [‘content’][‘message’] - F) [‘model’][‘message’]

Enter your answer below as a a string with one of A,B,C,D,E,F ie. “A”

answer1 = #your answer here

Tests.test1(answer1)

Question 2

Out of the options below, which best describes what an LLM (Large Language Model) is?

• 1. A specialized algorithm for analyzing large datasets and generating insights.
• 2. A type of neural network that excels in generating human-like text based on extensive training data.
• 3. A tool designed for processing and translating spoken language into text.
• 4. A machine learning model primarily used for image and object recognition.

Enter your answer below as a a string with one of A,B,C,D ie. “A”

answer2 = #your answer here 

Tests.test2(answer2)

Worked Example: Natural language processing and sentiment Analysis

Natural language processing is a field of artificial intelligence that focuses on the interaction between computers and human languages. It aims to enable machines to understand, interpret, generate, and respond to human language in a way that is both meaningful and useful. One category of natural language processing is sentiment analysis: the NLP technique used to determine the emotional tone or sentiment expressed in a piece of text. It aims to classify text as positive, negative, neutral, or sometimes more granular emotional categories (e.g., anger, joy, sadness). Sentiment analysis, also referred to as “opinion mining”, is widely used for analyzing opinions, feedback, and emotions in social media posts, reviews, surveys, and other types of text data.

Problem Statement

One real world application of what we learned above is when we have a pdf that we want our LLM to be able to answer questions about. This is a process called “fine tuning” where we train the LLM to answer our prompts under the context of the contents of our pdf or more broadly the information that we give to it. In this example, we will fine tune our LLM using The Gobal Risks Report 2024 from the World Economic Forum. After doing so, we will ask the LLM to give us some contextual based answers to questions we prompt the LLM with.

Solution Using the LLM

Follow the steps below to get a comprehensive analysis using an LLM.

Step 1: Installing Required Python Libraries

We did this above when ran the command pip install pandas and import pandas as pd We will install pandas which will help us convert our survey responses into a machine readable data frame. Pandas is a popular Python library used for data manipulation and analysis. It provides powerful data structures like DataFrames and Series, which allow users to work with labeled and relational data intuitively. With pandas, you can easily read, clean, transform, and analyze data from various formats such as CSV, Excel, SQL databases, and more.

Step 2: Read the CSV File

First, we will read the survey_responses_election.csv file into a pandas DataFrame to load the survey responses.

# Read the CSV file with the survey responses
df = pd.read_csv('survey_responses_election.csv')

# Display the first few rows of the DataFrame to verify the content
print(df.head())

Step 3: Prepare the Responses for Sentiment Analysis

Next, we’ll convert the survey responses from the DataFrame into a list format that can be passed to the LLM for sentiment analysis.

# Convert the 'Survey Response' column into a list
responses = df['Survey Response'].tolist()

# Print the list of responses to verify
print(responses[:5])  # Display the first 5 responses

Step 4: Perform Sentiment Analysis Using an LLM

Now that we have the responses in a list, we will use an LLM model (like llama3 from ollama) to perform sentiment analysis. The LLM will analyze the sentiments expressed in the survey responses.

# Define a prompt for sentiment analysis
response = ollama.chat(
    model='llama3',  # specify the model
    messages=[{'role': 'user', 'content': f"Analyze the sentiment of the following survey responses:\n{responses}"}]
)

# Print the model's output (the sentiment analysis) formatted the output for better readability
print("Sentiment Analysis Results:")
print(response['message']['content'])

Conclusion

Recap of What Was Learned

• We re-introduced the concept of Large Language Models (LLMs) and their applications.
• We set up the environment and connected to the Ollama API.
• We explored how to use LLMs with example prompts and responses.
• We created our own embeddings from which we could make api calls to the Ollama API with the additional context of the given pdf.

For more information about word embeddings and retrieval-augmented generation (RAG) see our other applicable notebooks.