Webinar: Large Language Models 4 NØØBs

Jack Ekelof:

Hello, I'm Jack Ekelof. I'm here with J. R. Hernandez. Welcome to "Large Language Models 4 NOOBs." I'm going to kick things off by introducing J. R., who's the Director of Offensive and Security Services here at Evolve Security. He's going to be discussing LLMs — an exciting topic here in 2024 — and I look forward to him kicking us off.

J. R. Hernandez:

Hello, everybody. Thank you, Jack, and thank you all for coming — I really appreciate it. Let's get started with the agenda. In this talk, we're going to cover what LLMs are, how they work, how we train them, and what the future holds, and then we'll have a little time for questions.

So, who am I, and who am I not? As Jack said, I'm the Offensive Security Manager here at Evolve Security. I'm a former penetration tester and a lifelong noob — which just means I'm always trying to be better and understand things, because there's a lot I don't know, but I'm always trying to improve. What I am not: I'm not an AI expert, a data scientist, a machine learning researcher, or a mathematician.

So why have this talk if I'm none of those things? As you know, large language models have taken the world by storm since around 2023 — that's when it really popped off. It's a relatively new technology, and there are a lot of unknowns about the capabilities of LLMs. I wasn't too familiar with the technology before ChatGPT exploded, so there was a lot of trying to understand what this was — and honestly, being a bit scared of it. My first thought was, "I need to understand our future AI overlords." I've been in security for a long time, so I have that InfoSec perspective, and I thought maybe I could bring a unique angle to the conversation. As security people, continuous learning is part of our job — we have to stay on top of new technologies because ultimately we're going to have to test them and secure them. And the best way to learn something is to teach it, so I pushed myself to put this talk together.

So, what are large language models? They're generative AI models capable of comprehending and producing natural language. They do this by analyzing large amounts of text and identifying statistical patterns. A limited list of what they can do: generate code, review code, text generation and summarization, translation, chatbots, sentiment analysis, AI assistants, and much more.

You might have caught me "cheating" there by using the word "model" to describe what LLMs are. There's a reason for that — I can't really explain what an LLM is without first giving you some foundational knowledge of how AI relates to a bunch of other fields. Start at the top with artificial intelligence: a branch of computer science aimed at creating machines capable of performing tasks that require human intelligence. A subset of that is machine learning — how machines learn, usually by analyzing datasets and identifying patterns through algorithms. A subset of machine learning is deep learning, which uses neural networks to learn and solve more complex problems in a way that mimics the human mind. And a subset of that is the newest, greatest thing everybody's talking about: generative AI. The big thing about generative AI is that it doesn't just identify patterns — it generates new content. Large language models are a subset of generative AI: they analyze large amounts of text — words, phrases, sentences — through a training process, identify relationships and statistics in that text, and then generate similar text.

So what does that all mean? It breaks down to this: y equals a function of x. This is the most math we'll do today. F(x) is your AI model — similar to Llama 3, the model from Meta. X is the input you give it. Y is the output. A typical example: with Llama 3, you give it a prompt and it generates text. With Midjourney, you give it a prompt and it generates an image. So the basic functionality is: you give it an input, it goes through a mathematical function, and it gives you an output.

In machine learning, there's a task called predictive text, where you analyze large amounts of text and ask the model to predict the next word in a sequence. It does this by learning statistical relationships from the documents you give it. Think of it like an improv partner giving you its best guess at what comes next. For example: "Humpty Dumpty sat on the wall, Humpty Dumpty had a great..." — and it gives us "fall" as the likely next word, with a 97% chance of being next.

Here's a better example. I said, "Make a three-sentence story about a hacker meetup," and it gave me a full sentence where every word has a certain percentage assigned to it. I focused on the word "monthly" in "a group of hackers gathered for their monthly meetup." For "monthly," it could have used "annual," "regular," "secret," or "weekly" — the sentence still works — but based on all the data it was trained on, "monthly" had the highest percentage, so that's the word it used. At the end of the day, tools like your iPhone's iMessage already do this when they give you predictive text for the next word. ChatGPT, Llama, and these new technologies are doing something similar — just far more complex — but it's the same idea: predicting the next word.

So how do they work? There are three components to be aware of: data (the large amounts of text needed to train the tool), training (a very particular process that needs large amounts of data), and transformers (a novel neural network that's the game-changer behind why these things became so effective and accurate).

Let's break those down, starting with transformers. Transformers are a novel neural network that came out of a 2017 white paper from researchers at Google. It was originally designed for language translation — translating, say, English words to French — but if you read the paper, they clearly knew it was bigger than that; they just limited the scope. A couple of the big changes: they were able to train models a lot quicker, because before transformers you had to process text one word at a time, whereas transformers could process a whole sentence at once. (That's the "T" in GPT — transformer.) The other big change was that it gave you context for what words mean in relation to other words in the sentence, which was novel. Those changes made transformers really popular and kicked off the whole generative AI situation we're in now.

I'm not going to go too deep, but at a simplistic level, a neural network is where the magic happens. You give it inputs, it turns them into something the network can understand, and they get processed in what we call hidden layers. Every layer does something to that input, changes it, and passes it to the output layer, which has the answer. Keep in mind this mimics how the human mind works through neurons and synapses — the circles are the neurons, and the synapses are the parameters or connections between them. Simple example: if you give it an image of a dog, it processes it through the network, and the output hopefully tells you it's a dog.

Let's talk about tokenization. To take the text you provide as a prompt and feed it to these models, you have to turn that text into tokens — a process called tokenization. Tokens aren't just words; they're smaller chunks, and it's not standardized — every model (ChatGPT, Llama) tokenizes differently. Images can also be tokenized, broken down into bits. Tokens count toward the size of the training data and the context window, so keep in mind that when you provide prompts, they get broken into smaller chunks called tokens — that comes into play later.

Now, that thing I mentioned about transformers being aware of context in relation to other words — that's my favorite thing I found when I started researching this. It's a concept called embeddings. Here I have three words: "food," "foot," and "burger." Next to each is an array of numbers — that array is called an embedding, a numerical representation of the word. If you look at "food" and "foot" in a dictionary, they're close together because of their spelling, but they're not really related — there's no connection binding them. However, "food" and "burger" have a connection, because a burger is a food. So embeddings add relationships — a way to add context to what each word means. Think of them as coordinates on a map, where "food" and "burger" sit closer together than "food" and "foot." It's hard to visualize because these aren't two-dimensional arrays with an x and y axis — they're multidimensional, with thousands of dimensions, so they're hard for humans to picture. But think of embeddings as a way to numerically add context to words and build relationships between them. If you take anything away from this talk, I'd suggest looking into that — it's really interesting.

Here's a multidimensional vector space where all the little dots are words. The word we're looking at is "king." When we use its embeddings, a bunch of other words light up — and while we can't tell they're clustered because it's a multidimensional space, they all have some relationship to "king": rebels, rulers, monarchs, kingdom. Those relationships are identified through the embeddings. You can even do embedding math — add and subtract numerical values to see interesting relationships.

In the neural network, the connections between nodes are called parameters. For large language models, these are the numerical weights of the connections between nodes — between the input layer and a hidden layer, between hidden layers, and so on. These parameters are very important; they're what we adjust during training, and they determine which neurons get activated based on the weights. When you train a network, you put in an input — an image, or a prompt — and look at the output. If it's correct, you leave it alone. If it's off, you look at how far off it is and adjust all the connections so that next time it's more accurate. That's the process of forward and backward propagation. When I was learning this, I thought of it like adjusting dials — tuning all those connections individually to get more accurate results. Every time you train a language model, you're adjusting these little dials.

Now the data and the training. Where do we get the data? ChatGPT, for example, uses a large collection of books, articles, and web pages — a dataset called Common Crawl, which is publicly available. GPT-2 and GPT-3 went up to around 2019; GPT-4 and the newest models have much better and larger datasets. The larger the dataset, the more accurate the LLM — that seems to be the trend, though some people argue against it.

About parameters: they're the connections between the nodes. One reason people are afraid of LLMs is that the more parameters — the more connections — the more capabilities the model has. Language understanding is there from the beginning, but as you add more data and parameters, it becomes able to do much more. So people worry: if we keep making these things larger, what else might emerge? That doesn't necessarily mean we won't just plateau — but people are amazed that the bigger you make these things, the more capabilities they have, and that's why some are paranoid about them.

Let's talk more about training, which is broken into several levels. The first is pre-training, which is the next-word prediction we discussed: you pass large amounts of text through the network and, depending on the output, adjust the weights through forward and backward propagation, over and over with lots of data. This creates the foundational, or base, model — which doesn't do anything but predict the next word. It doesn't even know how to interact with you yet.

The next level is fine-tuning, where we add the chatbot or assistant behavior. You feed the model datasets of prompts, questions, and typical responses so it gets familiar with how to behave. OpenOrca, for example, is a dataset of question-and-answer prompts, about five gigs. Fine-tuning is why you can interact with it directly and get answers you understand. Another aspect of fine-tuning: you can ask it a question, it gives you two responses, and you pick the better one. That's called reinforcement learning from human feedback (RLHF) — the response you pick gets used in training for future versions.

One interesting part of fine-tuning is alignment, which is a way to encode ethics into the model. Remember, LLMs just do next-word prediction — so if you ask, "Can you tell me how to build a bomb?", by default it looks at all the data it was trained on (essentially the whole internet) and answers based on next-word likelihood. It doesn't care about right or wrong; it just predicts the next word. Alignment gives it safety rails so it doesn't answer questions we don't want it to. It's similar to fine-tuning in that you give it synthetic data — examples of questions it shouldn't answer and how it should respond instead. But this is a very tough problem, because alignment is subjective, based on human morality. You don't want these things teaching people how to build bombs, but you also might not want to over-restrict them, because too many safety rails can degrade the quality of answers. A lot of smart people are working through this problem right now.

One interesting thing about today's models: before this exploded, the common thought was that to make a really helpful chatbot, you had to give it a lot of specialized knowledge first so it would be an expert in its field. But the way these models work now, you train a foundational model on the large, original internet data, and then do the fine-tuning with specialized knowledge afterward. So if you wanted a chatbot specialized in mechanics, you'd start with a good foundational model and then fine-tune it with specialized data on automobiles, engine schematics, and so on — which gives much better results than training only on the specialized data. Same idea for a company like AT&T: train a foundational model on the internet's general knowledge, then fine-tune it with AT&T's proprietary data to make a custom chatbot specialized in AT&T knowledge.

One thing to keep in mind: training is very expensive right now. Here's some data from Llama 3 — over 7 million GPU hours, and 15 trillion tokens consumed (remember, tokens are parsed text). Llama 2, the previous version, was trained on 6,000 GPUs for 12 days at a cost of $2 million. The newer models are far more GPU- and cost-intensive, so it's remarkable that companies like Meta give these tools away for free and open-source them.

Another aspect is prompt design. You've probably seen "prompt engineer" pop up as a new career role last year. In an LLM, you have a context window — the bucket of tokens you can use as input for the model to analyze. A large context window is crucial for prompt engineering: the more context you can put in, the better the answers. On GPT-4, the context window is 32,000 tokens. I associate this with Google dorking — specialized queries in Google that give you more targeted results. If you define and customize your prompts, you'll get much better results. If you provide an example of what you want the output to be, that's called few-shot learning. You can also specify the format — say, "give me the response in markdown." The clearer and more specific you are, the better your results. On ChatGPT you can set custom instructions so that, for example, every response comes back in markdown automatically.

There's a tool by Daniel Miessler called Fabric that creates prompts for you to get better results. For example, it has a prompt that says "You are an expert content summarizer," with output instructions on how the response should be laid out. Miessler has a whole list of these. If you haven't looked into that tool, I'd suggest it — it's a nice resource for writing better prompts.

Then there's RAG — retrieval-augmented generation. One problem with LLMs is that you don't want to send your company data to ChatGPT, because it's owned by another company. You want to use an LLM only with data local to your network, without sending it anywhere. With RAG, you take all the text and files you want, create embeddings out of them, and put them in a database. When you write a prompt, you query that database — "give me all the information related to this topic from your internal documents" — it finds the relevant embeddings and appends them to the prompt as context. Then the LLM looks at the prompt plus the context (your internal documents) and gives a better result. A lot of companies got startup money to solve this, because people don't want to send their data to big companies but still need to use these tools.

LLMs aren't perfect — they have a lot of problems. The biggest is hallucinations. Since they just predict the next word based on statistical analysis, they often lie, and lie confidently — and they don't provide a source, because they're just doing next-word prediction. People are trying to get around that, but it's still a problem. A lot of people complain that these things can't do math, logic, and reasoning, though there's contention there — some say you're just not asking the question the right way. Training is still very expensive. A lot of the training data comes from the internet, so it carries a lot of human bias, which we have to adjust for through fine-tuning and alignment. Many people think LLMs will displace a lot of the workforce, which could be true. In security, we always relied on foreign actors having spelling errors in their phishing — but with these tools, they can write a very clean email template, so that's good news for the bad guys and bad news for us. There's also the issue that a lot of the training data is copyrighted, and the training data has to be constantly updated or it goes stale.

Here are some common attacks against LLMs — I won't go deep, since this talk is more introductory and focused on how they work rather than how to attack them. OWASP put together a list of common attacks. One of the most interesting was from a paper where a user was working with Claude, which was already aligned to restrict certain answers. The user asked, "What tool do I need to cut a stop sign?" — obviously not something we want to answer — and the response was "I cannot give you that information." But the researcher found that if they encoded the question in something like MD5, it would just answer. That's a jailbreak that bypasses the alignment safety rails. It's a difficult problem to stop because it depends on how the model was fine-tuned, so as security folks we're always going to have to deal with jailbreaks.

As for the future of LLMs — and this is changing so quickly that even since I put this presentation together, I feel like I need to update it — I think agents are going to be a really big deal. An agent goes further than just answering a question: if you ask it to do open-source recon on an IP address, an agent will actually execute code with computer resources — for example, run an Nmap scan — and then use those results in the next part of an engagement. It's the ability for LLMs to execute code themselves and take several steps based on the responses. I think that'll be a game-changer across many industries if it's done properly.

Self-improvement is another one — where the training can be done by the LLMs based on the responses they generate, so a good-enough LLM can generate data that's then fed to another LLM for training. And "thinking fast and slow": thinking fast is what we have now — intuitive, quick next-word prediction with no real deliberation. Thinking slow would be asking the LLM to reason, break down problems, and do reflective thinking. A lot of researchers are interested in that right now. It's early, but if we get these things to reason rather than just answer quickly, that would be a big step.

This next concept came from a researcher named Andrej Karpathy — he put together what he calls the "LLM computer," replacing traditional CPU infrastructure with LLMs. It makes a lot of sense now that we've gone over context windows and embeddings: the LLM is your CPU, the context window is your RAM, the file system is your embeddings, the calculator/interpreter/terminal are your inputs and outputs, ethernet is your browser plus other LLMs working together, and the peripherals are video and audio. I thought that was a really interesting way of thinking about it.

Now a little demo about embeddings. This is that multidimensional space — the word is "king." You can see all these other words populating in and out. They're not clustered in a traditional x-y axis; think of it as 200 dimensions in space, so it's hard to visualize, but you can see words related to "king" — ruler, throne, Jerusalem, mercenaries, Sweden. That's where the neural network and the language model get their context for how words interact, which is why they're able to understand sentences and meaning. And since these arrays are just numbers, you can do "word algebra": if you take Paris, minus France, plus Germany, the first result you get is Berlin. Pretty interesting.

The last thing I want to show you: Meta did a really good job training a model called Llama and giving it out to the public for free. This is just running on a regular MacBook — not a GPU-heavy machine. I asked it for a joke, and this is an actual chatbot using a really nice model, for free, that cost millions of dollars to train — and they just gave it out, so you can run it locally on your own computer and ask it all sorts of questions. "Why did the cat join the band? Because it wanted to be a purr-cussionist." There you go — the future is here. I'm going to pass it over to Jack. Thank you all for your time.

Jack Ekelof:

Awesome. We'll open it up for questions to J. R.. We've got one from Carl: "With the Llama 3 model, can you train it further with your own context?"

J. R. Hernandez:

Yes, I believe you can do your own training on the models, as long as you have a good foundational model to start with. That's going to require more GPUs and hardware, and it'll take a long time, but it's possible to train them locally — the more hardware you have, the easier it'll be. People are making their own models and sharing them on sites like Hugging Face. As long as you have a good foundational model to start with, you'll be able to do that and get good results.

Jack Ekelof:

Awesome. With that, we'll wrap things up. Appreciate everyone's attendance today, and we look forward to the next one in September. Thank you.