In This Handbook
What are Large Language Models (LLMs)?
Generative AI isn’t limited to text, but large language models (LLMs) are a cornerstone of the field, powering popular systems like ChatGPT, Bing Chat, Google Bard, and many others. In this lesson, we’ll demystify what an LLM is and how it works at a high level. We’ll see why an LLM can appear to reason or be creative, when in fact it’s following a form of advanced pattern prediction. We’ll also outline how these models are trained (pretraining, fine-tuning, alignment) and discuss their capabilities and limitations. By the end, you should understand why an LLM responds the way it does and what it can or cannot do.
ⓘ LLMs are models, not products
A large language model is a trained neural network that generates text. Systems like ChatGPT are full products built around an LLM. These products include interfaces, safety layers, memory handling, tool access, and system prompts. When we say “LLMs can do X,” we often mean the combined system, not the raw model alone.
LLMs as next-token predictors
Large language models are giant next-word prediction machines. This sounds almost too simple to be true given their abilities, but it really is the fundamental mechanism. An LLM takes some input text (called the prompt or context) and then predicts the most likely next token (a token is typically a word or piece of a word) to continue that text. It then predicts the next token after that, and so on, generating a sequence of tokens that hopefully forms a coherent and relevant response. If you’ve ever used the autocomplete feature while texting (where your phone suggests the next word), you’ve seen a primitive version of this idea. LLMs take it to another level entirely by using huge neural network models trained on billions of examples of text sequences, so they’ve learned very sophisticated patterns of language.
How does an LLM “know” what the next word should be?
During training (more on that in upcoming lesson), the model was shown countless sentences and passages and trained to fill in missing words or continue the text. For example, it might see “The capital of France is ___” and it learns that “Paris” is a highly likely completion. Over time, by doing this for essentially the entire internet and libraries of books, the model encodes probabilistic associations between words. When you prompt it later, it uses those learned associations to generate a continuation. If you prompt it with “Once upon a time, there was a princess who”, the model’s internal knowledge of fairy tales makes it likely to continue with something like “lived in a castle” because that phrase often follows. The key is that the model isn’t retrieving a stored answer; it’s constructing an answer on the fly by rolling out one token at a time, each chosen based on probabilities.
LLMs generate the next token based on a probability distribution. At each step, the model calculates the probability for thousands of possible next tokens given the context (e.g., “The CEO of Apple is” might yield 80% for “Tim”). It then selects a token, often the most probable one, but sometimes a lower-probability one is chosen through sampling with randomness to create varied output. This is like choosing from a weighted menu. The temperature setting controls this randomness: high temperature means more creative, less deterministic output (selecting less likely tokens), while zero temperature is deterministic (always picking the top candidate).
ⓘ Sampling Is not random guessing
Even when randomness is introduced, the model samples only from tokens it considers plausible in context. High-probability tokens dominate the distribution, and unlikely or incoherent tokens are extremely rare. Creativity comes from exploring reasonable alternatives, not from uninformed guesses.
The next-token prediction framework of Large Language Models (LLMs) is incredibly powerful when scaled. By prompting an LLM correctly, this next-word predictor can be “tricked” into performing complex tasks like answering questions, writing code, or translating. It acts like an improv actor who continues the scene you set. This allows LLMs to handle diverse tasks without specific programming; the prompt shapes the task, and the model simply predicts the next text.
ⓘ Not all prompts are visible to you
LLMs process tokens (subwords or characters) rather than whole words, converting them into numeric representations (embeddings). They use the Transformer neural network architecture, which is key to considering broad context when predicting the next token, not just the last few words. The amount of text an LLM can consider at once is its context window (ranging from thousands to 100k+ tokens), setting a limit on the input or ongoing conversation it can effectively handle.
Why LLMs appear to reason
LLMs appear to reason or understand by breaking down problems step-by-step, not through built-in logic, but through emergent capabilities from massive training data. By predicting text, the model absorbs patterns for problem-solving, essentially imitating the form of human reasoning, which often yields correct results.
This ability stems from storing vast world knowledge in their parameters. The model “knows” facts and structures (like logic or programming) statistically, as a byproduct of minimizing prediction error. This implicit learning allows them to perform tasks they weren’t explicitly trained for, with emergent abilities appearing suddenly as models scale up.
ⓘ Why scale matters
As models grow in size and are trained on more diverse data, they can represent more complex patterns and abstractions. This allows them to generalize across tasks without being explicitly trained for each one.
However, this is pattern-based generalization, not robust, grounded reasoning. LLMs lack a stable internal model of cause and effect, leveraging only correlations. They can fail with slight variations or unfamiliar problems, revealing their apparent reasoning is brittle. They mimic the logic present in human-written text.
Ultimately, coherence emerges because generating incoherent babble is statistically low-probability, having seen countless examples of coherent text.
Training preview: Pretraining, fine-tuning, alignment
How do we get an LLM in the first place? We’ll cover the training process in depth in Lesson 4, but let’s sketch it briefly to understand what “large” means and how these models are built:
- Pretraining: The initial, main stage is pretraining. The model, with random weights, is fed a vast corpus of text (trillions of words from sources like Wikipedia, books, and websites). It learns to predict the next token, adjusting its weights over iterations. This process builds a general understanding of grammar, facts, and style, enabling it to generate realistic and reasonable text continuations for almost any prompt. However, a purely pretrained model lacks the ability to follow explicit instructions; for example, a prompt like “Translate this sentence to French:” might not be obeyed because it was only taught language flow, not command adherence.
- Fine-tuning (Instruction tuning): This is a second training phase on a specific dataset of prompts and responses (like Q&A and instructions) to make a pretrained model useful for interactive tasks. This process taught models like InstructGPT and ChatGPT how to be helpful assistants by following instructions (e.g., “Write a poem”). Though it still uses next-token prediction, the curated data biases the model toward desirable responses (e.g., polite, concise). Instruction fine-tuning significantly improves the model’s responsiveness to users.
- Alignment (RLHF): Alignment techniques, like the standard Reinforcement Learning from Human Feedback (RLHF), adjust instruction-tuned models to human preferences and values to prevent undesirable outputs (e.g., toxic or incorrect). RLHF uses human-ranked model outputs to train a reward model. The language model is then optimized via reinforcement learning to generate outputs that maximize this reward score, effectively “nudging” the model (like ChatGPT) toward desired, aligned behavior (factual, non-offensive, compliant).
After pretraining on text, fine-tuning for instructions, and alignment for human preferences, the resulting model can be deployed as a chatbot. Not all LLMs follow these steps identically. Proprietary models like OpenAI’s GPT-4 use unpublicized methods, likely including these elements. Many open-source LLMs (e.g., LLaMA, Falcon) are released pretrained, with the community handling subsequent fine-tuning and alignment (hence variants like “LLaMA2-chat”). While details vary, these stages form the core conceptual process.
What LLMs can do
Large Language Models (LLMs) have diverse applications, all stemming from their ability to predict the next token based on extensive training data:
- Answering questions: Provide detailed answers to factual or explanatory queries, often matching encyclopedia quality, though accuracy can be limited by the model’s knowledge cutoff.
- Creative writing/Content generation: Create stories, poems, essays, or lyrics in various styles by mimicking the literature ingested during training.
- Summarization: Condense long texts into shorter summaries by extracting key points, which is highly useful for quick information extraction.
Translation: Translate between many languages effectively by leveraging bilingual text seen during training. - Coding assistance: Generate, explain, and debug code, accelerating developer workflows by using programming literature from its training data.
- Reasoning and planning: Attempt logic puzzles, math problems, and multi-step reasoning, with capabilities improving significantly in larger models like GPT-4, though they can still make errors.
- Dialog and role play: Excel at multi-turn conversations and role-playing, useful for chatbots and virtual assistants, by maintaining context and adapting tone.
Crucially, these versatile capabilities are all a product of next-token prediction applied to broad training data, allowing the LLM to emulate human tasks without true understanding or empathy.
Limitations and challenges of LLMs
While LLMs can do a lot, they also have significant limitations you should be aware of:
- Hallucinations: They confidently generate false or nonsensical information because their goal is plausible text, not guaranteed truth. This is a major challenge for critical applications as there is no built-in reality check.
- Lack of true understanding: LLMs only know language, not the physical world, leading to common-sense errors (e.g., impossible spatial reasoning) and a lack of memory beyond the current context window.
- Context window limit: They can only process a finite amount of text at once (tokens). Older or earlier parts of very long inputs/conversations are dropped, requiring summarization for long interactions.
ⓘ LLMs Are stateless
By default, an LLM does not remember past conversations or users. Any “memory” you observe is either within the current context window or implemented by the surrounding system. Persistent memory requires external storage and explicit design.
- Biases: They reflect biases, stereotypes, and offensive content from their training data (the internet). Even with alignment tuning, subtle biases can persist, requiring careful development and user awareness.
- Inability to express ignorance: LLMs tend to invent answers rather than saying “I don’t know” when asked about information outside their training or knowledge cutoff, unless explicitly prompted to express uncertainty or supplemented with external data.
ⓘ Knowledge cutoff
LLMs are trained on data only up to a certain point in time, known as the knowledge cutoff. They have no awareness of events, updates, or facts that occurred after this cutoff unless connected to external tools or live data sources.
- Long-term inconsistency: In very long outputs (like complex plans or narratives), they can lose coherence, contradict themselves, or make logical errors that a human would catch by reviewing steps.
- Sensitivity to wording (Prompt engineering): Output quality is highly sensitive to the exact phrasing of the prompt, making it a trial-and-error process to get the best result.
Despite these issues, LLMs are powerful tools for tasks like drafting text or explaining code. The key to safe and effective use is understanding these weaknesses, verifying critical information, and using them with human oversight or augmented with external data.
Before we move on, let’s touch on some visual concepts mentioned for this lesson:
- Context window: Like a sliding window, the model’s “attention span” covers a set number of tokens (e.g., 4,000). Only the text within this window influences the next output; older details “slide out,” causing the AI to forget early information in long chats. Newer models have larger windows (e.g., 32k). Long inputs are often summarized or chunked to fit.
- Probability distribution: A chart where each potential next word has a probability bar. For “The capital of France is,” “Paris” has the tallest bar. The model samples from this. High-confidence predictions mean one bar is dominant. In creative writing, the distribution is flatter. Parameters like “temperature” adjust this distribution, controlling the model’s creativity vs. focus.
- Learning patterns: An LLM learns by mapping language structure in a high-dimensional space, clustering common phrases. It stores a complex web of connections representing linguistic and semantic patterns, not whole sentences. When prompted, it navigates this learned “map” to produce a likely path (sentence) connecting the input ideas.
To learn more about how LLMs work, check out our course on large language models, where we break down real-world applications and hands-on deployment strategies.
Now that we’ve got a handle on what LLMs are and the broad strokes of how they function, let’s move on to using them effectively. In the next lesson, we’ll discuss prompting, which is the art of telling LLMs what we want in the right way, which is key to steering these models.
I’m Fahim, a software engineer specializing in generative AI, system design, and responsible AI adoption.