How a language model produces text

When you finish, you can describe in plain words how a language model turns your question into an answer.

Predicting the next piece

A large language model, or LLM, is trained on a huge amount of written material. During training it practices one skill again and again: given some text, guess what comes next. After a very large amount of practice, it gets good at continuing text in ways that fit the pattern.

When you type a question, the model does not hunt for a stored reply. It reads your words, then picks a likely next piece of text, then another, then another, until the answer is complete. Each small choice leans on everything before it, including your question and the earlier parts of its own reply.

Tokens and the context window

Models do not read whole words the way you do. They break text into small chunks called tokens. A token might be a whole short word or part of a long one. You do not need to count them. What matters is that a model can only keep a limited amount of text in view at once. That limit is called the context window.

If a chat gets very long, early details can slip out of view or get less attention. If your assistant seems to forget something you said earlier, this is often why. The fix is easy: restate the key facts, or start a fresh chat with a short summary.

Why answers change

Most tools add a little randomness when choosing the next piece. That is why asking the same thing twice can give different wording, or sometimes different details. This is normal behavior, not a glitch. It also means a single answer is one sample, not the final word.

A useful habit follows from this. If a detail matters, ask again in a new chat, or ask the tool to double-check itself. If the details agree every time, that is mildly reassuring. If they change, that is a flag to verify elsewhere.

Training versus your chat

The model learned its general knowledge during training, which ended at some point in the past. Your chat is different. It is the live conversation where you give new facts, files and instructions. The model can use what you put in the chat, even though it never studied it during training.

This is why giving context works so well. Compare a weak request, “Help me with my presentation,” with a stronger one: “I am presenting to eight new volunteers at a food bank in Manila on Thursday. Suggest a five-slide outline that explains how to sort donations safely.” The second gives the model real material to continue from, so the answer fits your life.

Practice now

Ask a chat tool the same open question twice in two separate chats, such as “Suggest three names for a neighborhood book club.” Compare the answers and note what stayed the same and what changed.
In a single chat, give three facts about yourself near the start. Then chat for several turns about other things and ask the tool to recall the three facts. See whether it still has them.

Use any AI assistant you have. Do not paste private, client or patient information into it.

Check yourself

1. What is the core skill a language model practices during training?

2. Your assistant seems to forget a detail you gave at the start of a very long chat. What is a likely reason?

3. Why can the same question get different answers each time?

Ask the AI tutor

Ask anything about this lesson, in any language. The tutor is an AI: it can be wrong, and it never needs your personal details.

Next lesson: Why AI makes things up, and how to catch it

AI-drafted. Checked by an automated validator and a separate AI reviewer; a person has not read every lesson. Translations and tutor answers are AI-made. Found a mistake? Tell us. AI can be wrong. General information only, not professional advice. Not affiliated with any AI company named. Current University is not accredited.