Few-Shot Learning
In short
Few-shot learning is getting an AI model to perform a task from just a handful of examples, most often by placing a few sample inputs and outputs in the prompt.
What is few-shot learning?
Few-shot learning means teaching a model a task with only a few examples instead of thousands. With large language models, this is usually done through the prompt: you include two to five sample inputs with their correct outputs, then add the new input, and the model follows the pattern. This ability, also called in-context learning, became widely known in 2020, when researchers showed that large models could pick up new tasks this way without any retraining.
The terms describe how many examples you provide. Zero-shot means you give only instructions, one-shot means a single example, and few-shot means several. The model's weights never change: the examples only guide that one request, so they must be sent again every time and they take up space in the context window. Good examples are short, consistent in format, and varied enough to cover the tricky cases, since the model tends to copy whatever patterns they show, including mistakes and biases.
It is like showing a new colleague two filled-in expense forms before asking them to fill in a third; they understand the format far faster than from a written description alone. Few-shot prompting is common for classification, data extraction, converting text into a fixed format, and matching a particular writing style. In classic machine learning, few-shot learning also names a research area in which models learn to recognize a new category, such as a new product in photos, from just a few labeled images.
Few-shot learning is often confused with fine-tuning. Fine-tuning trains the model on many examples and permanently changes its weights, while few-shot prompting leaves the model unchanged and only works while the examples are in the prompt. It is also different from chain-of-thought prompting, which asks for reasoning steps; the two can be combined by writing examples that include the reasoning.
Key takeaways
- Few-shot learning uses a handful of examples to show a model what to do.
- With LLMs, the examples go in the prompt and the model's weights don't change.
- Zero-shot uses no examples, one-shot uses one, and few-shot uses several.
- Examples should be short, consistent, and representative of real inputs.
- Fine-tuning is the alternative when you have many examples and need lasting behavior.
Example
// Few-shot prompt: two worked examples, then the real input
const messages = [
{ role: "system", content: "Classify the sentiment as positive, negative, or neutral." },
{ role: "user", content: "The update fixed every crash. Love it!" },
{ role: "assistant", content: "positive" },
{ role: "user", content: "The app logs me out every five minutes." },
{ role: "assistant", content: "negative" },
{ role: "user", content: "Checkout now takes longer than before." }, // real input
];
// callModel is a placeholder for a real model client
const label = await callModel(messages); // "negative"Readers ask
What is the difference between zero-shot and few-shot prompting?
Zero-shot prompting gives the model only instructions, while few-shot prompting adds several worked examples of inputs and expected outputs. Few-shot usually gives more consistent formats and better accuracy on unusual tasks, at the cost of a longer prompt.
How many examples should a few-shot prompt include?
Usually two to five are enough. Add more only if tests show they help, because every example uses tokens on every request and too many similar examples can make the model overly rigid.
Is few-shot learning the same as fine-tuning?
No. Few-shot examples live in the prompt and affect only the current request, while fine-tuning trains the model on examples so its weights change permanently.
See also
- Prompt EngineeringAI & Machine Learning, p. 36Prompt engineering is the practice of designing, testing, and refining the instructions given to an AI model so it produces accurate, consistent, useful output.
- PromptAI & Machine Learning, p. 35A prompt is the input text or instructions you give an AI model, such as an LLM, to tell it what task to perform and what kind of answer you want.
- Chain-of-Thought PromptingAI & Machine Learning, p. 7Chain-of-thought prompting is a technique that asks an LLM to reason through intermediate steps before its final answer, improving accuracy on complex tasks.
- Fine-tuningAI & Machine Learning, p. 19Fine-tuning is the process of taking a pretrained machine learning model and training it further on a smaller, specific dataset to adapt it to one task.
- LLMAI & Machine Learning, p. 25An LLM is a machine learning model trained on huge amounts of text that generates language by repeatedly predicting the next most likely piece of text.
- Context WindowAI & Machine Learning, p. 12A context window is the maximum amount of text, measured in tokens, that an LLM can consider at once, including the prompt, conversation history, and its reply.
Spotted a mistake or something missing on this page?Suggest an edit