Book 08 · AI & Machine LearningPage 8 of 52

Chatbot

Updated October 3, 20262 min read

In short

A chatbot is a program that converses with people in text or speech, answering questions or helping with tasks, using scripted rules or a language model.

What is a chatbot?

One of the first chatbots was ELIZA, written at MIT in the mid-1960s by Joseph Weizenbaum. It matched keywords and turned the user's sentences back into questions, and people were surprised by how human it seemed. For decades afterwards most chatbots worked the same way: rules, keyword matching and decision trees that led the user through fixed menus.

Modern chatbots such as ChatGPT, Claude and Gemini are built on large language models. Instead of picking a scripted answer, they generate a reply word by word from the whole conversation so far, which lets them handle questions nobody planned for, write and explain code, and keep the context of a long exchange.

Businesses use chatbots for customer support, booking, internal help desks and searching documentation. A useful production chatbot usually combines the language model with a system prompt that sets its role and limits, retrieval of the company's own documents (RAG) so it answers from real facts, and tool calls so it can look up an order or open a ticket.

A common misconception is that a chatbot knows or checks what it says. A language model predicts likely text, so it can state wrong things confidently, an error called hallucination. Grounding answers in trusted documents, showing sources and handing over to a person for important cases make chatbots far more reliable.

Key takeaways

A chatbot holds a conversation in text or speech.
ELIZA, from the mid-1960s, was one of the first chatbots.
Older chatbots follow scripts; modern ones generate replies with an LLM.
Production chatbots add a system prompt, retrieval (RAG) and tool calls.
They can hallucinate, so grounding and human handover matter.

Example

A minimal chatbot loop that keeps the conversationpython

import anthropic

client = anthropic.Anthropic()
history = []

while True:
    history.append({"role": "user", "content": input("You: ")})
    reply = client.messages.create(
        model="claude-sonnet-5-5",
        max_tokens=500,
        system="You are a friendly support assistant for a bookshop.",
        messages=history,  # the whole conversation, so the bot keeps context
    )
    text = reply.content[0].text
    history.append({"role": "assistant", "content": text})
    print("Bot:", text)

Readers ask

What is the difference between a chatbot and an AI agent?

A chatbot mainly talks: it answers questions in a conversation. An AI agent works toward a goal on its own, planning steps and using tools such as search, code or APIs. Many modern assistants are both.

Is ChatGPT a chatbot?

Yes. ChatGPT is a chatbot built on OpenAI's GPT language models. It was released in November 2022 and made LLM-based chatbots widely known.

How do chatbots remember the conversation?

Usually the application sends the earlier messages along with each new one, so the model sees the whole exchange. The model itself does not remember between requests, and very long conversations are limited by its context window.

Often compared

Chatbot vs AI AgentA chatbot converses and answers questions, while an AI agent pursues a goal on its own, planning steps and using tools such as search, code execution or APIs.