Normandie Digital

L'actualité du web, du référencement et de l'IA, expliquée pour les TPE et PME.

🇫🇷 FR

How ChatGPT works: mechanics without jargon

ChatGPT does not look up information. It predicts the next word from probabilities learned during training. That is why it writes quickly and why it sometimes invents false but plausible answers.

How it works when you type a question

The core action is repeated: predict the next word. Type “the capital of Portugal is” and the model calculates a probability for every possible word. “Lisbon” comes top, it displays it, then starts again with the longer sentence. A whole answer is built this way, block by block, dozens of times.

The CNIL says generative models “are not knowledge bases: they follow a probabilistic logic, which means they generate only the statistically most likely result given the data on which they were trained” (CNIL). GPT-3, ChatGPT’s ancestor, already weighed 175 billion parameters, ten times more than any previous non-sparse language model (arXiv, 2020). These parameters are tuning knobs adjusted during training. No human wrote them by hand.

Before and during training

The model first “read” massive volumes of text until next-word prediction became very good. It was then fine-tuned to hold useful conversations, follow instructions and refuse some requests. This turns a sentence-completion machine into an assistant.

Distillation dates from March 2015, when Geoffrey Hinton, Oriol Vinyals and Jeff Dean published at Google the article that named the method (arXiv, 2015). A large “teacher” model trains a small “student”. Compact versions cost less per request, answer faster and can sometimes run locally. Labs use this to sell fast models to companies, and some have accused others of using it without their knowledge.

Anthropic identified massive extraction campaigns against its Claude model by three competing labs, DeepSeek, Moonshot and MiniMax, with more than 16 million exchanges generated through around 24,000 fraudulent accounts (Anthropic). The issue now divides US and Chinese labs, with distillation at the centre of the dispute. A “fast and cheap” assistant version is often a student trained by a bigger teacher.

ChatGPT and a search engine

The most common confusion is treating ChatGPT as an improved Google. The two systems do not work in the same way.

Why it invents answers

When information is not in its training data, the model does not say “I don’t know”. It keeps predicting the most probable word, producing a firm, well-written and completely false answer. The CNIL calls this a hallucination and says it happens especially with events after the model’s training (CNIL). According to the same source, excessive confidence without verification leads to wrong decisions. Figures and proper names are the first affected.

Which tool for which need

For writing a letter or rephrasing a muddled paragraph, ChatGPT is made for that. To check a figure, find information dated yesterday or look for the source of a claim, a search engine remains the right tool, because its answer is a list of sources, not a prediction.

A question that mixes both, such as “what changed in regulation this month”, can be answered by an assistant connected to a search tool if that tool is actually activated. Otherwise, the model answers from its training memories, which have an expiry date.

Sources

Raphaël Marieux Mis à jour le 13/11/2026

J'explique le web et l'IA en français, sans jargon ni pub.

Tous ses articles