← The Basics of Generative AI — Know it, understand it and take your first steps in prompt engineering

Module 2 / 4 · 1 h

Understanding how it works

Prediction, training data, limits: understand in order to trust better… and to doubt better.

Log in to save your progress. Log in

1 / 4 · 12 min

Predicting the next word

A language model does not “know” in the human sense: at each step it computes which piece of text is the most likely given what precedes. These pieces are called tokens (a word, part of a word, a sign).

By chaining thousands of predictions, the model produces fluent and often relevant sentences. But fluent does not mean accurate.

2 / 4 · 10 min

Training data

The model learns from very large volumes of text. Three practical consequences follow:

  1. Its knowledge has a cut-off date: it may ignore recent news.
  2. It reflects the biases in its data (stereotypes, over-representation of some languages or cultures).
  3. It does not always cite its sources, and may invent some.

3 / 4 · 12 min

Limits: hallucinations and errors

A hallucination is a plausible but false answer, delivered with confidence. It is more frequent when the question is specialised, ambiguous or about precise facts (dates, figures, references, quotations).

Habit to adopt: for any important fact, cross-check with a reliable source before using it.

4 / 4

An example of hallucination

Example files

question.txt
Cite three scientific articles from 2021 on the impact of remote work, with full references.
reponse-plausible-mais-fausse.txt
1. Dupont, M. (2021). Remote work and productivity. Management Review, 45(2), 112-130.
(This reference is made up: it looks credible but does not exist.)

Never enter personal or confidential data into a public generative AI tool.

Quick test: the limits

Pass mark: 60%. Tick every right answer.

Log in to save your progress.

Module quiz

Pass mark: 70%. Tick every right answer.

Log in to save your progress.