• Foundation
  • About 15 min
  • 5 practice questions

Neural networks, deep learning and generative AI

Explain how a neural network is organised, what makes learning 'deep', how generative AI and large language models produce content, and why they can be biased or wrong.

If something looks wrong, please tell us.

The AI that writes text, draws pictures from a sentence and understands speech is built on neural networks. These tools are powerful, but they also make confident mistakes and can repeat unfair patterns from their data. To judge them sensibly - and to answer exam questions on them - you need to know how they work in outline.

Artificial neural networks

An artificial neural network is a computer model loosely inspired by the brain. It is made of simple units called neurons arranged in layers: an input layer that receives the data (for example the pixels of a photo), one or more hidden layers that transform it, and an output layer that gives the answer ('cat'). Each connection has a number called a weight. During training the weights are adjusted again and again so that the outputs become more accurate.

Deep learning

Deep learning is machine learning with neural networks that have many hidden layers - that is what 'deep' means. Early layers pick up simple features such as edges; later layers combine them into shapes, faces or meanings. Deep learning works best with very large amounts of data and a great deal of computing power, which is why it advanced quickly once powerful graphics processors (GPUs) and huge digital data sets became available.

From the widest idea to the narrowest
TermRelationship
Artificial intelligenceThe broad field of machines doing intelligent tasks
Machine learningA part of AI: learning patterns from data
Deep learningA part of machine learning: neural networks with many layers
Generative AIModels, usually deep learning, that create new content

Generative AI and large language models

Most AI you met in the last lesson recognises or predicts (is this spam?). Generative AI creates new content - text, images, audio, video or computer code - in response to an instruction called a prompt. A large language model (LLM) is a generative model trained on a very large amount of text. At heart it learns to predict the next word (more precisely, the next small piece of text, called a token) given the words before it. Repeating this prediction word after word produces whole answers, essays or programs.

Worked example 1: prediction is not looking up facts

Give a language model the words 'The capital of France is'. From patterns in its training text, 'Paris' is by far the most likely next word, so it writes 'Paris'. Now ask about an obscure book. If its training text said little about that book, the model still produces the most plausible-sounding words - perhaps an author or date that looks right but is invented. A confident, fluent answer that is false is called a hallucination. The model is predicting likely text, not checking a database of facts, so its answers must be verified.

Worked example 2: how bias gets in

Suppose a company trains a model to shortlist job applicants using its past hiring records, and in the past most people hired for a role were men. The model may learn that being male goes with being hired and start ranking women lower, even though nobody told it to. The fault lies in the training data: a model learns whatever patterns the data contains, fair or unfair. Checking the data and testing results separately for different groups helps catch such bias.

Deep learning = neural networks with many layers. A language model predicts the next word; fluent is not the same as true. A model is only as fair and accurate as its training data.

Deepfakes

Generative AI can also produce deepfakes: realistic fake videos, images or voices showing a real person saying or doing things they never did. They can be used for fraud, harassment and spreading false news, so it is wise to check the source before trusting a striking clip.

Key takeaways: a neural network learns by adjusting the weights between layers of neurons; deep learning uses many layers and needs lots of data and computing power. Generative AI creates new content, and large language models do it by predicting the next word. Their limits - hallucination, bias from training data and misuse such as deepfakes - are as important as their abilities.

Check your understanding

  1. 1. 'Deep learning' refers to:

    1. Computers that store data deep underground for safety
    2. Machine learning using neural networks with many layers
    3. Learning that happens only in the deep sea research field
    4. A program that a person writes rule by rule
    Show answer

    Answer: B. The 'deep' in deep learning means many hidden layers in a neural network. It is a branch of machine learning, not a storage method or rule-writing.

  2. 2. At its core, a large language model is trained to:

    1. Look up every answer in a fixed database of verified facts
    2. Copy whole web pages exactly as they were written
    3. Translate only between two fixed languages
    4. Predict the next word or token from the text before it
    Show answer

    Answer: D. A large language model learns patterns from a huge amount of text and generates answers by repeatedly predicting the next token. It does not consult a verified fact database, which is why it can be wrong.

  3. 3. A chatbot confidently gives a reference to a book that does not exist. This problem is called:

    1. Hallucination
    2. Encryption
    3. Overclocking
    4. Compression
    Show answer

    Answer: A. A fluent, confident but false or made-up output from a generative AI model is called a hallucination.

  4. 4. A face-recognition model trained mostly on photographs of one group of people works poorly for others. The main cause is:

    1. The speed of the internet connection
    2. The colour of the computer's screen
    3. Unrepresentative training data
    4. The model being too new
    Show answer

    Answer: C. A model learns from its training data. If some groups are under-represented, it performs worse on them. This is a form of bias that comes from the data.

  5. 5. Which of these is an example of generative AI?

    1. A calculator adding two numbers
    2. A system that creates a new picture from a written description
    3. A thermostat switching off a heater at 25 °C
    4. A spreadsheet sorting names alphabetically
    Show answer

    Answer: B. Generative AI creates new content from a prompt, such as a picture made from a text description. The others follow fixed rules and create nothing new.

Privacy · Terms · Refunds · Delivery