- Foundation
- About 15 min
- 5 practice questions
Neural network basics: neurons, weights, layers and how training works
Describe an artificial neuron, its inputs and weights, name the layers of a neural network, explain in plain words how training adjusts the weights, and say why large data and GPUs made deep learning possible.
If something looks wrong, please tell us.
By the end of this lesson you can
- Describe an artificial neuron and the job of its inputs, weights and output.
- Name the input, hidden and output layers and say what 'deep' means.
- Explain in plain words how training adjusts weights, and why data and GPUs mattered.
Most of the AI that recognises faces, understands speech and translates languages runs on artificial neural networks. The mathematics can be heavy, but the idea is simple enough to follow with a monsoon umbrella - no equations needed.
Inspired by the brain, not a copy
The human brain has billions of nerve cells, called neurons, connected to one another. A neuron receives signals from others and, if the signals are strong enough, passes a signal on. An artificial neural network borrows only this rough idea: many simple units, connected in layers, each passing a number to the next. It is a mathematical program running on a computer. It does not think or feel like a brain.
One artificial neuron
- Inputs - numbers coming in, such as the brightness of pixels or answers like yes (1) and no (0).
- Weights - one number for each input, showing how much that input matters. A large weight means the input counts for a lot.
- Weighted sum - each input is multiplied by its weight and the results are added. Many networks also add a fixed extra number called a bias.
- Activation - the sum is passed through a simple rule, for example 'output 1 if the sum is above a threshold, otherwise 0'.
- Output - the number sent on to the next layer, or the final answer.
Worked example 1: an umbrella neuron
Suppose a neuron decides 'take an umbrella?'. Input 1: the sky is cloudy (1 or 0), weight 0.3. Input 2: the forecast says rain (1 or 0), weight 0.7. Threshold: 0.6. Day A, cloudy and rain forecast: 1 × 0.3 + 1 × 0.7 = 1.0, above 0.6, so take the umbrella. Day B, cloudy but no rain forecast: 1 × 0.3 + 0 × 0.7 = 0.3, below 0.6, so no umbrella. Day C, clear sky but rain forecast: 0 + 0.7 = 0.7, above 0.6, so take it. The larger weight on the forecast means it matters more than the clouds. (All numbers are illustrative.)
Layers
| Layer | What it does | In a network that recognises animals in photos |
|---|---|---|
| Input layer | Receives the raw data | One input for each pixel value of the photo |
| Hidden layers | Combine the signals step by step to find patterns | Early layers pick out edges, later ones shapes, then parts such as ears and eyes |
| Output layer | Gives the answer | A score for each label, such as 'cat' or 'dog' |
Each neuron in one layer sends its output to neurons in the next, and every connection has its own weight. A network with many hidden layers is called a deep neural network, and machine learning with such networks is deep learning. More layers let a network learn complex patterns from simple ones.
Layers of a neural network
- Input layer Receives the raw data.
- Hidden layers Combine the signals step by step to find patterns; many make a network deep.
- Output layer Gives the answer.
How training adjusts the weights
- Start with weights set to small random numbers. The network's answers are mostly wrong.
- Show it a labelled example, such as a photo marked 'cat'.
- The network gives its answer, for example 'dog'.
- Measure the error: how far the answer is from the correct label.
- Nudge each weight a little in the direction that reduces the error. The standard method for working out these nudges layer by layer, from the output back towards the input, is called backpropagation.
- Repeat over thousands or millions of examples, many times over, then test on examples the network has never seen.
Nobody types in the weights; training finds them, and the trained weights are the model. That is how a network recognises a cat although no one wrote a rule describing one.
How training adjusts the weights
- Show a labelled example Such as a photo marked 'cat'.
- Network gives its answer
- Measure the error How far the answer is from the correct label.
- Nudge the weights A little, to reduce the error (backpropagation).
Then back to step 1: Show a labelled example
Worked example 2: one training nudge
Take the umbrella neuron (cloudy 0.3, forecast 0.7, threshold 0.6). On one day the sky was cloudy, there was no rain forecast, and it did rain. The neuron's sum was 0.3, so it said 'no umbrella' - an error. Training raises the cloudy weight slightly, say by 0.2 to 0.5. The sum is now 0.5, still below 0.6, still wrong. Another nudge of 0.2 gives 0.7, above 0.6, and the answer is now right. But one day proves little: if most cloudy days stay dry, other examples push the weight back down. That is why training uses many examples, not one.
Why data and GPUs mattered
The idea of neural networks is decades old, but deep networks became practical only from around the 2010s. Three things came together:
- Data - huge amounts of digital text, images and speech from the internet and phones, needed because a network with very many weights must see very many examples.
- Computing power - graphics processors (GPUs), first built to draw computer graphics such as video games, do many simple calculations at the same time, which is exactly what training needs.
- Better methods - improved ways to train networks with many layers.
Neuron: multiply each input by its weight, add up, compare with a threshold. Training: compare the answer with the label, nudge the weights to reduce the error, repeat. Deep = many hidden layers.
Key takeaways: a neuron multiplies inputs by weights, adds them and applies an activation rule. Networks have input, hidden and output layers; 'deep' means many hidden layers. Training adjusts the weights to reduce error. Large data, GPUs and better methods made deep learning work.
Check your understanding
-
1. In an artificial neural network, a weight is:
- The physical mass of the computer that runs the network
- The number of layers in the network
- The size of the training dataset
- A number on a connection that sets how much an input counts
Show answer
Answer: D. Each connection carries a weight. The input is multiplied by it, so a large weight makes that input count for more. Weights are what training adjusts.
-
2. A neuron has inputs 1, 1 and 0 with weights 0.2, 0.5 and 0.4, and fires if the weighted sum is above 0.6. What is the sum, and does it fire?
- 0.7, and it fires
- 1.1, and it fires
- 0.7, and it does not fire
- 0.5, and it does not fire
Show answer
Answer: A. Weighted sum = 1 × 0.2 + 1 × 0.5 + 0 × 0.4 = 0.2 + 0.5 + 0 = 0.7. Since 0.7 is above 0.6, the neuron fires.
-
3. What happens during the training of a neural network?
- Programmers type in the value of every weight by hand
- The network's layers are removed one by one
- The network's answers are compared with the correct labels and the weights are adjusted to reduce the error
- The network memorises the test data before it is tested
Show answer
Answer: C. Training shows labelled examples, measures how wrong each answer is and nudges the weights to reduce that error, over many examples. Weights are found by training, not typed in.
-
4. A neural network is called 'deep' when it has:
- A very expensive computer
- Many hidden layers between the input and output layers
- Only one neuron
- No output layer
Show answer
Answer: B. 'Deep' refers to the number of hidden layers. Many layers let the network build complex patterns from simple ones - edges, then shapes, then whole objects.
-
5. Why did deep learning take off only from around the 2010s, although neural networks are an old idea?
- Neural networks were invented only in the 2010s
- Computers stopped needing electricity
- Laws before then banned neural networks
- Large amounts of digital data, GPUs that do many calculations at once and better training methods came together
Show answer
Answer: D. Deep networks have very many weights, so they need huge data and huge numbers of calculations. Internet-scale data, GPUs working in parallel and improved training methods made them practical.
Sign in to keep track of the lessons you finish.