From vectors to Transformers

4 lessons are available. More lessons will follow.

IFoundational mathematics

IINeural networks

07The structure of a neuron
08Activation functions
09Neural networks and training
10Loss functions
11Softmax
12Gradient descent and optimisers
13Backpropagation

IIINatural language processing

14The probability game of language: N-grams
15Word vectors
16Feedforward neural language models
17Recurrent neural networks
18Long short-term memory

IVLarge language models

19Attention
20Multi-head attention
21The Transformer architecture
22Tokenizers
23Encoders, decoders and large language models
24Residual connections and layer normalisation
25Pretraining, supervised fine-tuning and reinforcement learning
26KV caching, sparse attention and FlashAttention
27Mixture of experts
28Model distillation
29From N-grams to Transformers: a recap
30Frontiers and the road ahead

VAppendices

A1A panoramic view of the Transformer
A2Attention Is All You Need: the original paper

Course chapters

From vectors to Transformers

4 lessons are available. More lessons will follow.

IFoundational mathematics

IINeural networks

07The structure of a neuron
08Activation functions
09Neural networks and training
10Loss functions
11Softmax
12Gradient descent and optimisers
13Backpropagation

IIINatural language processing

14The probability game of language: N-grams
15Word vectors
16Feedforward neural language models
17Recurrent neural networks
18Long short-term memory

IVLarge language models

19Attention
20Multi-head attention
21The Transformer architecture
22Tokenizers
23Encoders, decoders and large language models
24Residual connections and layer normalisation
25Pretraining, supervised fine-tuning and reinforcement learning
26KV caching, sparse attention and FlashAttention
27Mixture of experts
28Model distillation
29From N-grams to Transformers: a recap
30Frontiers and the road ahead

VAppendices

A1A panoramic view of the Transformer
A2Attention Is All You Need: the original paper

Notes / AI & large language models

From vectors to Transformers

A step-by-step introduction to the ideas behind large language models.

4 lessons are available. The remaining lessons are listed below and are not available yet.

Start with What is a vector? →

IFoundational mathematics

01
What is a vector? →
02
Common vector operations →
03
What is a matrix? →
04
What is a linear transformation? →
05
What is a gradient?
Not available yet
06
Probability and information
Not available yet

IINeural networks

07
The structure of a neuron
Not available yet
08
Activation functions
Not available yet
09
Neural networks and training
Not available yet
10
Loss functions
Not available yet
11
Softmax
Not available yet
12
Gradient descent and optimisers
Not available yet
13
Backpropagation
Not available yet

IIINatural language processing

14
The probability game of language: N-grams
Not available yet
15
Word vectors
Not available yet
16
Feedforward neural language models
Not available yet
17
Recurrent neural networks
Not available yet
18
Long short-term memory
Not available yet

IVLarge language models

19
Attention
Not available yet
20
Multi-head attention
Not available yet
21
The Transformer architecture
Not available yet
22
Tokenizers
Not available yet
23
Encoders, decoders and large language models
Not available yet
24
Residual connections and layer normalisation
Not available yet
25
Pretraining, supervised fine-tuning and reinforcement learning
Not available yet
26
KV caching, sparse attention and FlashAttention
Not available yet
27
Mixture of experts
Not available yet
28
Model distillation
Not available yet
29
From N-grams to Transformers: a recap
Not available yet
30
Frontiers and the road ahead
Not available yet

VAppendices

A1
A panoramic view of the Transformer
Not available yet
A2
Attention Is All You Need: the original paper
Not available yet