8
Backpropagation
dlfc_notes
Preface
1
The Deep Learning Revolution
2
Probabilities
3
Standard distributions
4
Single layer networks: regression
5
Single layer networks: classification
6
Deep Neural Networks
7
Gradient Descent
8
Backpropagation
9
Regularisation
10
Convolutional networks
11
structured distributions
12
Transformers
13
Graph neural networks
14
Sampling
15
Discrete latent variables
16
Continuous latent variables
17
Generative adversarial networks
18
Normalizing Flows
19
Autoencoders
20
Diffusion Models
References
Table of contents
8.1
Evaluation of gradient
8.1.1
Single layer networks
8.1.2
General feed-forward networks
8.1.3
A simple example
8.1.4
Numerical differentiation
8.1.5
The Jacobian matrix
8.1.6
The Hessian matrix
8.2
Automatic differentiation
8.2.1
Forward-mode autodiff
8.2.2
Reverse-mode autodiff
8
Backpropagation
How to do backpropagation, and how to do it automatically.
8.1
Evaluation of gradient
8.1.1
Single layer networks
8.1.2
General feed-forward networks
8.1.3
A simple example
8.1.4
Numerical differentiation
8.1.5
The Jacobian matrix
8.1.6
The Hessian matrix
8.2
Automatic differentiation
8.2.1
Forward-mode autodiff
8.2.2
Reverse-mode autodiff
7
Gradient Descent
9
Regularisation