| Author: | Francesco Lelli |
| Learning Line: | Gradient Descent and cost functions. |
| Course: | AI1: Introduction to Neural Network |
| Week: | 4 |
| Competencies: | 1- Understand and be able to use neural network dynamics and factors that influence their performance 2- Implement a basic pattern recognition |
| BoKS: | 3aS2 The student knows how to apply the right statistical & mathematical tools for specific AI-contexts |
| Learning Goals: | Understand Gradient Descent and cost functions. |
Artificial neural networks (ANN) or simply “neural networks” are computing systems inspired by “classical biological” neural networks that are in animal brains. Typically an ANN “learn” to perform tasks by considering examples, without being programmed with task-specific rules.
Example of Neural Network
Image recognition is a classical example where neural networks are used; in particular they learn to identify images that contain dogs by analyzing example images that have been manually categorized (labelled) as “dog” or “no dog”. Once the system “learn” the difference it can be used for identifying dogs in other images.
An ANN does not use conceptual reasoning, for example is not capable to devise that “all dogs have 4 legs”, or “a dog must have a nose” etc. Instead, they hide the features of a dog into their data structure and they “internally learn these characteristics” from the examples that are used for training them.
Practical applications are more complex that the classical example mentioned in these videos. Nevertheless, learning how to recognize hand-written digits is always a good place to start. In other words you can consider this example as the “hello world” example for learning the basics of Neural Networks.
In this video and the coming lecture we introduce the Gradient Descent and cost functions.
Gradient descent is a first-order iterative optimization algorithm for finding a local minimum of a differentiable function. To find a local minimum of a function using gradient descent, we take steps proportional to the negative of the gradient (or approximate gradient) of the function at the current point.
Practical Information for the assignment
The code mentioned in the videos is also available at this link:
https://github.com/mnielsen/neural-networks-and-deep-learning
In the case you would like to replicate the example you can access the database of hand-written digits at this link:
http://yann.lecun.com/exdb/mnist/
And, in the case you would like to learn about Artificial Neural Networks in details I would recommend this open book:
http://neuralnetworksanddeeplearning.com/
This is week 4 of AI1: Introduction to Neural Networks, link to the all material of the course: