Backpropagation Learning in Feedforward Neural Nets

We cannot use the Perceptron Learning Rule for learning in Feedforward Nets because for hidden units we don't have a teacher to provide desired values.
Our solution is to use the backpropagation approach: gradient-descent algorithm to minimize the error on the training data by propagating errors backwards through the network starting at the output units and working backwards towards the input units.
Algorithm
1. Initialize the weights in the network (often randomly)
2. repeat
  - foreach example e in the training set do
    1. O = neural-net-output(network, e) ; forward pass
    2. T = teacher output for e
    3. Calculate error (T - O) at the output units
    4. Compute delta_wi for all weights from hidden layer to output layer ; backward pass
    5. Compute delta_wi for all weights from input layer to hidden layer ; backward pass continued
    6. Update the weights in the network
  - end
3. until all examples classified correctly or stopping criterion satisfied
4. return(network)