RNN

Updated at: 13 Mar 2019 14:37:44

Description

ANN builds pyrimidal cells in the cerebral cortex. Uses shortcuts to jump over layers.

Motivation for skipping over layers is to avoid problem of vanishing gradients by reusing activations from a previous layer until adjacent layer has learnt weights.

The weights adapt to mute the upstream layer, amplify the previously skipped layer.

Forward propagation for a simple RNN: alttext