Below is a demonstration of a simple one-layer back-propagation network that learns to classify a set of hand-drawn digits.
The network consists of an input layer with 20 units layed out in a 4 by 5 grid, and 4 output units. Each output unit receives connections from the whole input grid. Hence, the weights on the connections from the input layer to each output unit are also displayed in a 4 by 5 grid. Each output unit also has a "bias weight", shown immediately to the left of the unit's activation, which determines its resting level of activation.
Activation display: The states of the input units are displayed in grey levels on a white background; black means fully activated. The states of the output units are displayed both numerically, and via a color code ranging from black to red. Bright red means fully on. Once an output unit's activation level is 0.8 or greater, it is considered to be fully on for the purpose of the display. This feature makes it easy to see when the output units are producing nearly correct responses.
Weight display:The connection weights are displayed in two colors, red squares for positive weights and blue squares for negative weights. The size of the square indicates the magnitude of the weight. For example, a large blue square represents a very large negative connection weight. The bias weight of each output unit is displayed in a box just to the left of the unit.
Error display: the total error, summed over the 40 training patterns, (see back-propagation description) is displayed during training, once the the number of learning iterations (displayed as # back-props) exceeds 40.
Training patterns: The model is trained on a set of hand-drawn digits. There are 10 examples of each of the four digits, shown at the bottom left region of the display, below the network.
Testing the network: Any one of the training patterns may be presented to the network for testing, by simply clicking on the pattern. Alternatively, you can enter your own test pattern by clicking on Clear and then drawing a digit by hand onto the display of the input unit grid by clicking down and dragging the mouse.
Activation rule:When the network is presented with a training or testing pattern, the states of the input layer units are set to the corresponding values in the training pattern. The output layer units are then activated as follows: Each unit first computes the sum of the activations on each of its incoming connections, weighted by the connection strengths. The final output is computed by passing the weighted summed input through a nonlinear squashing function (the sigmoid function), to get an activation value that lies in the range 0 to 1.
Learning:The network can be trained by clicking on animate; the label for the animate button will then change to stop. Learning will continue until you click on stop.
The network updates its weights by a
supervised learning procedure called back-propagation. The desired states of the output
units are indicated by the little boxes above the output units. When an output
unit should be on, its corresponding box is filled in black.
McMaster students: Please report any difficulties with this software to
your instructor.