How a computer reads handwriting

This is a small neural network: a program that learned to recognize handwritten digits by studying 55,000 examples. It was built from scratch, without any AI libraries. Draw a digit and look inside it while it thinks.

1Draw a digit

Use your mouse or finger to draw any digit from 0 to 9.

Draw a digit here

The network thinks it's a

?

How sure it is about each digit

2What the computer actually sees

A computer can't look at a picture the way you do. First your drawing is cut out, shrunk to a tiny grid of 28 × 28 squares and centered, exactly like every example the network learned from. Then each square becomes a number: 0 for empty, 1 for full ink.

Your drawing, cut out
Shrunk to 28 × 28 squares
The same 784 squares as one long row of numbers, row after row of the grid. This row of numbers is all the network ever gets. Each spike is ink.

3Inside the network

The network has three layers. The 784 numbers flow into 512 neurons. Each neuron has learned to look for its own pattern of ink, and lights up when it finds it. Then every neuron casts votes: for some digits, against others. The digit with the most votes wins.

784 numbers in
512 neurons, brighter = more active
10 digits out

The neurons that voted hardest

For each neuron: on the left, the pattern it looks for (blue = wants ink here, red = wants this spot empty). On the right, that pattern laid over your drawing: the more of your ink lands on blue, the more the neuron lights up and the louder its vote.

4How it learned

Nobody told the network what a 7 looks like. It started out completely random and practiced: look at an image, guess, check the right answer, then nudge all of its numbers a tiny bit so the right answer becomes a little more likely. Repeat. Press play to watch it happen.

of 10,000 test digits right

Test digits right, by images practiced on. Each step to the right is 10 times more practice.

Its guesses for 12 test digits

What 8 of its neurons look for

At the start every pattern is random noise. Watch shapes form in the middle as it practices. The speckled edges barely change: digits almost never reach the edges, so the network rarely gets a reason to adjust them.

5Where it still gets it wrong

Out of the 10,000 test digits it never saw while practicing, it gets wrong. Look through them: many would fool a person too.

Which digits get mixed up

Each row is the real digit, each column is the network's guess. The numbers count its mistakes. Click one to see those digits.

Network's guess →

Real digit →

6From 306 mistakes to

The first version of this network got 306 of the same 10,000 test digits wrong. Five changes, each tried one at a time, brought that down. The basic design never changed: still one layer of neurons, still the same math.

Mistakes on the 10,000 test digits after each change.