The architecture that learned to see. Shared filters slide across an image, detecting patterns wherever they appear — the spatial sense an MLP completely lacks.
In 3.1 we ended on a limitation: a plain MLP treats an image as a flat, unordered list of pixels. It has no notion of space — a pixel's neighbors mean nothing to it, and a cat in the top-left is, to the network, a completely different input than the same cat in the bottom-right. Feed it a modest 200×200 image and every first-layer neuron needs 40,000 weights. Wasteful, and blind to structure.
The Convolutional Neural Network (CNN) fixes exactly this. Instead of connecting every pixel to every neuron, it slides a small set of shared weights — a filter (or kernel) — across the image, computing a little weighted sum at each position. The same filter is reused everywhere, so a pattern it learns to detect (an edge, a curve, a texture) is found wherever it appears. This single idea — local connections plus shared weights — is what gave us AlexNet in 2012 and launched the modern era of computer vision.
Imagine a huge mural in a dark room, and you have a small flashlight that lights up only a 3×3 patch at a time. You sweep it methodically left to right, top to bottom. At every position you ask one fixed question — say, "is there a vertical edge here?" — and jot down a single number for how strongly the answer is yes. When you're done sweeping, your page of numbers is a map of where vertical edges live in the mural.
That flashlight is the filter. The fixed question it asks is the filter's learned weights. The page of jotted numbers is the feature map. And because you asked the same question at every position, you'd spot a vertical edge whether it sat in the corner or dead center — that is translation invariance, the property the MLP never had. Swap in a second flashlight asking "is there a curve here?" and you get a second feature map. Stack enough of these and the network sees edges, then shapes, then objects.
A convolution layer has three moving ideas. First, the filter: a small grid of weights, typically 3×3 or 5×5. Second, the sliding: the filter steps across the input one position at a time, and at each stop it does an element-wise multiply with the patch underneath, then sums the results into one output number. Third, shared weights: it's the same filter at every position, so the layer needs only nine weights (for a 3×3) instead of one weight per pixel.
The grid of output numbers the filter produces as it sweeps is the feature map — literally a map of where that filter's pattern was found. A few terms you'll meet: stride is how far the filter jumps each step (stride 1 = every position, stride 2 = skip every other, shrinking the output). Padding adds a border of zeros so the filter can reach the edges and the output stays the same size. And channels are the depth dimension — more on those below.
For an output position (i, j), with a K×K filter w over input X:
out(i,j) = ╲╲ X(i+m, j+n) · w(m,n) + b
╱╱
m,n = 0 .. K-1
In words: line the filter up over the patch, multiply
each overlapping pair, add them all up, add the bias.
Output size (one dimension):
out = (N - K + 2P) / S + 1
N=input size, K=filter, P=padding, S=stride
Here is one convolution step. A 3×3 patch of an image meets a classic vertical-edge filter (bright on the left, dark on the right). Wherever the image goes from bright to dark horizontally, this filter fires strongly.
Image patch: Filter (vertical edge): [ 90 20 10 ] [ +1 0 -1 ] [ 95 25 15 ] [ +1 0 -1 ] [ 88 18 12 ] [ +1 0 -1 ] Element-wise multiply, then sum: (90×1) + (20×0) + (10×-1) + (95×1) + (25×0) + (15×-1) + (88×1) + (18×0) + (12×-1) = (90 - 10) + (95 - 15) + (88 - 12) = 80 + 80 + 76 = 236 → large positive: strong edge here Same filter on a flat patch [50 50 50 / ...]: = (50-50)+(50-50)+(50-50) = 0 → no edge
The 5×5 grid on the left is a tiny image (a bright region meeting a dark one). The 3×3 filter sweeps across it; the highlighted patch shows where it sits right now. On the right, the feature map fills in one cell at a time as the filter moves. Watch the calculation update live, switch filters to see different patterns emerge, and notice the output is 3×3 — exactly what the size formula predicted.
Notice three things: the same nine filter weights are reused at every position (that's weight sharing — the whole reason a CNN is efficient). The vertical-edge filter lights up the feature map down the middle, where bright meets dark, and stays near zero on the flat regions. And switching to the blur filter produces a totally different map from the identical image — the filter is the question being asked.
After convolution, feature maps are still large. Pooling shrinks them while keeping the strongest signals. The common form, max pooling, slides a small window (usually 2×2) across the feature map and keeps only the maximum value in each window — discarding three of every four numbers. A 2×2 max-pool halves the width and height, cutting the data to a quarter.
Feature map region: After 2×2 max pool (stride 2): [ 236 12 | 40 8 ] [ 18 90 | 30 22 ] [ 236 40 ] ----------+-------- [ 110 205 ] [ 110 45 | 18 205 ] [ 60 33 | 90 12 ] Each 2×2 block → its single largest value. Top-left block: max(236,12,18,90) = 236 Top-right block: max(40,8,30,22) = 40 ...and so on.
Two more pieces complete the picture. Channels are the depth of the data. A color image isn't one grid, it's three stacked grids — red, green, blue. So a filter on a color image isn't 3×3, it's 3×3×3: it spans all three channels at once and sums across them into a single output number. The input depth always matches the filter depth.
And a single convolution layer doesn't learn just one filter — it learns many, often 32, 64, or hundreds. Each produces its own feature map, and those maps stack into the output's channels. So a layer might take a 3-channel image and emit a 64-channel result: 64 different learned questions, each answered across the whole image. The next layer's filters then span all 64 of those channels, combining simple features into complex ones. This is the same feature hierarchy from 3.1 — early layers find edges, middle layers find shapes and textures, deep layers find whole objects — now built spatially.
A convolutional network doesn't memorize where things are. It learns what to look for, then looks everywhere at once. That is the whole trick — and why a few thousand shared weights can outperform millions of independent ones. On why weight sharing changed computer vision
CNNs are masters of spatial structure — grids of pixels, where nearby things relate. But much of the world is sequential: language, audio, time series, where meaning depends on order and on what came earlier. A CNN's fixed, local window has no memory of a sequence and no natural way to handle inputs of varying length.
That gap is what 3.3 Recurrent Neural Networks sets out to fill — adding a hidden state that carries information forward through a sequence, one step at a time. The MLP gained space here in 3.2; next it gains memory.
| Property | Value | Notes |
|---|---|---|
| Core operation | Convolution | Slide a shared filter, multiply-and-sum each patch |
| Key idea | Local + shared weights | Few weights, reused everywhere — efficient and translation-aware |
| Output | Feature maps | One map per filter, marking where its pattern appears |
| Downsampling | Pooling | Max pooling keeps strongest signal, shrinks data, adds robustness |
| Depth | Channels | Input channels match filter depth; many filters stack into output channels |
| Strength | Spatial structure | Images, video frames, anything on a grid |
| Blind spot | Sequence & order | No memory of time or order — motivates RNNs |
| Best use | Vision tasks | Classification, detection, segmentation; still strong in 2026 |
A CNN gives a network the sense of space the MLP lacked. By sliding small shared filters across an image, it detects patterns wherever they appear, using a tiny fraction of the weights a fully-connected layer would need. Convolution finds features, pooling shrinks and stabilizes them, and stacking many filters across channels builds a hierarchy from edges to objects. CNNs still dominate vision in 2026 — but they see only space, not sequence. For language, audio, and time, the network needs memory, and that is where recurrent architectures come in next.