Deep Learning / Architectures and training dynamics / Lesson 9

Convolution, padding, and receptive fields

Make local weight sharing, spatial geometry, and channel mixing explicit.

About 4 min readContent reviewed 2026-10-28

On this page

Lesson goals

Compute a shared local filter.
Derive output sizes and convolution parameter counts.
Separate equivariance, invariance, and receptive-field reach.

A local filter reuses one set of weights

A convolutional layer applies a shared local operator at different positions. Many deep-learning APIs implement cross-correlation and call it convolution: they do not reverse the kernel as a mathematical convolution would. State the convention when comparing hand calculations. Sharing reduces parameter count relative to a dense map connecting every spatial position, and encodes a useful assumption about repeated local structure. It does not make all tasks spatially local or guarantee that a feature transfers across arbitrary transformations. Channel order and the units or meaning of each input channel remain part of the input contract.

Inspect the simplest shared computation

The first fixture slides a two-position kernel across a one-dimensional signal with no padding and stride one. The result contains three adjacent differences. Every position reuses the same two coefficients, so the layer can detect a local pattern wherever the valid window fits. A shifted signal produces correspondingly shifted interior responses under appropriate assumptions, a property called translation equivariance. Finite boundaries, padding, striding, and later operations qualify this statement. Translation invariance means the final result does not change under a translation, which is stronger and does not follow merely from using a convolution.

python Deterministic standard-library calculation

signal = [1, 2, 4, 7]
kernel = [-1, 1]
result = [sum(signal[i+j]*kernel[j] for j in range(2))
          for i in range(len(signal)-len(kernel)+1)]
print(result)

Expected output

[1, 2, 3]

The shared two-position cross-correlation kernel detects adjacent differences without reversing the kernel.

Spatial dimensions follow a geometry contract

For input size N, symmetric padding P, kernel size K, dilation D, and stride S, one standard output length is floor((N+2P-D(K-1)-1)/S)+1. The second fixture evaluates valid, padded-strided, and dilated cases. Named padding modes may use asymmetric padding or framework-specific rules, so do not infer their exact geometry from a friendly name alone. Verify the actual receptive positions with a small numbered input. A spatial output that looks plausible can still be shifted relative to targets, particularly in dense prediction tasks where each output location must align with a specific input region.

python Deterministic standard-library calculation

def output_size(n, k, p=0, s=1, d=1):
    return (n + 2*p - d*(k-1) - 1)//s + 1
print("valid", output_size(7, 3))
print("padded stride", output_size(7, 3, p=1, s=2))
print("dilated", output_size(7, 3, d=2))
cin, cout, k = 2, 4, 3
print("parameters", cout * (cin*k*k + 1))

Expected output

valid 5
padded stride 4
dilated 3
parameters 76

The size formula is for a standard floor-based axis convention; the parameter count includes one bias per output channel.

Channels and receptive fields answer different questions

A standard two-dimensional convolution with Cin input channels, Cout output channels, and a K by K kernel has Cout*Cin*K*K weights plus optional biases. It mixes channels as well as nearby positions. Grouped and depthwise convolutions change this connectivity and therefore the count. A receptive field describes which input locations can affect an output through the architecture. Stacking layers, adding dilation, or downsampling changes its reach. The theoretical field is a possible dependency set; learned weights and nonlinearities determine the actual influence, so the size alone is not a measured explanation.

Downsampling needs a task-aware interpretation

Pooling or strided operations reduce spatial resolution and work, but can discard details and make alignment harder. Max pooling and average pooling preserve different information and have different derivative paths. A tiny object or short signal event can disappear under a coarse step. Before choosing downsampling, define the smallest structure the task needs and inspect boundary cases. Compare exact shape and coordinate alignment before measuring validation performance. The fixtures here validate indexing and counts only; they are not trained image models, signal-processing benchmarks, or claims that a particular kernel detects meaningful objects in real data.

Common errors and fixes

A hand-computed filter has the opposite sign

Why it happens: The hand calculation reversed a kernel while the API used cross-correlation.
How to fix it: State and align the operator convention.
Prevent it next time: Test a non-symmetric kernel on a short numbered signal.

Predictions are spatially shifted despite matching sizes

Why it happens: Padding or stride changed coordinate alignment.
How to fix it: Trace receptive positions and target coordinates explicitly.
Prevent it next time: Keep an impulse or numbered-grid alignment fixture.

A parameter count ignores input channels

Why it happens: The kernel was counted as one spatial array per output only.
How to fix it: Include each input-to-output channel connection.
Prevent it next time: Calculate standard and grouped connectivity separately.

Check your understanding

1. Count parameters for three input channels, five output channels, and a 3 by 3 kernel with biases.

Show worked solution

Each output channel has 3*3*3=27 weights and one bias. Five output channels therefore have 5*28=140 parameters. Batch size and image width do not enter this count, although both affect activations and computation.

2. Why does a larger theoretical receptive field not prove that a model uses distant context?