PyTorch / Training state and architecture interfaces / Lesson 9

Convolutions and spatial layout

Make PyTorch channel conventions, kernel connectivity, and spatial alignment explicit.

About 4 min readContent reviewed 2026-10-28

On this page

Lesson goals

Read Conv2d input and weight dimensions correctly.
Distinguish logical axis order from channels-last memory format.
Calculate output geometry and verify a fixed local response.

Logical dimensions describe the interface

For the batched case used here, Conv2d accepts logical shape (N,Cin,H,W) and returns (N,Cout,Hout,Wout). A weight tensor has output-channel, connected-input-channel, and kernel dimensions under the documented groups rule. Moving a channel axis into an image axis can preserve element count while changing the operation completely. Record what every channel represents and how input normalization was fitted. A channels_last memory format changes storage strides for a logically NCHW tensor; it is not permission to pass an arbitrary NHWC tensor to an interface expecting NCHW. Keep layout semantics separate from storage optimization.

A fixed kernel makes the indexing inspectable

The illustrative program supplies one image, one channel, and an explicitly assigned two-by-two kernel. PyTorch Conv2d uses cross-correlation indexing, so the first response is one minus five, not the result of reversing the kernel. All four valid windows give minus four. The output shape is (1,1,2,2), and max pooling over that complete response gives one value with shape (1,1,1,1). Explicit weights remove initialization uncertainty. This fixture demonstrates the documented local operator without training an image model or claiming that the kernel detects a meaningful real object.

python Illustrative CPU PyTorch 2.14 API program; torch unavailable, expected output derived

import torch
from torch import nn
x = torch.tensor([[[[1.0,2.0,3.0],[4.0,5.0,6.0],[7.0,8.0,9.0]]]],dtype=torch.float64,device="cpu")
layer = nn.Conv2d(1,1,kernel_size=2,bias=False,dtype=torch.float64,device="cpu")
with torch.no_grad():
    layer.weight.copy_(torch.tensor([[[[1.0,0.0],[0.0,-1.0]]]],dtype=torch.float64,device="cpu"))
y = layer(x)
pooled = nn.MaxPool2d(kernel_size=2)(y)
print("convolution shape", tuple(y.shape))
print("response", y[0,0].tolist())
print("pool shape", tuple(pooled.shape), "value", pooled.item())

Expected output

convolution shape (1, 1, 2, 2)
response [[-4.0, -4.0], [-4.0, -4.0]]
pool shape (1, 1, 1, 1) value -4.0

The fixed Conv2d kernel follows cross-correlation indexing and is not reversed. These CPU convolution and pooling outputs are derived expectations, not executed kernels.

Geometry includes padding, stride, and dilation

For a standard floor-based axis convention, output length is floor((N+2P-D(K-1)-1)/S)+1. The companion checks one padded stride example and the hand-computed response. Named padding modes, asymmetric boundaries, and pooling ceil_mode need their own documented rules. Verify which input coordinates affect each output, not only the final size. A dense prediction target can be shifted relative to the output even when dimensions match. Use a numbered grid or impulse fixture to detect alignment and boundary behavior before trusting an apparently plausible image-shaped result.

python Executed deterministic standard-library companion

image = [[1.0,2.0,3.0],[4.0,5.0,6.0],[7.0,8.0,9.0]]
kernel = [[1.0,0.0],[0.0,-1.0]]
response = [[sum(image[i+a][j+b]*kernel[a][b] for a in range(2) for b in range(2))
             for j in range(2)] for i in range(2)]
def size(n,k,p=0,s=1,d=1):
    return (n+2*p-d*(k-1)-1)//s+1
print("response", response)
print("padded stride size", size(7,3,p=1,s=2))
print("kernel weights", 1*1*2*2)

Expected output

response [[-4.0, -4.0], [-4.0, -4.0]]
padded stride size 4
kernel weights 4

The executed loops check finite-window arithmetic and one standard output-size formula. They do not verify Conv2d layout, dispatch, gradients, or pooling semantics.

Groups change channel connectivity and parameter count

With groups one, each output channel combines every input channel through a spatial kernel. Grouped convolution divides that connectivity, and input and output channel counts must meet the documented divisibility conditions. The parameter count follows actual connections plus optional biases, not batch size or image width. Pooling and strided operations reduce spatial resolution and can discard small structures needed by a task. A larger theoretical receptive field describes possible dependency reach rather than a measured explanation of what trained weights use. Compare task requirements and evaluation before choosing a downsampling pattern.

Inspect the complete module boundary

Check NCHW meaning, dtype, device, weight shape, groups, bias policy, output coordinates, and flattening before a dense head. Flattening across a batch axis accidentally merges examples. Use registered modules and deliberate initialization when building a small reference network, and inspect mode-dependent layers separately. This course creates no image files or external dataset and runs no convolution, pooling, memory-format conversion, or framework model. The torch program is illustrative; the executed companion validates only its supplied loops, counts, and size arithmetic, not device performance or runtime implementation correctness.

Common errors and fixes

A tensor with the right element count produces an unrelated convolution

Why it happens: Logical channel and spatial axes were rearranged incorrectly.
How to fix it: Restore the documented logical layout and validate channel meaning.
Prevent it next time: Keep a non-square multi-axis sentinel fixture.

A hand calculation has the opposite local response

Why it happens: The kernel was reversed while Conv2d used cross-correlation.
How to fix it: Align the operator convention before comparing values.
Prevent it next time: Use an asymmetric kernel on a numbered grid.

A flatten operation mixes observations

Why it happens: The batch dimension was included in feature flattening.
How to fix it: Preserve the observation axis and calculate head input width explicitly.
Prevent it next time: Assert output shape and identity for at least two distinct examples.

Check your understanding

1. Count parameters for Conv2d with three inputs, two outputs, a 3 by 3 kernel, groups=1, and biases.

Show worked solution

Each output has three times nine weights, or twenty-seven, plus one bias. Two outputs therefore have 2*(27+1)=56 parameters. Batch size and image dimensions do not change that count, although they change activation storage and work.

2. Why does channels_last not mean the logical tensor shape becomes (N,H,W,C)?