PyTorch / Tensor, gradient, and module foundations / Lesson 1

CPU tensors, dtypes, and shapes

Construct a tensor with explicit ownership, numeric representation, and axis meaning before building a model.

About 5 min readContent reviewed 2026-10-28

On this page

Lesson goals

State shape, dtype, device, and axis semantics separately.
Distinguish copying construction from possible shared storage.
Check reductions and broadcasting against a known non-square fixture.

Documented APIs and executed evidence have different scopes

This course targets the explicitly versioned PyTorch 2.14 documentation. No torch runtime is installed in the review environment, so framework programs are complete illustrative CPU examples with derived expected outputs and runnable:false. Separate standard-library companions are executed with Python 3.12.3 and verify only their own arithmetic or contracts. They do not validate PyTorch behavior indirectly. Keep that distinction when reproducing an example in a supported local installation. The conceptual Deep Learning course supplies mathematical foundations; this track explains PyTorch objects, module state, gradient modes, data interfaces, and documented execution boundaries.

A tensor has representation and ownership choices

torch.tensor constructs a new leaf tensor by copying supplied data and infers a dtype unless one is explicit. The example chooses float64 and device="cpu" to make those assumptions visible. torch.as_tensor can avoid copies and preserve history where possible, while torch.from_numpy shares storage under its documented conditions; this lesson imports no NumPy and creates no such array. Treat each constructor as a specific ownership contract rather than assuming all conversions copy. A value that prints like an integer can have a floating dtype, and a tensor’s device is distinct from both its shape and its numeric meaning.

Assign meaning to axes before reducing them

The illustrative framework program represents three observations with two features each. tuple(x.shape) expresses lengths, while the declared row and feature meanings explain what those lengths represent. mean(dim=0) reduces the observation axis and leaves two feature means. The original nested list is changed after construction to expose the constructor’s copy behavior. No random values, GPU, model, or external dataset is involved. In actual model code, record feature order and units outside the bare tensor, because a shape assertion cannot detect swapped meanings or an input that uses the wrong measurement scale.

python Illustrative CPU PyTorch 2.14 API program; torch unavailable, expected output derived

import torch
source = [[1.0,2.0],[3.0,4.0],[5.0,6.0]]
x = torch.tensor(source, dtype=torch.float64, device="cpu")
source[0][0] = 99.0
print(tuple(x.shape), x.dtype == torch.float64, x.device.type)
print("copied first value", x[0,0].item())
print("feature means", x.mean(dim=0).tolist())

Expected output

(3, 2) True cpu
copied first value 1.0
feature means [3.0, 4.0]

The documented constructor copies the supplied nested list into a CPU float64 tensor; changing the list does not change the tensor. This expected behavior was not executed in PyTorch.

Use a non-square independent arithmetic fixture

The companion calculates feature and observation means from plain lists. The two feature means and three observation means have different lengths and meanings, making an accidental axis swap visible. PyTorch broadcasting aligns dimensions under specific rules; it does not know that one axis means examples. A prediction column with shape (B,1) compared with a target vector (B,) can produce a (B,B) result instead of paired errors. Assert the unreduced result shape at a loss boundary. A square test array can conceal a transpose mistake, so include unequal batch and feature dimensions.

python Executed deterministic standard-library companion

rows = [[1.0,2.0],[3.0,4.0],[5.0,6.0]]
assert len({len(row) for row in rows}) == 1
feature_means = [sum(row[j] for row in rows)/len(rows) for j in range(2)]
row_means = [sum(row)/len(row) for row in rows]
print("feature means", feature_means)
print("observation means", row_means)
print("shape", (len(rows),len(rows[0])))

Expected output

feature means [3.0, 4.0]
observation means [1.5, 3.5, 5.5]
shape (3, 2)

The companion verifies the axis arithmetic on supplied lists. It does not test tensor construction, dtype, storage, broadcasting, or any PyTorch runtime behavior.

Preserve contracts when adding convenience operations

Reshaping changes the interpretation of storage positions under stated rules; it does not by itself exchange arbitrary axis meanings. A transpose or permutation expresses another operation, and views can share underlying storage. Later lessons examine strides, contiguity, and in-place changes. Begin with explicit construction and a small independently checked input before optimizing conversions or changing dtype. Validate finite values, required lengths, and meaningful units according to the task. A passing synthetic shape calculation establishes local evidence only; it does not prove data correctness, learned-model quality, runtime compatibility, or performance on a device.

Common errors and fixes

A correct-looking tensor uses the wrong feature meaning

Why it happens: Only axis lengths were validated, not names, units, or order.
How to fix it: Validate the documented input contract alongside shape.
Prevent it next time: Keep a known non-square input with distinct feature scales.

An input conversion unexpectedly shares or loses storage identity

Why it happens: Different construction APIs were treated as equivalent.
How to fix it: Read and choose the documented copy or sharing behavior explicitly.
Prevent it next time: Test ownership separately from equal numeric values in the supported runtime.

A loss compares every prediction with every target

Why it happens: Broadcasting expanded a column against a vector.
How to fix it: Align the intended paired shapes before subtraction.
Prevent it next time: Assert unreduced loss dimensions before aggregation.

Check your understanding

1. For a batch of five observations with three features, what shape remains after mean(dim=0), and what does it mean?

Show worked solution

The remaining shape is (3,), with one mean for each feature across the five observations. mean(dim=1) would instead produce shape (5,), one average across features per observation. The latter can be meaningless if the three features have incompatible units; valid arithmetic does not establish a valid summary.

2. Why does changing source[0][0] not change x in the illustrated program?