Deep Learning / Architectures and training dynamics

Deep Learning β€” Architectures and training dynamics

Initialization, normalization, batching, convolution, sequences, attention, and diagnosis.

6 lessons Β· No account required

What you will learn

  • Initialization, normalization, batching, convolution, sequences, attention, and diagnosis.

  1. Initialization, normalization, and residual paths

    Reason about signal scale and gradient paths without treating architectural techniques as universal repairs.

    Start learning

  2. Minibatches, accumulation, and schedules

    Preserve objective weighting and update counts when changing how examples are processed.

    Start learning

  3. Convolution, padding, and receptive fields

    Make local weight sharing, spatial geometry, and channel mixing explicit.

    Start learning

  4. Sequences, recurrence, and padding

    Define temporal state, valid lengths, and target alignment before training a sequence model.

    Start learning

  5. Attention, masks, and position

    Trace scaled dot-product attention and identify the information that masks and position mechanisms supply.

    Start learning

  6. Diagnosing training dynamics

    Use boundary checks, controlled fitting, and gradient evidence to isolate a failed training procedure.

    Start learning
Back to course