Lab 01 — Attention and masks

Track 1 practice lab

Practice material, not a published lesson. Not counted in the Track 1 chapter count.

Prerequisite lessons

• Self-Attention: Queries, Keys, and Values
• Causal and Padding Masks: Implementation and Tests
• Multi-Head Attention: Splitting, Merging, and Projecting

Setup

• Python 3.12.10, PyTorch 2.15.0 (CPU), pytest 9.0.2.
• Create a virtual environment and install `labs/track1/01-attention/requirements.txt`.
• No GPU, model download, API key, or network access is required.

Exercise

Replace the lower-triangular causal mask with an upper-triangular one. Shape checks still pass, but the future-token leak test fails. Apply the fix and rerun parity.

Verification commands

cd labs/track1/01-attention
python3 -m pytest -q
python3 exercises/wrong_mask.py

Recorded results

28 tests pass. Single-head non-causal parity max difference 2.38e-07; causal 1.19e-07; padding mask 1.19e-07; multi-head 8.94e-08. The wrong mask produces 6 leaking weights (largest 0.740962); the repaired mask produces 0.

Runnable files

• README.md
• attention.py
• exercises/wrong_mask.py
• tests/test_shapes.py
• requirements.txt
• HANDOFF.md

All practice labs