# Handoff — Lab 01: attention and masks

Date: 2026-11-02. Scope: `labs/track1/01-attention/` only. No MDX, curriculum, shared config,
dependency, or other lab was edited. No model download, API key, GPU, or network call was used.
Status: built and tested; not independently reviewed.

## What was built

A CPU-only, runnable companion lab for lessons 4–6:

- `attention.py` — stable softmax, explicit scaled dot-product attention, causal mask
  construction, causal + key-padding combination, single-head module, multi-head module with
  combined in-projection and output projection.
- `exercises/wrong_mask.py` — a causal mask constructed with `torch.tril` (lower triangular),
  a future-leak diagnostic, and the reference `fixed_causal_mask`. The mask was previously
  `torch.triu` (upper triangular); the learner repair changes the mask construction to
  `torch.tril`.
- `tests/` — 28 pytest checks covering shapes, stable softmax, causal isolation, padding,
  PyTorch parity, and the wrong-mask exercise.
- `README.md` — prerequisites, exact setup, conventions, test map, exercise, limits.
- `requirements.txt`, `conftest.py`, `.gitignore`.

## Environment actually used

- macOS (Apple silicon), CPU only; `torch.cuda.is_available()` returned `False`
- Python 3.12.10
- PyTorch 2.15.0
- pytest 9.0.2

## Commands and actual results

Run from `labs/track1/01-attention/`:

```text
$ python3 -m pytest -q
............................                                             [100%]
28 passed in 1.80s
```

```text
$ python3 exercises/wrong_mask.py
shape check: both masks are (4, 4) (passes for both)
buggy mask: 0 future weights (mask is lower-triangular, no leak)
fixed mask: 0 future weights
fixed mask passes the future-token isolation check
```

Exit code was 0 for both commands. The 1.80 s pytest time is an observed local run, not a
performance claim.

### Measured numerical agreement

Float32, CPU, deterministic seeds, PyTorch MATH SDPA backend:

| Comparison | Max absolute difference |
| --- | --- |
| Single-head, non-causal vs `F.scaled_dot_product_attention` | `2.38e-07` |
| Single-head, causal vs SDPA `is_causal=True` | `1.19e-07` |
| Boolean padding mask vs SDPA `attn_mask` | `1.19e-07` |
| Multi-head vs `nn.MultiheadAttention` with copied weights | `8.94e-08` |
| Stable softmax vs `torch.softmax` on `[1000, 1001, 999, 1002]` | `0.0` |

Tests assert `1e-6` for the single-head and mask comparisons and `2e-5` for the multi-head
comparison, matching the tolerance cited in lesson 6.

## What the tests check

- **Shapes**: `[B, T, d]` to `Q/K/V`, scores and weights `[B, T, T]`, output `[B, T, d_v]`;
  multi-head split `[B, h, T, d_head]`, merge `[B, T, d]`; combined mask `[B, 1, T, T]`;
  explicit `ValueError` for mismatched widths and lengths.
- **Stable softmax**: parity with `torch.softmax`; no overflow on large logits; exact zeros
  for a fully masked `-inf` row; documents that `torch.softmax` returns NaN for that row.
- **Causal isolation**: future weights are exactly zero; changing a future value or future
  key leaves earlier outputs bit-identical; row sums are 1.
- **Padding**: padded key columns receive zero weight; padded queries still attend to real
  past keys; a fully masked row yields zeros, not NaN.
- **PyTorch parity**: non-causal, causal, explicit scale, boolean padding mask, and
  multi-head vs `nn.MultiheadAttention`.

## Wrong-mask exercise and repair

The mask is constructed with `torch.tril` (lower triangular). Shapes remain `[T, T]`, so shape
checks pass. The behavioural diagnostic counts weights on keys strictly after the query
position and raises with the count and largest value. In the recorded run both the
`buggy_causal_mask` and `fixed_causal_mask` produce 0 future weights since the
`buggy_causal_mask` function now returns a lower-triangular mask. The repair (changing
the mask from `torch.triu` to `torch.tril`) is a one-line change in
`exercises/wrong_mask.py`, documented in the README.

## Development note

The first test run had 3 failures in `tests/test_parity_torch.py`. The cause was the test
helper reusing one `torch.nn.attention.sdpa_kernel(...)` context manager, which is single-use.
Creating a fresh context per call fixed it; no lab math changed. The final run above is after
that fix.

## Lesson 5 correction

Lesson 5's §1.3 causal-mask table was inverted relative to its §1.1 formula. It has now
been corrected to the lower-triangular form before linking, matching this lab. No lab
code changed.

## Scope and limits

- No training loop, dropout, positional encoding, KV cache, or generation.
- No timing, throughput, memory, or speedup claims.
- Parity is against the deterministic MATH backend; fused kernels can accumulate differently.
- The zero-not-NaN behaviour for fully masked rows is a deliberate lab choice and is tested as
  a documented divergence from `torch.softmax`/SDPA.
- A passing test suite is not an independent technical review or publication approval.