# Track 1 Lab C — Cached Generation and GQA

**Lesson coverage:** Lessons 11–15 (Generation and context: orders 11–14; Modern architectures: order 15, MHA/GQA).
**Directory:** `labs/track1/03-cache-and-gqa/`

## What this lab teaches

A small, deterministic, CPU-only PyTorch model demonstrates:

1. **Cached vs uncached logit parity** — the KV cache produces identical next-token
   logits to a full-sequence forward pass, to within FP32 tolerance (1e-5).
2. **Cache growth** — exactly one K/V column is appended per layer per decode step;
   cache shape follows `[B, H_kv, P + S, D_HEAD]`.
3. **MHA vs GQA cache memory** — raw KV-cache bytes calculated from the lab model
   dimensions: 8,192 bytes per layer for MHA (`H_kv=2`) and 4,096 bytes per layer for
   GQA (`H_kv=1`), or 16,384 and 8,192 bytes across the two layers. GQA figures are
   calculated from the formula, not measured from an instantiated GQA cache.
4. **Broken cache position** — a broken cache write that overwrites slot 0 instead
   of appending at the current cache length; the cache never grows, logit parity
   fails, and the fix restores it.

Timing is reported as optional measured local data only; it is never presented as
a guaranteed speedup.

## Files

| File | Purpose |
|------|---------|
| `cache_model.py` | Tiny deterministic decoder with cached/uncached paths and MHA/GQA byte calculator |
| `test_logit_parity.py` | Cached vs uncached logit parity over 5 decode steps |
| `test_cache_growth.py` | One KV append per layer per step; cache shape assertions |
| `test_mha_gqa_memory.py` | Per-layer and total MHA vs GQA cache bytes computed from model dimensions; GQA is calculated, not instantiated |
| `test_broken_cache_position.py` | Broken cache-position write + repair; diagnostic test |
| `actual_output.txt` | Recorded output from all four test files |
| `setup.md` | Exact environment setup and tested versions |
| `HANDOFF.md` | Documentation of setup, results, and lab status |

## Prerequisites

- Python 3.12
- PyTorch 2.15.0 (CPU)
- No GPU, no model download, no API key, no checkpoint

## How to run

```bash
cd labs/track1/03-cache-and-gqa
python3 test_logit_parity.py
python3 test_cache_growth.py
python3 test_mha_gqa_memory.py
python3 test_broken_cache_position.py
```

All tests assert pass/fail and print `PASS` on success.

## Connection to lessons

- **Lesson 11 (llm-text-generation):** decoding loop, greedy sampling, stopping rules.
- **Lesson 12 (prefill-and-decode):** prefill fills cache once; decode appends one column per step.
- **Lesson 13 (kv-cache-explained):** per-layer KV cache update, cached/uncached logit parity,
  raw KV-cache byte formula `2 × L × H × D × S × B × b`.
- **Lesson 14 (context-windows):** context length vs KV-cache memory (linear scaling).
- **Lesson 15 (mha-mqa-gqa):** H_q vs H_kv; GQA reduces KV heads and thus cache bytes.

## Lab status

All four tests pass. Timing data is not included as a guaranteed result. No MDX or
curriculum files were modified.