Skip to content
intai
Search lessonsCtrl K

Track 03

Live

Fine-Tuning & Domain-Specific Models

Adapt a base model to a domain without losing the judgement you started with: datasets, adapters, distillation, and small models.

  • Advanced
  • 16 of 16 lessons live
  • 214 min live
Start with lesson 1View curriculum

What you'll be able to do

By the end of the core course.

  1. Design a dataset that teaches a behaviour instead of a format
  2. Choose between full fine-tuning, LoRA, and distillation for a constraint
  3. Evaluate a tuned model for regressions before it reaches users

Main subjects

The territory this track covers.

  1. 01Dataset engineering
  2. 02Supervised fine-tuning
  3. 03LoRA and QLoRA
  4. 04Distillation
  5. 05Small language models

Who this is for

Engineers turning a general-purpose model into a dependable specialist for one domain.

Before you start

  • Comfort with Python and a training loop
  • Access to a GPU or a hosted fine-tuning service
  • Familiarity with evaluation basics

Curriculum

16 of 16 lessons published

Core course

16 of 16 live

The main sequence, read in order.

Module 01

Decide Whether to Train

Compare prompting, retrieval, base-model choice, and fine-tuning before changing any weights.

3 of 3 lessons live

  1. 01Diagnose the Gap Before Changing WeightsEvaluate zero-shot, few-shot, and RAG on a held-out domain task before fine-tuning, diagnosing format, factual, and reasoning gaps.Advanced12 min readRead
  2. 02Select a Base Model for the TaskEvaluate open-weights base models across tokenizer fertility on structured JSON, VRAM budgets for LoRA versus serving, base versus instruct checkpoints, and enterprise licensing.Advanced13 min readRead
  3. 03Set a Baseline and Success ThresholdEstablish frozen held-out evaluation sets, behavioral slicing, safety recall asymmetry, general capability regression guardrails, and the formal Ship/Train/Hold decision record.Advanced12 min readRead

Module 02

Make Training Data Trustworthy

Turn real tasks into examples, format conversations, mask the loss, and audit the dataset before training.

4 of 4 lessons live

  1. 04Turn Real Tasks Into Training ExamplesSpecify desired behaviour, provenance, rights, and sensitive-data handling for each training example.Advanced14 min readRead
  2. 05Format Conversations and Mask the LossInspect token IDs and loss labels for the chosen objective, including chat-template boundaries, EOS, truncation, and packing; assistant-only loss is optional.Advanced13 min readRead
  3. 06Split, Deduplicate, and Audit the DatasetUse group-aware splits and exact or near-duplicate checks to limit leakage, imbalance, and evaluation contamination before training.Advanced13 min readRead
  4. 07Use Synthetic and Teacher Data CarefullyTrack teacher and data provenance, apply task-specific checks, and compare generated examples against real held-out tasks.Advanced13 min readRead

Module 03

Adapt Within a Compute Budget

Match the training objective to the missing capability, then choose LoRA, QLoRA, or full fine-tuning within a memory budget.

5 of 5 lessons live

  1. 08Choose Continued Pretraining or Supervised Fine-TuningDistinguish unsupervised domain-adaptive pretraining from instruction tuning, budget training compute, and avoid hallucination pitfalls when injecting knowledge.Advanced14 min readRead
  2. 09LoRA: Add Trainable Low-Rank AdaptersDecompose dense weight updates into low-rank matrices, calculate trainable parameter ratios, select target modules, and scale adapter updates with alpha.Advanced14 min readRead
  3. 10QLoRA: Quantize Base Weights to 4-BitBudget VRAM, compress base models with 4-bit NormalFloat and Double Quantization, and use paged optimizers to fine-tune on consumer GPUs.Advanced14 min readRead
  4. 11Full Fine-Tuning Versus AdaptersCompare full parameter training against low-rank adapters across compute costs, memory sharding, catastrophic forgetting, and multi-tenant serving.Advanced14 min readRead
  5. 12Debug and Reproduce a Training RunApply pre-flight sanity checks, execute tiny-batch overfit tests, track gradient norms, and generate deterministic reproducibility manifests.Advanced14 min readRead

Module 04

Prove a Specialist Is Worth Shipping

Distill behaviour into a compact student, evaluate gains and regressions, and package the model for release with rollback triggers.

4 of 4 lessons live

  1. 13Choose a Compact Student ModelSize a student model for the task and deployment constraints before distillation begins, comparing parameter scales, KV-cache footprints, and tokenizer alignment.Advanced13 min readRead
  2. 14Distill Useful Behaviour From a TeacherCompare teacher outputs with task labels and check transfer failures during distillation.Advanced14 min readRead
  3. 15Test Gains and RegressionsCompare paired domain, general, out-of-distribution, and safety results; audit judge bias where qualitative scoring is needed.Advanced14 min readRead
  4. 16Package the Model for ReleaseRecord model and adapter lineage, data rights, evaluation evidence, deployment boundaries, and rollback triggers in a model card.Advanced13 min readRead

Lessons are published in reading order. A coming-soon entry has no page yet — it is not a broken link.