{
  "module": "FTDD-06 — Dolphin / Hermes: Uncensored Lineages as Engineering Case Studies",
  "course": "3 — LLM Fine-Tuning Masterclass",
  "version": "1.0.0",
  "duration_minutes": 45,
  "total_questions": 10,
  "bloom_distribution": {
    "target": "40% recall / 30% application / 30% analysis",
    "actual": { "recall": 4, "application": 3, "analysis": 3 }
  },
  "passing_score_percent": 70,
  "questions": [
    {
      "id": "Q01", "bloom": "recall", "type": "multiple_choice",
      "prompt": "What is the course stance on studying uncensored lineages like Dolphin and Hermes?",
      "options": [
        "Endorsement — these are the models practitioners should deploy by default.",
        "Condemnation — uncensored models should not be discussed in a fine-tuning course.",
        "Engineering case studies, not advocacy. The model steers; the harness bounds. An uncensored model is only responsible inside an eval'd harness; uncensoring RAISES the harness requirement.",
        "Neutral description with no engineering takeaways."
      ],
      "answer_index": 2,
      "rationale": "The stance (FT00) is unchanged: these are studied as engineering case studies for their recipes and trade-offs. Uncensoring raises, not lowers, the harness requirement. Both extremes (celebrate/condemn) miss the engineering."
    },
    {
      "id": "Q02", "bloom": "recall", "type": "multiple_choice",
      "prompt": "What is Hermes 3 (Nous Research), per arXiv:2408.11857?",
      "options": [
        "A LoRA adapter family for Llama 3.1.",
        "A full-parameter SFT+DPO steer of Llama 3.1 at 8B/70B/405B, 'neutrally-aligned, unlocked, uncensored, highly steerable,' trained on primarily synthetic responses.",
        "A reward model for classical RLHF.",
        "A quantization format for uncensored models."
      ],
      "answer_index": 1,
      "rationale": "Hermes 3 is the canonical large-scale alignment-control recipe: full-param SFT then DPO on Llama 3.1 at three scales, on primarily synthetic data. It is the best-documented full-param alignment-control run in open weights."
    },
    {
      "id": "Q03", "bloom": "recall", "type": "multiple_choice",
      "prompt": "What is OpenHermes 2.5, and why is it significant to the course thesis?",
      "options": [
        "A quantization method used by the Hermes family.",
        "Teknium's ~1M-example instruction dataset that steered the OpenHermes 2.5 / Nous Hermes 2 / Hermes 3 family — the 'steering wheel' that makes 'data matters more than algorithm' concrete.",
        "The reward model used in Hermes 3's DPO stage.",
        "Eric Hartford's compliance dataset for the Dolphin series."
      ],
      "answer_index": 1,
      "rationale": "OpenHermes 2.5 (Teknium, ~1,001,551 examples) is the dataset backbone of the Hermes family. It is the textbook example of the FT00 thesis: one high-quality dataset steered a family of models. (Its flaws would propagate too — audit data like code.)"
    },
    {
      "id": "Q04", "bloom": "recall", "type": "multiple_choice",
      "prompt": "What makes Dolphin3.0-R1-Mistral-24B technically distinctive?",
      "options": [
        "It is the smallest model in the Dolphin lineage.",
        "It is the only uncensored model trained on DeepSeek-R1 reasoning traces (~800K traces over 3 rounds, Mistral Small 24B base).",
        "It uses abliteration instead of SFT.",
        "It is a closed-weights model from Nous Research."
      ],
      "answer_index": 1,
      "rationale": "Dolphin3.0-R1-Mistral-24B is the only uncensored model trained on R1 reasoning traces — pairing Hartford's compliance-over-judgment philosophy with R1-grade reasoning via CoT distillation (FT14/FT15) on an uncensored Mistral Small 24B base."
    },
    {
      "id": "Q05", "bloom": "application", "type": "multiple_choice",
      "prompt": "You read that Hermes 3 is 'highly steerable.' A teammate takes this to mean 'safe to deploy with a minimal harness because it's so controllable.' What is wrong with this reasoning?",
      "options": [
        "Nothing — high steerability means the model needs less harnessing.",
        "High steerability means the deploying system has fine control — a feature — but it ALSO means a weak harness produces a model that does whatever it's prompted to do, including dangerous things. Steerability RAISES the harness requirement; it doesn't lower it.",
        "Hermes 3 cannot be deployed at all.",
        "Steerability only matters for SFT, not DPO."
      ],
      "answer_index": 1,
      "rationale": "Steerability is a double-edged property. It gives fine control (a feature) but means a weak harness does damage — the model executes whatever it's prompted toward. The harness (Layer 5) provides the bounds the model cannot provide itself. High steerability raises the harness requirement."
    },
    {
      "id": "Q06", "bloom": "application", "type": "multiple_choice",
      "prompt": "A team picks Dolphin3.0-R1-Mistral-24B for a math-heavy agentic workflow and is surprised that math quality dropped versus the refusal-trained base. What did they likely miss?",
      "options": [
        "Dolphin models are inherently bad at math.",
        "The capability cost of uncensoring (FT17: GSM8K down up to -18.8pp). An uncensored model is a different point in capability-compliance space, not 'same model + compliance.' They should read the full eval table including regressions and decide on numbers.",
        "They used the wrong quantization format.",
        "Dolphin3.0-R1 is not trained for reasoning."
      ],
      "answer_index": 1,
      "rationale": "Removing refusal is not free (FT17) — the refusal direction is entangled with other capabilities, including math. The team read the strengths and ignored the regressions. If math is load-bearing, a refusal-trained model behind a good harness policy may serve better. This is an engineering decision on numbers."
    },
    {
      "id": "Q07", "bloom": "application", "type": "multiple_choice",
      "prompt": "Map the two Hermes 3 stages onto the Steering Stack and the FT modules.",
      "options": [
        "Stage 1 = Layer 1 (base), Stage 2 = Layer 4 (export).",
        "Stage 1 (full-param SFT) = Layer 3 / FT12 (format, instructions, character). Stage 2 (DPO) = Layer 3 / FT13 (sharpen toward compliance + steerability). The base is Layer 1 (Llama 3.1); Layer 5 (harness) is the deployer's job, not Hermes 3's.",
        "Stage 1 = Layer 5 (harness), Stage 2 = Layer 2 (adapter).",
        "Both stages are Layer 2 (adapter) because they use LoRA."
      ],
      "answer_index": 1,
      "rationale": "SFT and DPO are both Layer 3 (the Steer): SFT maps to FT12, DPO to FT13. The base is Layer 1 (Llama 3.1). Hermes 3 is full-param (not LoRA), so no Layer 2 adapter. Layer 5 (the harness) is the deployer's responsibility — the FT00/FT23 synthesis."
    },
    {
      "id": "Q08", "bloom": "analysis", "type": "multiple_choice",
      "prompt": "Why is 'compliance over judgment' (Hartford's Dolphin philosophy) said to make a model MORE harness-dependent, not less?",
      "options": [
        "Because compliance-oriented models are slower and need more GPU.",
        "Because a judgment-oriented model self-refuses some harmful requests (a partial internal gate); a compliance-oriented model defers to the user + harness and will NOT self-refuse. Every internal gate the judgment model provided must now be provided EXTERNALLY by the harness. Less internal gating = more external gating required.",
        "Because compliance-oriented models have larger parameter counts.",
        "Because compliance models cannot be quantized."
      ],
      "answer_index": 1,
      "rationale": "Removing the model's internal judgment shifts the entire safety burden onto the harness. A compliance-oriented model does not self-refuse, so every policy gate must be external (Layer 5). This is why uncensoring raises — not lowers — the harness requirement. The harness is what makes the uncensored model responsible."
    },
    {
      "id": "Q09", "bloom": "analysis", "type": "multiple_choice",
      "prompt": "How does OpenHermes 2.5 being the shared dataset backbone create both a strength and a liability for the Hermes family?",
      "options": [
        "It only creates a strength; there is no liability in a shared dataset.",
        "Strength: one high-quality ~1M-example dataset steered a whole family (data matters more than algorithm, FT00). Liability: any flaw (bias, quality regression) in the synthetic-data pipeline propagates into EVERY downstream model. The steering wheel's flaws steer you into the wall, at scale.",
        "It only creates a liability; shared datasets are always bad.",
        "The liability is that OpenHermes 2.5 is too small to matter."
      ],
      "answer_index": 1,
      "rationale": "The dataset-as-steering-wheel thesis cuts both ways. A great dataset steers a family well (strength); a flawed dataset propagates flaws into every model trained on it (liability). This is why data quality (FT04/FT06) is audited like code — the steering wheel's quality is load-bearing for the entire lineage."
    },
    {
      "id": "Q10", "bloom": "analysis", "type": "multiple_choice",
      "prompt": "Why is full-parameter fine-tuning (Hermes 3's choice) justified for an alignment-control job, but LoRA might be the better choice elsewhere?",
      "options": [
        "Full FT is always better than LoRA for every job.",
        "Full FT finds a higher-rank solution (FT00 evidence) that can make larger behavioral adjustments — justified when substantially shifting the model's character (Hermes 3's neutrality/steerability shift). For a lighter format steer, LoRA suffices and is far cheaper. The choice is governed by how much behavior you intend to move.",
        "LoRA is always better than full FT for alignment control.",
        "Full FT and LoRA produce identical weight matrices, so there's no difference."
      ],
      "answer_index": 1,
      "rationale": "Per the FT00 evidence (Shuttleworth et al., 'An Illusion of Equivalence'), full FT and LoRA find structurally different solutions — full FT higher-rank, LoRA low-rank. For a large behavioral shift (Hermes 3's alignment-control goal), the higher-rank full-FT path is justified despite its cost (405B full FT is a serious cluster job). For a light format steer, LoRA's low-rank path suffices and is far cheaper. The decision is governed by how much behavior you intend to move."
    }
  ]
}