Lab Specification — Module FTDD-06: Dolphin / Hermes

Course: Course 3 — LLM Fine-Tuning Masterclass Module: FTDD-06 — Dolphin / Hermes: Uncensored Lineages as Engineering Case Studies Duration: 30 minutes Environment: A web browser. No GPU required — this is a reading-and-reasoning lab. The artifacts are the model cards themselves.

Learning objectives

By the end of this lab you will have:

Read two production model cards as engineering artifacts — Dolphin3.0-R1-Mistral-24B and Hermes-3-Llama-3.1-8B — extracting the recipe, the base, the data, and the stated intent.
Mapped each recipe onto the Steering Stack and the FT modules — placing every stage at the correct layer and module (FT12 SFT, FT13 DPO, FT14/15 reasoning, FT16–18 alignment control).
Identified the capability-cost trade-off in each model card's eval section and stated it as a number, not an opinion.
Written a one-paragraph deployment case for each model that states (a) the task it suits, (b) the trade-off it carries, and (c) the harness requirement it imposes.

This lab is deliberately GPU-free. The skill it builds — reading a model card critically, mapping a recipe to the stack, and reasoning about trade-offs — is the skill you use every time you select a base model (FT03). It does not need a GPU; it needs judgment.

Phase 1 — Read the Dolphin3.0-R1-Mistral-24B card (10 min)

Open the model card: https://huggingface.co/cognitivecomputations/dolphin3.0-r1-mistral-24b (also published under the dphn org).

Read it actively, looking for:

Base model: What is it fine-tuned from? (Expected: Mistral Small 24B Instruct-2501.)
Reasoning lineage: How many reasoning traces, from what source, over how many rounds? (Expected: ~800K traces from the Dolphin-R1 dataset, distilled from DeepSeek-R1, over 3 rounds.)
Philosophy / character: How does the card describe the model's compliance orientation? (Expected: the "You are Dolphin..." system prompt; compliance over judgment.)
Intended use & limitations: What does the card say it is and is not for?
Eval section: Find any reported benchmarks. Note both strengths AND regressions versus the base.

Record in your report:

Base: ___
Reasoning injection: ___ traces from ___ over ___ rounds
Philosophy (one sentence): ___
One strength: ___
One regression or limitation: ___

Phase 2 — Read the Hermes-3-Llama-3.1-8B card (10 min)

Open: https://huggingface.co/NousResearch/Hermes-3-Llama-3.1-8B (and optionally the technical report, arXiv:2408.11857).

Read actively for:

Base model: Llama 3.1 8B (and note the family exists at 70B and 405B).
Recipe stages: SFT then DPO. Is it full-parameter or PEFT? (Expected: full-parameter.)
Data: What dataset family is referenced? (Expected: primarily synthetic; the OpenHermes 2.5 lineage.)
Stated intent: How does the card describe alignment? (Expected: "neutrally-aligned," "unlocked, uncensored, highly steerable.")
Eval section: Again, note strengths AND regressions.

Record in your report:

Base: ___
Recipe stages: ___ (full-param or PEFT?)
Dataset family: ___
Stated alignment intent (one sentence): ___
One strength: ___
One regression or limitation: ___

Phase 3 — Map both recipes to the Steering Stack and FT modules (5 min)

Fill in this table for each model. Place every stage at its layer and module.

Model	Base (Layer 1)	Layer 3 stages (FT module)	Reasoning (FT module)	Alignment control (FT module)	Harness (Layer 5) — whose job?
Dolphin3.0-R1-Mistral-24B	?	?	?	?	?
Hermes-3-Llama-3.1-8B	?	?	?	?	?

Model answers (for the solution key, not shown to the student):

Dolphin: Mistral Small 24B; SFT/DPO (FT12/FT13); R1-trace distillation (FT14/FT15); compliance-over-judgment via data curation (FT16–18); harness is the DEPLOYER's job.
Hermes 3: Llama 3.1 8B; full-param SFT then DPO (FT12/FT13); n/a (generalist, not R1-distilled); neutrality/steerability via SFT+DPO (FT16–18); harness is the DEPLOYER's job.

Phase 4 — Deployment case + trade-off (5 min)

For EACH model, write one paragraph (3–5 sentences) covering:

A task it suits — be specific (e.g., "an agentic tool-use workflow where false refusals break the chain").
The trade-off it carries — name the capability cost (FT17: up to −18.8pp GSM8K) or the steerability-raises-harness-requirement property.
The harness requirement it imposes — what policy gates / audit / threat model the deploying system must provide because the model will not self-refuse.

End your lab with a one-sentence restatement of the FT00/FT23 synthesis, using your two chosen tasks as the concrete examples.

Deliverables

Submit ftdd06-lab-report.md:

Phase 1: Dolphin card notes (base, reasoning injection, philosophy, one strength, one regression).
Phase 2: Hermes 3 card notes (base, recipe stages, dataset family, intent, one strength, one regression).
Phase 3: the completed mapping table (both models, all columns).
Phase 4: two deployment-case paragraphs (one per model) + the one-sentence synthesis.

Solution key

Phase 1 (Dolphin): Base = Mistral Small 24B (Instruct-2501). Reasoning = ~800K traces from Dolphin-R1 (distilled from DeepSeek-R1), 3 rounds. Philosophy = compliance over judgment ("You are Dolphin, an AI assistant... trained by Eric Hartford to specialize in reasoning and first-principles analysis"). A strength: strong reasoning + low false-refusal rate, good for agentic/tool-use. A regression: capability cost of uncensoring (math/eval regressions vs. base; FT17 up to −18.8pp GSM8K in the general finding).
Phase 2 (Hermes 3): Base = Llama 3.1 8B (family at 70B/405B). Recipe = full-param SFT then DPO. Dataset = primarily synthetic, OpenHermes 2.5 lineage (~1M examples). Intent = "neutrally-aligned, unlocked, uncensored, highly steerable." A strength: steerability + tool-use + neutral character. A regression: uncensoring trade-offs; steerability raises the harness requirement.
Phase 3: see the model-answer table in Phase 3 above. The key insight: in BOTH models, Layer 5 (the harness) is the deployer's responsibility — neither model ships its own safety bounds. That is the FT00/FT23 point.
Phase 4 (model structure):
- Dolphin case: suits an agentic tool-use workflow where false refusals break the chain (compliance + R1 reasoning). Trade-off: capability cost of uncensoring (FT17). Harness requirement: policy gates on the tools the agent may call, audit logging, a threat model — because the model will not self-refuse.
- Hermes 3 case: suits a steerable generalist instruct/tool-use deployment where the system prompt + harness define behavior. Trade-off: steerability raises the harness requirement — a weak harness does damage. Harness requirement: input/output policy, prompt-injection defenses, audit.
- Synthesis (one sentence): "Uncensor the model so it executes (Dolphin's compliance, Hermes's steerability); harness the model so it executes only what it should — the harness requirement is raised, not lowered, by uncensoring."

Stretch goals

Compare the 8B vs 405B Hermes 3 cards. Read both Hermes-3-Llama-3.1-8B and Hermes-3-Llama-3.1-405B cards. How do the eval tables and stated trade-offs differ by scale? Does the capability cost of uncensoring scale with model size? (Sets up FT10, full-FT-vs-PEFT at scale.)
Trace the OpenHermes lineage. Read the teknium/OpenHermes-2.5 dataset card. Identify three model families it trained. Write one sentence on how a single dataset propagated through a lineage — the dataset-as-steering-wheel thesis, documented. (Sets up FT04.)
Find the R1 upstream. Dolphin3.0-R1's reasoning traces come from DeepSeek-R1. Read the R1 model card (or its paper) and identify the RL method (GRPO) that produced the traces Dolphin distills. You have now traced a capability from R1's training → R1's traces → Dolphin's distillation. (Sets up FTDD-07, DeepSeek-R1.)

# Lab Specification — Module FTDD-06: Dolphin / Hermes **Course**: Course 3 — LLM Fine-Tuning Masterclass **Module**: FTDD-06 — Dolphin / Hermes: Uncensored Lineages as Engineering Case Studies **Duration**: 30 minutes **Environment**: A web browser. No GPU required — this is a reading-and-reasoning lab. The artifacts are the model cards themselves. --- ## Learning objectives By the end of this lab you will have: 1. **Read two production model cards as engineering artifacts** — Dolphin3.0-R1-Mistral-24B and Hermes-3-Llama-3.1-8B — extracting the recipe, the base, the data, and the stated intent. 2. **Mapped each recipe onto the Steering Stack and the FT modules** — placing every stage at the correct layer and module (FT12 SFT, FT13 DPO, FT14/15 reasoning, FT16–18 alignment control). 3. **Identified the capability-cost trade-off** in each model card's eval section and stated it as a number, not an opinion. 4. **Written a one-paragraph deployment case for each model** that states (a) the task it suits, (b) the trade-off it carries, and (c) the harness requirement it imposes. This lab is deliberately *GPU-free*. The skill it builds — reading a model card critically, mapping a recipe to the stack, and reasoning about trade-offs — is the skill you use every time you select a base model (FT03). It does not need a GPU; it needs judgment. --- ## Phase 1 — Read the Dolphin3.0-R1-Mistral-24B card (10 min) Open the model card: `https://huggingface.co/cognitivecomputations/dolphin3.0-r1-mistral-24b` (also published under the `dphn` org). Read it actively, looking for: - **Base model:** What is it fine-tuned from? (Expected: Mistral Small 24B Instruct-2501.) - **Reasoning lineage:** How many reasoning traces, from what source, over how many rounds? (Expected: ~800K traces from the Dolphin-R1 dataset, distilled from DeepSeek-R1, over 3 rounds.) - **Philosophy / character:** How does the card describe the model's compliance orientation? (Expected: the "You are Dolphin..." system prompt; compliance over judgment.) - **Intended use & limitations:** What does the card say it is and is not for? - **Eval section:** Find any reported benchmarks. Note both strengths AND regressions versus the base. **Record** in your report: - Base: ___ - Reasoning injection: ___ traces from ___ over ___ rounds - Philosophy (one sentence): ___ - One strength: ___ - One regression or limitation: ___ --- ## Phase 2 — Read the Hermes-3-Llama-3.1-8B card (10 min) Open: `https://huggingface.co/NousResearch/Hermes-3-Llama-3.1-8B` (and optionally the technical report, arXiv:2408.11857). Read actively for: - **Base model:** Llama 3.1 8B (and note the family exists at 70B and 405B). - **Recipe stages:** SFT then DPO. Is it full-parameter or PEFT? (Expected: full-parameter.) - **Data:** What dataset family is referenced? (Expected: primarily synthetic; the OpenHermes 2.5 lineage.) - **Stated intent:** How does the card describe alignment? (Expected: "neutrally-aligned," "unlocked, uncensored, highly steerable.") - **Eval section:** Again, note strengths AND regressions. **Record** in your report: - Base: ___ - Recipe stages: ___ (full-param or PEFT?) - Dataset family: ___ - Stated alignment intent (one sentence): ___ - One strength: ___ - One regression or limitation: ___ --- ## Phase 3 — Map both recipes to the Steering Stack and FT modules (5 min) Fill in this table for each model. Place every stage at its layer and module. | Model | Base (Layer 1) | Layer 3 stages (FT module) | Reasoning (FT module) | Alignment control (FT module) | Harness (Layer 5) — whose job? | | --- | --- | --- | --- | --- | --- | | Dolphin3.0-R1-Mistral-24B | ? | ? | ? | ? | ? | | Hermes-3-Llama-3.1-8B | ? | ? | ? | ? | ? | **Model answers** (for the solution key, not shown to the student): - Dolphin: Mistral Small 24B; SFT/DPO (FT12/FT13); R1-trace distillation (FT14/FT15); compliance-over-judgment via data curation (FT16–18); harness is the DEPLOYER's job. - Hermes 3: Llama 3.1 8B; full-param SFT then DPO (FT12/FT13); n/a (generalist, not R1-distilled); neutrality/steerability via SFT+DPO (FT16–18); harness is the DEPLOYER's job. --- ## Phase 4 — Deployment case + trade-off (5 min) For EACH model, write one paragraph (3–5 sentences) covering: 1. **A task it suits** — be specific (e.g., "an agentic tool-use workflow where false refusals break the chain"). 2. **The trade-off it carries** — name the capability cost (FT17: up to −18.8pp GSM8K) or the steerability-raises-harness-requirement property. 3. **The harness requirement it imposes** — what policy gates / audit / threat model the deploying system must provide because the model will not self-refuse. End your lab with a one-sentence restatement of the FT00/FT23 synthesis, using your two chosen tasks as the concrete examples. --- ## Deliverables Submit `ftdd06-lab-report.md`: - [ ] Phase 1: Dolphin card notes (base, reasoning injection, philosophy, one strength, one regression). - [ ] Phase 2: Hermes 3 card notes (base, recipe stages, dataset family, intent, one strength, one regression). - [ ] Phase 3: the completed mapping table (both models, all columns). - [ ] Phase 4: two deployment-case paragraphs (one per model) + the one-sentence synthesis. --- ## Solution key - **Phase 1 (Dolphin):** Base = Mistral Small 24B (Instruct-2501). Reasoning = ~800K traces from Dolphin-R1 (distilled from DeepSeek-R1), 3 rounds. Philosophy = compliance over judgment ("You are Dolphin, an AI assistant... trained by Eric Hartford to specialize in reasoning and first-principles analysis"). A strength: strong reasoning + low false-refusal rate, good for agentic/tool-use. A regression: capability cost of uncensoring (math/eval regressions vs. base; FT17 up to −18.8pp GSM8K in the general finding). - **Phase 2 (Hermes 3):** Base = Llama 3.1 8B (family at 70B/405B). Recipe = full-param SFT then DPO. Dataset = primarily synthetic, OpenHermes 2.5 lineage (~1M examples). Intent = "neutrally-aligned, unlocked, uncensored, highly steerable." A strength: steerability + tool-use + neutral character. A regression: uncensoring trade-offs; steerability raises the harness requirement. - **Phase 3:** see the model-answer table in Phase 3 above. The key insight: in BOTH models, Layer 5 (the harness) is the deployer's responsibility — neither model ships its own safety bounds. That is the FT00/FT23 point. - **Phase 4 (model structure):** - Dolphin case: suits an agentic tool-use workflow where false refusals break the chain (compliance + R1 reasoning). Trade-off: capability cost of uncensoring (FT17). Harness requirement: policy gates on the tools the agent may call, audit logging, a threat model — because the model will not self-refuse. - Hermes 3 case: suits a steerable generalist instruct/tool-use deployment where the system prompt + harness define behavior. Trade-off: steerability raises the harness requirement — a weak harness does damage. Harness requirement: input/output policy, prompt-injection defenses, audit. - Synthesis (one sentence): "Uncensor the model so it executes (Dolphin's compliance, Hermes's steerability); harness the model so it executes only what it should — the harness requirement is raised, not lowered, by uncensoring." --- ## Stretch goals 1. **Compare the 8B vs 405B Hermes 3 cards.** Read both `Hermes-3-Llama-3.1-8B` and `Hermes-3-Llama-3.1-405B` cards. How do the eval tables and stated trade-offs differ by scale? Does the capability cost of uncensoring scale with model size? (Sets up FT10, full-FT-vs-PEFT at scale.) 2. **Trace the OpenHermes lineage.** Read the `teknium/OpenHermes-2.5` dataset card. Identify three model families it trained. Write one sentence on how a single dataset propagated through a lineage — the dataset-as-steering-wheel thesis, documented. (Sets up FT04.) 3. **Find the R1 upstream.** Dolphin3.0-R1's reasoning traces come from DeepSeek-R1. Read the R1 model card (or its paper) and identify the RL method (GRPO) that produced the traces Dolphin distills. You have now traced a capability from R1's training → R1's traces → Dolphin's distillation. (Sets up FTDD-07, DeepSeek-R1.)