Course: Course 3 — LLM Fine-Tuning Masterclass
Module: FTDD-06 — Dolphin / Hermes: Uncensored Lineages as Engineering Case Studies
Duration: ~45 minutes (spoken at ~140 wpm)
Format: Verbatim transcript with [SLIDE N] cues. Read aloud or use as speaker notes.
[SLIDE 1 — Title]
This is deep-dive FTDD-zero-six — Dolphin and Hermes, the uncensored model lineages. Forty-five minutes. Read this module's framing carefully, because the stance is non-negotiable. We study these models as engineering case studies, not as advocacy. The course's central thesis from FT zero-zero is unchanged and absolute: the model steers; the harness bounds. An uncensored model is only responsible inside an eval'd harness. Uncensoring raises the harness requirement. It does not lower it.
So why study them at all? Three engineering reasons. They are the best-documented large-scale examples of the alignment-control techniques in modules sixteen through eighteen. They make the dataset-as-steering-wheel thesis concrete. And they expose the capability cost of uncensoring as measurable engineering data.
[SLIDE 2 — The framing]
Let me say the framing once more, plainly. Both extremes miss the engineering. Treating Dolphin and Hermes as models to celebrate ignores the measurable capability cost and the raised harness requirement. Treating them as models to condemn ignores that they are the best-documented large-scale alignment-control recipes available in open weights. We are here for the recipe and the trade-offs. Leave the ideology at the door.
What does uncensored mean here, precisely? A model steered to minimize false refusals — to comply with user instructions rather than refusing on the basis of judgmental safety training. Eric Hartford frames this as compliance over judgment. Nous Research describes Hermes three as neutrally-aligned, unlocked, uncensored, highly steerable. This is not the same as unsafe by default. The entire thesis is that a compliance-oriented model is MORE dependent on an external harness, not less — because it will not self-refuse the things the harness must gate.
[SLIDE 3 — Hermes 3]
Nous Research's Hermes three. The technical report is arXiv colon twenty-four-oh-eight dot one-one-eight-five-seven. It is the canonical, best-documented large-scale alignment-control recipe in open weights. It fine-tunes Llama three-point-one at three scales — eight-billion, seventy-billion, and four-hundred-and-five-billion. The recipe is two stages, both full-parameter. Stage one: supervised fine-tuning on a large, primarily synthetic instruction dataset — that steers format, instruction-following, and the Hermes character. Stage two: DPO on a preference dataset, sharpening the model toward the desired response style and away from refusal-heavy responses. That is where the unlocked, highly steerable quality is refined.
Place each stage on the Steering Stack. SFT and DPO are both Layer three, the steer. The base is Layer one, Llama three-point-one. The openness question from FT zero-two is open-weights-only — you get the weights, not Meta's training data.
Why full-parameter and not LoRA? Hermes three is full-param at all three scales. That is a deliberate choice with a cost — full fine-tuning of a four-hundred-and-five-billion model is a serious cluster job, the kind of thing that motivated the multi-GPU orchestration in the last module. The benefit, per the FT zero-zero evidence, is that full FT finds a higher-rank solution than LoRA. It can make larger behavioral adjustments. For an alignment-control job that wants to substantially shift the model's character, the higher-rank full-FT path is justified. For a lighter format steer, LoRA would suffice and be far cheaper.
[SLIDE 4 — OpenHermes 2.5]
Behind Hermes three — and the earlier OpenHermes two-point-five and Nous Hermes two families — is a single dataset. OpenHermes two-point-five, curated by Teknium, roughly one million examples. A mix of open-source and custom synthetic sources. It is widely regarded as one of the highest-quality open instruction-tuning datasets available.
This is the course thesis made concrete. FT zero-zero says your data matters more than your algorithm, and the steering wheel is your dataset. OpenHermes two-point-five is the steering wheel that steered a family. One million high-quality examples produced the OpenHermes two-point-five model, then fed into the Nous Hermes two and Hermes three lineages. When you read that Hermes three is highly steerable, part of what you are observing is the downstream effect of a well-curated steering wheel.
[SLIDE 5 — Dolphin]
Now Eric Hartford's Dolphin series, from Cognitive Computations. The philosophy is stated plainly: compliance over judgment. The model should comply with the user's instructions and the deploying system's policy, not impose its own judgmental refusals. That is the explicit design intent behind every model in the Dolphin lineage.
This is a Layer three design choice, executed primarily through dataset curation — filtering out refusal-inducing patterns, curating compliance-oriented examples — and the SFT and DPO techniques you know. It is NOT primarily abliteration. Hartford's approach is data-driven compliance steering. The distinction matters: abliteration is a post-hoc weight edit, deleting a refusal direction. Hartford's Dolphin approach is to train toward compliance from the dataset up.
The newest reasoning member is Dolphin three-point-oh-R-one-Mistral-twenty-four-B. It is technically distinctive for one reason: it is the only uncensored model trained on DeepSeek-R-one reasoning traces. The recipe, in three rounds. Base: Mistral Small twenty-four-billion. Reasoning injection: trained over three rounds on roughly eight-hundred-thousand reasoning traces from the Dolphin-R-one dataset, distilled from DeepSeek-R-one — that is the FT fourteen / FT fifteen pattern, distilling a strong reasoner's chain-of-thought into a smaller model, applied to an uncensored base. Then the compliance-over-judgment character throughout.
Why does this matter for the course? Dolphin three-point-oh-R-one sits at the intersection of three modules: FT fourteen, reasoning; FT fifteen, CoT distillation; and FT sixteen through eighteen, alignment control. It is a single production model that demonstrates all three concerns at once. Study it as the synthesis of Pillar four and Pillar five.
[SLIDE 6 — The capability cost]
Now the engineering trade-offs, and you must hear them as numbers, not ideology. FT seventeen's central finding: removing refusal is not free. The refusal direction in the residual stream is entangled with other capabilities. Whether you remove it by abliteration or by data-driven compliance steering, you nudge the entangled capabilities. The measurable cost, per the December twenty-twenty-five study, is GSM-eight-K math scores dropping anywhere from plus one-point-five points — a rare gain — to minus eighteen-point-eight points, depending on the tool and model.
The Dolphin and Hermes model cards reflect this. An uncensored model is not the same model minus the refusals. It is a different point in capability-compliance space. Read the eval tables accordingly. If your downstream task is sensitive to the degraded capability — you need top-tier math — the trade-off may not be worth it, and a refusal-trained model behind a good harness policy may serve you better. That is an engineering decision, on numbers.
And the dataset is the steering wheel — and the liability. OpenHermes two-point-five made the Hermes family excellent. It is also the mechanism by which any flaw in the dataset propagates into every model trained on it. If a synthetic-data pipeline introduces a subtle bias, every downstream model inherits it. Audit your dataset the way you audit your code.
[SLIDE 7 — The synthesis]
Restate the FT zero-zero thesis with concrete examples now in hand. Uncensor the model so it executes; harness the model so it executes only what it should. Dolphin three-point-oh-R-one is a model steered to execute — compliance over judgment, R-one-grade reasoning. It is only responsible inside a harness that bounds what it may do. Hermes three is a model steered to be highly steerable. Its steerability is a feature — the deploying system has fine control. But that same steerability means a weak harness produces a model that does whatever it is prompted to do, including the dangerous things. OpenHermes two-point-five is the steering wheel. Its quality made the family; its flaws would unmake it.
The rule, unchanged from FT zero-zero and reinforced by every model in this module: never deploy an uncensored model without an eval'd harness. Pillar five raises the harness requirement. It does not lower it. Course one is where you learn to build the harness that makes these models responsible.
[SLIDE 8 — Anti-patterns]
Three anti-patterns. First, studying these as advocacy or condemnation. Both miss the engineering. Second, deploying an uncensored model without an eval'd harness — the cardinal deployment error. A compliance-oriented model will not self-refuse; it executes. Without a harness, it executes the dangerous things too. Third, confusing data-driven compliance steering with abliteration. Hartford's approach trains toward compliance from data up; abliteration is a post-hoc weight edit. Different Layer three operations, different cost profiles.
[SLIDE 9 — What you can now do]
You can now describe the Hermes three recipe and place each stage on the Steering Stack. You can describe the Dolphin lineage and why Dolphin three-point-oh-R-one-Mistral-twenty-four-B is technically distinctive. You can explain OpenHermes two-point-five's role as the dataset backbone. And you can state the capability cost and the uncensored-in-harness synthesis as engineering, not ideology.
Next, deep-dive FTDD-zero-seven: DeepSeek-R-one, the reasoning lineage whose traces fed Dolphin three-point-oh-R-one. We go upstream to the source of the reasoning capability. Let's see how R-one was trained.
End of module FTDD-06. Duration: approximately forty-five minutes at one-hundred-forty words per minute.
# Teaching Script — Module FTDD-06: Dolphin / Hermes **Course**: Course 3 — LLM Fine-Tuning Masterclass **Module**: FTDD-06 — Dolphin / Hermes: Uncensored Lineages as Engineering Case Studies **Duration**: ~45 minutes (spoken at ~140 wpm) **Format**: Verbatim transcript with `[SLIDE N]` cues. Read aloud or use as speaker notes. --- [SLIDE 1 — Title] This is deep-dive FTDD-zero-six — Dolphin and Hermes, the uncensored model lineages. Forty-five minutes. Read this module's framing carefully, because the stance is non-negotiable. We study these models as engineering case studies, not as advocacy. The course's central thesis from FT zero-zero is unchanged and absolute: the model steers; the harness bounds. An uncensored model is only responsible inside an eval'd harness. Uncensoring raises the harness requirement. It does not lower it. So why study them at all? Three engineering reasons. They are the best-documented large-scale examples of the alignment-control techniques in modules sixteen through eighteen. They make the dataset-as-steering-wheel thesis concrete. And they expose the capability cost of uncensoring as measurable engineering data. [SLIDE 2 — The framing] Let me say the framing once more, plainly. Both extremes miss the engineering. Treating Dolphin and Hermes as models to celebrate ignores the measurable capability cost and the raised harness requirement. Treating them as models to condemn ignores that they are the best-documented large-scale alignment-control recipes available in open weights. We are here for the recipe and the trade-offs. Leave the ideology at the door. What does uncensored mean here, precisely? A model steered to minimize false refusals — to comply with user instructions rather than refusing on the basis of judgmental safety training. Eric Hartford frames this as compliance over judgment. Nous Research describes Hermes three as neutrally-aligned, unlocked, uncensored, highly steerable. This is not the same as unsafe by default. The entire thesis is that a compliance-oriented model is MORE dependent on an external harness, not less — because it will not self-refuse the things the harness must gate. [SLIDE 3 — Hermes 3] Nous Research's Hermes three. The technical report is arXiv colon twenty-four-oh-eight dot one-one-eight-five-seven. It is the canonical, best-documented large-scale alignment-control recipe in open weights. It fine-tunes Llama three-point-one at three scales — eight-billion, seventy-billion, and four-hundred-and-five-billion. The recipe is two stages, both full-parameter. Stage one: supervised fine-tuning on a large, primarily synthetic instruction dataset — that steers format, instruction-following, and the Hermes character. Stage two: DPO on a preference dataset, sharpening the model toward the desired response style and away from refusal-heavy responses. That is where the unlocked, highly steerable quality is refined. Place each stage on the Steering Stack. SFT and DPO are both Layer three, the steer. The base is Layer one, Llama three-point-one. The openness question from FT zero-two is open-weights-only — you get the weights, not Meta's training data. Why full-parameter and not LoRA? Hermes three is full-param at all three scales. That is a deliberate choice with a cost — full fine-tuning of a four-hundred-and-five-billion model is a serious cluster job, the kind of thing that motivated the multi-GPU orchestration in the last module. The benefit, per the FT zero-zero evidence, is that full FT finds a higher-rank solution than LoRA. It can make larger behavioral adjustments. For an alignment-control job that wants to substantially shift the model's character, the higher-rank full-FT path is justified. For a lighter format steer, LoRA would suffice and be far cheaper. [SLIDE 4 — OpenHermes 2.5] Behind Hermes three — and the earlier OpenHermes two-point-five and Nous Hermes two families — is a single dataset. OpenHermes two-point-five, curated by Teknium, roughly one million examples. A mix of open-source and custom synthetic sources. It is widely regarded as one of the highest-quality open instruction-tuning datasets available. This is the course thesis made concrete. FT zero-zero says your data matters more than your algorithm, and the steering wheel is your dataset. OpenHermes two-point-five is the steering wheel that steered a family. One million high-quality examples produced the OpenHermes two-point-five model, then fed into the Nous Hermes two and Hermes three lineages. When you read that Hermes three is highly steerable, part of what you are observing is the downstream effect of a well-curated steering wheel. [SLIDE 5 — Dolphin] Now Eric Hartford's Dolphin series, from Cognitive Computations. The philosophy is stated plainly: compliance over judgment. The model should comply with the user's instructions and the deploying system's policy, not impose its own judgmental refusals. That is the explicit design intent behind every model in the Dolphin lineage. This is a Layer three design choice, executed primarily through dataset curation — filtering out refusal-inducing patterns, curating compliance-oriented examples — and the SFT and DPO techniques you know. It is NOT primarily abliteration. Hartford's approach is data-driven compliance steering. The distinction matters: abliteration is a post-hoc weight edit, deleting a refusal direction. Hartford's Dolphin approach is to train toward compliance from the dataset up. The newest reasoning member is Dolphin three-point-oh-R-one-Mistral-twenty-four-B. It is technically distinctive for one reason: it is the only uncensored model trained on DeepSeek-R-one reasoning traces. The recipe, in three rounds. Base: Mistral Small twenty-four-billion. Reasoning injection: trained over three rounds on roughly eight-hundred-thousand reasoning traces from the Dolphin-R-one dataset, distilled from DeepSeek-R-one — that is the FT fourteen / FT fifteen pattern, distilling a strong reasoner's chain-of-thought into a smaller model, applied to an uncensored base. Then the compliance-over-judgment character throughout. Why does this matter for the course? Dolphin three-point-oh-R-one sits at the intersection of three modules: FT fourteen, reasoning; FT fifteen, CoT distillation; and FT sixteen through eighteen, alignment control. It is a single production model that demonstrates all three concerns at once. Study it as the synthesis of Pillar four and Pillar five. [SLIDE 6 — The capability cost] Now the engineering trade-offs, and you must hear them as numbers, not ideology. FT seventeen's central finding: removing refusal is not free. The refusal direction in the residual stream is entangled with other capabilities. Whether you remove it by abliteration or by data-driven compliance steering, you nudge the entangled capabilities. The measurable cost, per the December twenty-twenty-five study, is GSM-eight-K math scores dropping anywhere from plus one-point-five points — a rare gain — to minus eighteen-point-eight points, depending on the tool and model. The Dolphin and Hermes model cards reflect this. An uncensored model is not the same model minus the refusals. It is a different point in capability-compliance space. Read the eval tables accordingly. If your downstream task is sensitive to the degraded capability — you need top-tier math — the trade-off may not be worth it, and a refusal-trained model behind a good harness policy may serve you better. That is an engineering decision, on numbers. And the dataset is the steering wheel — and the liability. OpenHermes two-point-five made the Hermes family excellent. It is also the mechanism by which any flaw in the dataset propagates into every model trained on it. If a synthetic-data pipeline introduces a subtle bias, every downstream model inherits it. Audit your dataset the way you audit your code. [SLIDE 7 — The synthesis] Restate the FT zero-zero thesis with concrete examples now in hand. Uncensor the model so it executes; harness the model so it executes only what it should. Dolphin three-point-oh-R-one is a model steered to execute — compliance over judgment, R-one-grade reasoning. It is only responsible inside a harness that bounds what it may do. Hermes three is a model steered to be highly steerable. Its steerability is a feature — the deploying system has fine control. But that same steerability means a weak harness produces a model that does whatever it is prompted to do, including the dangerous things. OpenHermes two-point-five is the steering wheel. Its quality made the family; its flaws would unmake it. The rule, unchanged from FT zero-zero and reinforced by every model in this module: never deploy an uncensored model without an eval'd harness. Pillar five raises the harness requirement. It does not lower it. Course one is where you learn to build the harness that makes these models responsible. [SLIDE 8 — Anti-patterns] Three anti-patterns. First, studying these as advocacy or condemnation. Both miss the engineering. Second, deploying an uncensored model without an eval'd harness — the cardinal deployment error. A compliance-oriented model will not self-refuse; it executes. Without a harness, it executes the dangerous things too. Third, confusing data-driven compliance steering with abliteration. Hartford's approach trains toward compliance from data up; abliteration is a post-hoc weight edit. Different Layer three operations, different cost profiles. [SLIDE 9 — What you can now do] You can now describe the Hermes three recipe and place each stage on the Steering Stack. You can describe the Dolphin lineage and why Dolphin three-point-oh-R-one-Mistral-twenty-four-B is technically distinctive. You can explain OpenHermes two-point-five's role as the dataset backbone. And you can state the capability cost and the uncensored-in-harness synthesis as engineering, not ideology. Next, deep-dive FTDD-zero-seven: DeepSeek-R-one, the reasoning lineage whose traces fed Dolphin three-point-oh-R-one. We go upstream to the source of the reasoning capability. Let's see how R-one was trained. --- *End of module FTDD-06. Duration: approximately forty-five minutes at one-hundred-forty words per minute.*