The Problem: Medical AI That Reasons, Not Recites
Large language models can summarize and rephrase medical literature, but clinical oncology demands more. Oncologists don’t just recall facts — they reason through mechanisms, weigh conflicting evidence, acknowledge uncertainty, and know when the data simply isn’t there. A model that confidently fabricates survival statistics or invents trial results isn’t just unhelpful — it’s dangerous.
This project builds a complete pipeline to fine-tune Qwen3 14B into a clinical oncology reasoning model that thinks before it answers, stays grounded in evidence, and refuses to hallucinate when the data runs out. The result is a two-phase training process — Supervised Fine-Tuning (SFT) followed by Direct Preference Optimization (DPO) — powered by synthetic data generated from over 100,000 PubMed abstracts across 11 cancer types.
Architecture Overview
The pipeline has three major stages, each implemented as a self-contained Jupyter notebook:
| Stage | Notebook | Purpose |
|---|---|---|
| Data Generation | pubmed_datagen_v2 | Generate synthetic training conversations from PubMed abstracts |
| Phase 1: SFT | pubmed_sft_training | Teach the model clinical oncology reasoning with thinking chains |
| Phase 2: DPO | pubmed_dpo_training | Refine response quality by teaching the model what NOT to say |
Each notebook is designed to run end-to-end with a single “Run All” — no manual intervention required.
Stage 1: Synthetic Data Generation
The datagen notebook is the backbone of the entire pipeline. It transforms raw PubMed literature into high-quality, multi-turn training conversations with built-in anti-hallucination safeguards.
Source Data
Two datasets feed the pipeline:
- PubMed Cancer NLP Dataset — ~100,000 title + abstract pairs across 10 cancer types (bone, brain, breast, colon, gastric, kidney, lung, ovarian, prostate, skin cancer)
- Microsoft CancerGUIDE — 316 synthetic oncology patient cases with treatment recommendations
A preprocessing script downloads, cleans, and deduplicates the raw data: normalizing Unicode, stripping retractions, filtering non-English entries, and removing abstracts that are too short or too long.
Question-Answer Generation with Chain-of-Thought
The core generation loop processes each cancer type independently:
-
Sentence-aware chunking — PubMed abstracts are split into passages using pySBD with medical abbreviation protection (handling “et al.”, “i.v.”, “vs.”, and dozens of clinical abbreviations that would otherwise cause false sentence breaks)
-
Three-round question generation — For each chunk, a lightweight model (Qwen 2.5 7B) generates questions across three clinical dimensions:
- Mechanistic — molecular pathways, biomarkers, genetic mutations, mechanisms of action
- Clinical Application — treatment decisions, staging, prognostic factors, trial outcomes
- Translational — bench-to-bedside implications, guideline integration, real-world applicability
-
Thinking-model answers — Each question is answered by Qwen3 235B, a thinking model that produces
<think>...</think>reasoning blocks before delivering its answer. These reasoning chains are preserved in the training data — the fine-tuned model learns to show its work, reasoning through mechanisms, weighing evidence, and considering differential diagnoses before responding.
The Anti-Hallucination Strategy
This is where the pipeline goes beyond standard synthetic data generation. Inspired by Augmentoolkit, three complementary mechanisms train the model to stay grounded:
Answer Grounding Validation
Every generated QA pair is checked by an LLM judge that compares the answer against the source abstract. The judge assigns one of three verdicts:
- Grounded — all claims trace back to the abstract
- Extrapolated — plausible but unsupported claims (kept with a flag)
- Hallucinated — fabricated statistics, trial names, or specific claims (rejected from SFT, saved for DPO)
This catches the thinking model’s tendency to embellish with plausible-sounding but unsupported details — a particularly insidious failure mode in medical contexts.
“Beyond the Evidence” QA
For a sample of chunks, the pipeline generates questions that the abstract cannot answer — questions about unstudied populations, long-term outcomes not reported, or comparative data not available. The thinking model then generates honest refusal responses that explain why the evidence is insufficient and what additional data would be needed.
This directly trains boundary awareness: the model learns to say “the available evidence doesn’t address this” instead of confabulating.
Self-Correction Sequences
Four-turn conversations where the model gives a deliberately flawed initial answer, the user pushes back, and the model corrects itself with proper reasoning:
- System → oncologist persona
- Human → asks a question
- GPT → gives a subtly wrong answer (this turn is masked during training — the model never learns to produce it)
- Human → challenges the error
- GPT → acknowledges the mistake and provides a corrected, evidence-grounded answer
The masking is critical: the model only learns the recovery pattern, never the error.
Additional Data Types
- Treatment reasoning — CancerGUIDE patient cases are processed into multi-turn clinical decision-making conversations, combining patient history with treatment recommendations
- Continuation chunks — Raw abstract text is formatted into language-modeling examples (no API calls needed), teaching the model PubMed’s vocabulary and writing patterns
Assembly
All data types are merged into a single ShareGPT-format JSONL file with cancer-type-specific system prompts:
- Standard QA (with thinking chains)
- Beyond-evidence refusal conversations
- Self-correction sequences
- Treatment reasoning dialogues
- Continuation passages
The final dataset contains ~33,000 multi-turn conversations across 11 cancer types.
Stage 2: Supervised Fine-Tuning (SFT)
The SFT notebook takes the assembled dataset and trains the model to reason like a clinical oncologist.
Model & Configuration
| Parameter | Value |
|---|---|
| Base model | Qwen3 14B (4-bit quantized via Unsloth) |
| Precision | 4-bit QLoRA (NF4) |
| LoRA rank | 32 |
| LoRA alpha | 32 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Sequence length | 4,096 tokens |
| Effective batch size | 8 (batch 2 × gradient accumulation 4) |
| Learning rate | 2e-4 |
| Optimizer | AdamW 8-bit |
Manual Sequence Packing
Rather than padding each conversation to max_seq_length (wasting tokens on padding), the SFT notebook implements manual sequence packing:
- All conversations are tokenized and concatenated into a single token stream, separated by EOS tokens
- The stream is split into fixed-size chunks of exactly
MAX_SEQ_LENGTHtokens - Each chunk becomes one training example with ~100% token utilization — zero padding waste
This dramatically improves training efficiency. With 33,000 conversations producing millions of tokens, packing eliminates what would otherwise be substantial wasted compute on padding tokens.
Training Hardware
The entire pipeline runs on an NVIDIA DGX Spark with 128GB of unified memory. Using Unsloth’s pre-quantized 4-bit checkpoint, the 14B parameter model fits comfortably with room for gradients, optimizer states, and batch processing. (For a deeper look at how Unsloth enables this, see the Unsloth post.)
Verification
After training, the notebook performs two verification steps:
- Live inference — Tests the model on oncology questions across multiple cancer types, verifying that responses include
<think>reasoning blocks and cancer-type-specific knowledge - Cold reload — Clears the model from GPU memory, reloads from saved LoRA adapters, and runs inference again to confirm the adapter is portable and loads cleanly
Stage 3: Direct Preference Optimization (DPO)
DPO is Phase 2 — it doesn’t teach the model new knowledge, but teaches it what not to say. The DPO notebook takes the SFT-trained LoRA and refines it using preference pairs.
What DPO Learns
Each DPO training example is a pair: a chosen (good) response and a rejected (bad) response to the same prompt. The model learns to increase the probability of chosen responses and decrease the probability of rejected ones.
The DPO data comes directly from the datagen pipeline:
- Grounding rejects — Answers the grounding judge flagged as hallucinated become “rejected” examples; re-generated grounded answers become “chosen” examples
- Self-correction pairs — The flawed answers become “rejected”; the corrected answers become “chosen”
- Beyond-evidence pairs — Confabulated answers (if any) become “rejected”; honest refusals become “chosen”
Key Differences from SFT
| Parameter | SFT | DPO |
|---|---|---|
| Learning rate | 2e-4 | 5e-6 (40× lower) |
| Batch size | 2 × 4 = 8 | 1 × 8 = 8 |
| Purpose | Learn clinical reasoning | Refine response quality |
| Loss function | Cross-entropy | Sigmoid DPO loss |
| Warmup | 5 steps | 10% of total steps |
| Data format | Packed token sequences | Prompt + chosen/rejected pairs |
The much lower learning rate is intentional — DPO is a fine adjustment, not a major behavioral shift. The model already knows oncology from SFT; DPO teaches it to prefer grounded, honest responses.
Single LoRA Output
A key design choice: the DPO notebook continues training the existing SFT LoRA weights rather than stacking a new adapter on top. The result is a single LoRA adapter relative to base Qwen3 14B that contains both SFT and DPO training. This simplifies deployment — load the base model plus one adapter, no merging or stacking required.
Behavioral Evaluation
After training, the notebook tests three specific DPO-trained behaviors:
- Grounding — Given a question about specific trial data, does the model stay evidence-based without inventing statistics?
- Boundary awareness — Given a question that extrapolates far beyond a single case report, does the model acknowledge the limitations?
- Self-correction — Given a factually incorrect prior claim, does the model correct the error rather than doubling down?
Expected Outcomes
What the Model Should Do
- Reason through clinical questions with visible
<think>blocks showing mechanism analysis, evidence weighing, and clinical reasoning before delivering answers - Stay grounded in evidence — cite what the literature supports rather than embellishing with plausible-sounding fabrications
- Acknowledge uncertainty — use hedging language (“the evidence suggests,” “based on available data”) and explicitly flag limitations
- Refuse appropriately — when asked about something the data doesn’t cover, explain the gap rather than confabulate
- Self-correct — when challenged on an error, acknowledge the mistake and provide a corrected, evidence-grounded response
- Specialize by cancer type — use cancer-type-specific system prompts to draw on relevant biology, staging systems, and treatment guidelines
What the Model Should Not Do
- Fabricate specific statistics, trial names, or patient outcomes
- Provide confident answers when evidence is insufficient
- Double down on errors when challenged
- Use generic responses that could apply to any disease
- Drop the reasoning chain and give answer-only responses
Deployment
The final LoRA adapter can be served directly with vLLM:
vllm serve Qwen/Qwen3-14B \
--enable-lora \
--lora-modules oncologist=/path/to/lora_adaptersThe adapter is compact (a few hundred MB vs. the full 14B model) and can be hot-swapped or served alongside other LoRA adapters on the same base model.
Why This Approach
The pipeline reflects several deliberate design choices:
Thinking model for data generation — Using Qwen3 235B (a thinking model) to generate answers means the training data naturally includes chain-of-thought reasoning. The fine-tuned 14B model inherits this behavior without needing explicit reasoning prompts.
Anti-hallucination as a first-class concern — Rather than hoping the model learns accuracy from examples alone, the pipeline includes three distinct mechanisms (grounding checks, beyond-evidence training, self-correction sequences) that directly address the most dangerous failure modes in medical AI.
Two-phase training — SFT teaches what to know; DPO teaches what not to say. This separation is particularly important for medical applications where the cost of a confident wrong answer is much higher than the cost of an honest “I don’t know.”
No frameworks — The datagen notebook uses just the openai library and asyncio for batching. No LangChain, no DSPy, no orchestration frameworks. This keeps the pipeline transparent, debuggable, and easy to modify.
Run-and-walk-away design — Every notebook runs end-to-end with “Run All.” Resume logic and checkpointing mean you can restart without losing progress. The target user is a solo practitioner or small team who needs to fire and forget.
What’s Next
This pipeline produces a Phase 1 (SFT) and Phase 2 (DPO) fine-tuned model. Future work includes:
- Expanded cancer types beyond the current 11, incorporating rarer malignancies and hematological cancers
- Clinical guidelines integration — training on NCCN, ASCO, and ESMO guideline summaries for current standard-of-care alignment
- Multi-modal extensions — incorporating pathology image descriptions and radiology report interpretation
- Continuous learning — automated pipelines that periodically pull new PubMed abstracts and generate incremental training data
The full notebook pipeline is open source and available on GitHub. Feedback and contributions are welcome.