Medium Compare GGUF Fine-tune drift

Catastrophic forgetting, visible tensor by tensor

Scan report excerpt for Catastrophic forgetting, visible tensor by tensor

Two checkpoints from the same (toy) training run, three transformer blocks each: an early epoch and a late one where the run overcorrected. We rewrote every ffn_gate / ffn_up / ffn_down tensor across all 3 blocks, and left every attn_q / attn_k / attn_v / attn_output tensor and the embedding table completely untouched.

That's not an arbitrary split — it mirrors a real, common failure mode in fine-tuning: attention patterns (what the model attends to) staying stable while the feed-forward layers (what it does with that context) drift so far from the base model that general capability degrades, even though the architecture and every tensor shape are still identical.

The compare engine classifies this as FINE_TUNED — a broad, dense, distributed update (9 of 23 tensors, ~53% of weight values) rather than a localized edit, which is the correct call: this isn't one surgical change, it's the model's MLP layers being rewritten wholesale while attention is left alone. The per-tensor table makes the split immediately visible: identical rows for every attn_* tensor, changed rows for every ffn_* tensor.

See the real report

This isn't a mockup — the screenshot above is cropped from a real, cached compare report.