Fine tuning LLM performance: 5 factors that drive success

Five factors actually drive fine-tuning success — data quality, parameter-efficient methods, hyperparameter discipline, alignment strategy, and how we measure the result. We walk through each one with the gotchas, the workarounds, and the boilerplate that tends to trip teams up. No abstract theory. Code-side specifics only.

The Data Quality Imperative: Why Less Is Often More

The single biggest lever in fine-tuning isn't the model, the rank, or the learning rate — it's the dataset. Curated examples that reflect the exact output shape you want will outperform a noisy ten-times-larger corpus almost every time. The "less is more" principle shows up consistently in practice: a clean 5,000-example set frequently beats a 50,000-example dump scraped from logs.

We treat data work as engineering work. That means a sanity-check pipeline before training starts — deduplication, format validation, label balance checks, and a spot-read of a random 100 examples to catch silent corruption. One team we worked with spent two weeks tuning learning rates before realizing 12% of their training set had truncated assistant turns. The model wasn't broken. The data was.

Curate like an editor, not a scraper. A smaller, validated dataset beats a larger, noisy one in nearly every fine-tuning outcome.

Three practical checks before we touch a single hyperparameter:

• Deduplicate aggressively — near-duplicates inflate validation loss and inflate benchmark optimism in equal measure.
• Validate format end-to-end — tokenization should match what the serving stack will see at inference, including special tokens and chat templates.
• Stratify by intent — if the eval set covers eight user intents, the training set needs to cover the same eight, not just the three most common.

The mistake we see most often is treating data preparation as a one-time cleanup. It's an ongoing loop — as production edge cases surface, they go back into the training set with a label and a regression test. That loop is what separates a fine-tuned model from a fine-tuned checkbox. The dataset isn't a deliverable; it's a living artifact.

Dataset sizing is task-dependent, and we don't pretend otherwise. A classification head on 2,000 examples can hit strong numbers. A free-form generation task on a narrow domain usually needs 10,000+ examples to generalize. The validation set tells us when we've crossed the line — if validation loss diverges from training loss before the second epoch, the dataset is too small or too noisy for the rank we've chosen.

Parameter-Efficient Fine-Tuning: LoRA and QLoRA in Practice

Full fine-tuning a frontier model burns GPU memory and engineering time we usually don't have. Parameter-Efficient Fine-Tuning (PEFT) sidesteps that by freezing the pre-trained weights and injecting small trainable rank-decomposition matrices into the attention layers. LoRA — Low-Rank Adaptation — is the most widely deployed version. It trains less than 1% of total parameters in most configurations and lets us keep one base model with dozens of task-specific adapters stacked on top.

The rank parameter r is the first dial we'll set. In practice, r typically ranges from 8 to 64. Lower r trains faster and generalizes more aggressively — useful for broad style shifts. Higher r captures more task-specific nuance but risks overfitting on small datasets. A common starting point is r=16 with alpha=32, then we adjust based on validation loss and a held-out domain eval.

QLoRA pushes the same idea further by loading the base model in 4-bit precision. That brings a 33B parameter model down to roughly 12GB of GPU memory — consumer-grade hardware territory. The performance gap versus full-precision fine-tuning is small for most text tasks, which is why QLoRA has become the default for teams running on a single A100 or a 4090.

The boilerplate for getting started is short — peft from Hugging Face, LoraConfig with r, alpha, target_modules, and task_type, then wrap the base model with get_peft_model. The gotcha most teams hit: forgetting to set target_modules correctly for the architecture they're using. Different model families expect different module names. Run print([n for n, _ in model.named_modules()]) once to confirm — that's the sanity check that saves a two-hour debugging session.

One nuance worth flagging: PEFT is for style and task alignment, not knowledge injection. If the base model has never seen a concept, no amount of LoRA fine-tuning will teach it that concept — at best, you'll get fluent hallucination. Fine-tuning makes a capable model behave the way you want; it does not retrofit knowledge the model lacks. Teams that hit this wall usually need retrieval augmentation on top of fine-tuning, not more epochs.

Navigating Hyperparameter Optimization and Learning Rates

The most common mistake we see in fine-tuning is using a learning rate that was correct for pre-training. Pre-training uses aggressive rates designed to move a random initialization toward useful representations. Fine-tuning is adjustment — small shifts on top of an already-capable model. A learning rate 10x to 100x smaller than the pre-training rate is the standard range, often 1e-5 to 5e-5 for full fine-tuning and 1e-4 to 3e-4 for LoRA adapters.

Learning rate is not the only dial. The combination of rank, dropout, batch size, and scheduler matters as a system. We treat the validation loss curve as the source of truth — a flat curve at high loss means the rate is too low; a curve that drops fast then bounces means the rate is too high or the rank is overfitting on a small set.

Three hyperparameters get most of the attention:

• Learning rate — start at 1e-4 for LoRA, scale by dataset size. Larger datasets tolerate larger rates; smaller datasets need smaller rates.
• Effective batch size — after gradient accumulation, usually 16 to 64. Smaller batches add regularization; larger batches stabilize convergence.
• Number of epochs — two to four for most curated datasets. More than that and the model starts memorizing surface patterns instead of learning the task structure.

A common workaround when validation loss plateaus early: lower the learning rate by a factor of three and run one more epoch. This often unlocks another drop in loss without restarting from scratch. The trick is to log enough checkpoints — every 200 steps at minimum — to make that rollback possible.

Catastrophic forgetting is the other gotcha. If fine-tuning on a narrow domain destroys the model's general capabilities — measurable on a held-out general eval — the rank is too high or the dataset is too narrow. The fix is usually a small amount of general-domain data mixed into the training set, plus a lower rank. We also drop in a small evaluation of MMLU as a regression check before each checkpoint promotion. If general accuracy drops by more than two points, the checkpoint doesn't ship.

There's no universal "best" hyperparameter configuration across architectures. A 7B base model and a 70B base model with the same dataset will not converge at the same rate, the same rank, or the same batch size. The boilerplate gets you to a starting point. The validation curve gets you to the finish.

Alignment Strategies: Integrating RLHF for Human Intent

Supervised fine-tuning teaches the model what output to produce. RLHF — Reinforcement Learning from Human Feedback — teaches it which outputs to prefer when multiple are plausible. The two are complementary, not interchangeable, and the order matters: we run supervised fine-tuning first, then layer preference optimization on top.

The standard pipeline trains a reward model on human preference data — humans rank several model outputs for the same prompt, the reward model learns to predict those rankings, and then a policy optimization step tunes the model toward higher-reward outputs. The InstructGPT work in 2022 put this approach on the map, and it remains the dominant alignment pattern in production.

Three practical points we lean on when implementing RLHF:

• The reward model is the bottleneck — garbage preference data produces a garbage reward signal. We invest more in preference-label quality than in policy optimization tricks.
• KL divergence between the policy and the reference model needs explicit control — too loose and the policy drifts off-distribution, too tight and it stops learning.
• RLHF is expensive — for most production teams, doing one strong RLHF pass after supervised fine-tuning is the right tradeoff versus repeated rounds.

A workaround for teams without the compute budget for full RLHF: DPO and its variants. Direct Preference Optimization skips the reward model and trains directly on preference pairs. It costs less compute, requires less infrastructure, and produces comparable results for many alignment tasks. The boilerplate is shorter, the gotchas are subtler — preference-pair ordering errors are silent and degrade the model without crashing training. We log preference-pair accuracy at every checkpoint as a sanity check.

Alignment is not a single step we finish. It's a loop between supervised fine-tuning, preference optimization, and evaluation against the prompts users actually send.

The pragmatic order is SFT → preference data collection → DPO or RLHF → domain eval → iterate. Skipping the SFT step and going straight to preference optimization produces unstable training in our experience. The supervised pass gives the model a stable starting policy that preference optimization can refine.

Benchmarking Success: Beyond MMLU Scores

MMLU — Massive Multitask Language Understanding — covers 57 subjects across STEM, humanities, and social sciences. It's the standard reference benchmark and a useful one for cross-model comparison. It is not a useful proxy for whether a fine-tuned model works in our domain. Higher MMLU scores do not reliably correlate with better real-world performance in narrow enterprise applications, and we treat MMLU as a regression check rather than a target.

The right evaluation strategy has three layers:

• General capability regression — a small held-out set of general prompts confirms the model hasn't lost broad competence. A subset of MMLU works here, alongside custom general-knowledge prompts.
• Domain capability — a curated eval set built from real production traffic, labeled by the team that owns the use case. This is where the actual signal lives.
• Behavioral checks — adversarial prompts, edge cases, and the specific failure modes seen in earlier iterations. These don't measure accuracy; they measure safety and robustness.

The boilerplate here is to wire evaluation into the training loop, not run it as a separate step after training completes. Eval during training lets us catch regressions early and pick the right checkpoint without rerunning expensive training jobs. A checkpoint saved at step 1,200 of 2,000 will often outperform the final step on domain metrics — chasing the lowest loss is not the same as chasing the best deployed behavior.

MMLU is also a moving target. Newer benchmarks — domain-specific reasoning suites, coding evals, instruction-following grids — capture capabilities MMLU was never designed to measure. The discipline is to maintain a small benchmark suite that mirrors actual production traffic, not chase the latest leaderboard. A 200-example eval set built from last month's production logs, labeled by the team that owns the workflow, will outperform any public benchmark for predicting whether the next checkpoint will ship.

Putting It Together: The Fine-Tuning Checklist

Five factors, applied in order:

1. Data first — curate, deduplicate, validate, stratify. Spend a week here before touching a config file. The dataset is the moat.

2. PEFT by default — LoRA for most tasks, QLoRA when GPU memory is the constraint. Start with r=16, alpha=32, then adjust.

3. Hyperparameter discipline — learning rate 10x–100x below pre-training, two to four epochs, log enough checkpoints to roll back.

4. Alignment as a loop — supervised fine-tuning, then preference optimization (RLHF or DPO), then re-evaluate against production-shaped prompts.

5. Benchmark against the domain, not the leaderboard — MMLU is a sanity check; the production eval set is the score that matters.

The most expensive mistake in fine-tuning is treating it as a one-shot task. The models that ship successfully are the ones backed by a data loop, an eval loop, and the patience to iterate on both. Fine-tuning rewards discipline — same dataset, same eval, same checkpoint hygiene, run after run. The novelty is in the data, not in the configuration.