llm-fine-tuning
llm-fine-tuning is an engineering AI skill with a core value of Set up infrastructure for fine-tuning LLMs with QLoRA, LoRA, and full fine-tuning using Hugging Face TRL, Axolotl, and distributed training with DeepSpeed or FSDP. It
helps developers solve real-world problems in the engineering domain, boosting
efficiency, automating repetitive tasks, and optimizing workflows.
Set up infrastructure for fine-tuning LLMs with QLoRA, LoRA, and full fine-tuning using Hugging Face TRL, Axolotl, and distributed training with DeepSpeed or FSDP.
Quick Facts
mkdir -p ./skills/llm-fine-tuning && curl -sfL https://raw.githubusercontent.com/sickn33/antigravity-awesome-skills/main/skills/llm-fine-tuning/SKILL.md -o ./skills/llm-fine-tuning/SKILL.mdRun in terminal / PowerShell. Requires curl (Unix) or PowerShell 5+ (Windows).
Skill Content
# LLM Fine-Tuning Infrastructure
Train and fine-tune open-source LLMs efficiently — from LoRA on a single GPU to distributed full fine-tuning across multi-node clusters.
When to Use This Skill
Use this skill when:
- Fine-tuning an LLM on domain-specific data (legal, medical, code, support)
- Running QLoRA to fine-tune 70B models on consumer GPUs
- Setting up distributed training with DeepSpeed or FSDP
- Exporting fine-tuned adapters for production serving
- Implementing RLHF, DPO, or instruction tuning pipelines
Prerequisites
- NVIDIA GPU(s) with 24GB+ VRAM (RTX 4090 / A100 / H100)
- CUDA 12.1+ and `nvidia-smi` working
- Python 3.10+ with `pip`
- Hugging Face account and `HF_TOKEN` for gated models
- 500GB+ disk for model weights and training data
Quick Start: QLoRA Fine-Tuning
pip install transformers datasets trl peft bitsandbytes accelerate
python - <<'EOF'
from datasets import load_dataset
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import LoraConfig, get_peft_model
from trl import SFTTrainer, SFTConfig
import torch
model_id = "meta-llama/Llama-3.1-8B-Instruct"
# 4-bit quantization (QLoRA)
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16,
bnb_4bit_use_double_quant=True,
)
model = AutoModelForCausalLM.from_pretrained(
model_id, quantization_config=bnb_config, device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_id)
# LoRA configuration
peft_config = LoraConfig(
r=16, # rank
lora_alpha=32,
target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
"gate_proj", "up_proj", "down_proj"],
lora_dropout=0.05,
bias="none",
task_type="CAUSAL_LM",
)
dataset = load_dataset("your-org/your-dataset", split="train")
trainer = SFTTrainer(
model=model,
args=SFTConfig(
output_dir="./output",
num_train_epochs=3,
per_device_train_batch_size=2,
gradient_accumulation_steps=8,
learning_rate=2e-4,
bf16=True,
logging_steps=10,
save_strategy="epoch",
report_to="wandb",
),
train_dataset=dataset,
peft_config=peft_config,
processing_class=tokenizer,
)
trainer.train()
trainer.save_model("./fine-tuned-model")
EOFAxolotl (Production Fine-Tuning Framework)
# config.yaml — Axolotl QLoRA config for Llama 3.1
base_model: meta-llama/Llama-3.1-8B-Instruct
model_type: LlamaForCausalLM
tokenizer_type: PreTrainedTokenizerFast
load_in_4bit: true
adapter: qlora
lora_r: 32
lora_alpha: 64
lora_dropout: 0.05
lora_target_modules:
- q_proj
- k_proj
- v_proj
- o_proj
- gate_proj
- up_proj
- down_proj
datasets:
- path: your-org/your-dataset
type: alpaca # or sharegpt, chat_template, etc.
dataset_prepared_path: ./prepared-data
val_set_size: 0.05
output_dir: ./output
sequence_len: 4096
sample_packing: true # pack multiple short samples for efficiency
micro_batch_size: 2
gradient_accumulation_steps: 8
num_epochs: 3
learning_rate: 2e-4
optimizer: adamw_bnb_8bit
lr_scheduler: cosine
warmup_ratio: 0.05
bf16: true
flash_attention: true
logging_steps: 10
eval_steps: 100
save_steps: 200
wandb_project: my-fine-tune# Run with Axolotl
pip install axolotl[flash-attn,deepspeed]
accelerate launch -m axolotl.cli.train config.yamlDistributed Training with DeepSpeed
// deepspeed_zero3.json — ZeRO Stage 3 (split optimizer + gradients + params)
{
"zero_optimization": {
"stage": 3,
"offload_optimizer": {"device": "cpu", "pin_memory": true},
"offload_param": {"device": "cpu", "pin_memory": true},
"overlap_comm": true,
"contiguous_gradients": true,
"sub_group_size": 1e9,
"reduce_bucket_size": "auto",
"stage3_prefetch_bucket_size": "auto",
"stage3_param_persistence_threshold": "auto",
"stage3_max_live_parameters": 1e9,
"stag🎯 Best For
- Claude users
- AI users
💡 Use Cases
- Using llm-fine-tuning in daily workflow
- Automating repetitive engineering tasks
📖 How to Use This Skill
- 1
Install the Skill
Copy the install command from the Terminal tab and run it. The SKILL.md file downloads to your local skills directory.
- 2
Load into Your AI Assistant
Open Claude and reference the skill. Paste the SKILL.md content or use the system prompt tab.
- 3
Apply llm-fine-tuning to Your Work
Provide context for your task — paste source material, describe your audience, or share existing work to guide the AI.
- 4
Review and Refine
Edit the AI output for accuracy, tone, and completeness. Add human insight where the AI lacks context.
❓ Frequently Asked Questions
How do I install llm-fine-tuning?
Copy the install command from the Terminal tab and run it. The skill downloads to ./skills/llm-fine-tuning/SKILL.md, ready to use.
Can I customize this skill for my team?
Absolutely. Edit the SKILL.md file to add team-specific instructions, examples, or workflows.
⚠️ Common Mistakes to Avoid
Not reading the full skill
Skills contain important context and edge cases beyond the quick start.