llm-finetuning
PackAI & modelsllm-finetuning is a plugin that lets an AI agent run the full fine-tuning lifecycle for open-weight language models. It covers building an evaluation harness first, choosing a method and model, preparing data, training with Unsloth or TRL, gating checkpoints, and exporting GGUF or FP files. The first keyword phrase, eval-gated LLM fine-tuning lifecycle, describes its core flow: checkpoints only pass when evaluations approve them.
Unavailable. Delivery for this kind is on the roadmap — not serving yet.
Have an open-weights language model you want to fine-tune.
What your AI can do with it
- Builds an eval harness before training starts
- Selects fine-tuning method and model (LoRA/QLoRA, DPO, GRPO/RLVR, vision SFT)
- Prepares training data
- Trains with Unsloth or TRL
- Gates checkpoints on evaluation results
- Exports quantized GGUF/FP files
Getting started
- Have an open-weights language model you want to fine-tune.
- Add the llm-finetuning plugin to your agent setup.
- Ask the agent to build the eval harness first.
- Have the agent select the method and model, prepare data, and run training with Unsloth or TRL.
- Review gated checkpoints and export the result as GGUF or FP.
Signals
- GitHub stars
- 40k
- Forks
- 4k
- Last commit
- Sep 2026
Questions
- What training methods does it support?
- LoRA and QLoRA, DPO, GRPO/RLVR, and vision SFT, with quantized export of the trained model.
- Which training libraries does it use?
- Training runs with Unsloth or TRL.
- What export formats are available?
- It exports GGUF and FP files, including quantized exports.
- Why build the eval harness first?
- The lifecycle is eval-gated: checkpoints are gated on evaluation results, so the harness is set up before training begins.
Advanced
- Item type
- plugin
- Key
wshobson-agents-llm-finetuning- Source
- github.com/wshobson/agents