SimPO Loss
SkillAI & modelsSimPO (Simple Preference Optimization) loss computation for LLM alignment without a reference model.
Available today. Use it from your connected AI after setup.
No other account needed.
Connect ahel once, and every AI you use reads what you have installed.
Then ask your AI: use the SimPO Loss skill
What this skill tells your AI
The instructions your AI receives, as published by cxcscmu/skilllearnbench in skills/b1-one-shot-claude-opus-4-6/nlp-paper-reproduction/simpo-loss/SKILL.md and read by ahel’s review.
Overview
SimPO is a reference-free preference optimization algorithm. Its key innovation is using the average log probability of a sequence as the implicit reward, plus a target reward margin γ.
Loss Formula (Eq. 6 from the paper)
L_SimPO = -E log σ(β/|yw| · log πθ(yw|x) - β/|yl| · log πθ(yl|x) - γ)
Since the log probabilities passed to simpo_loss are already length-normalized (average log prob), the loss simplifies to:
logits = β * policy_chosen_logps - β * policy_rejected_logps - γ
where γ = gamma_beta_ratio * beta.
Loss Types
- sigmoid (default):
losses = -log σ(logits) * (1 - label_smoothing) - log σ(-logits) * label_smoothing - hinge:
losses = relu(1 - logits)
Rewards
chosen_rewards = β * policy_chosen_logpsrejected_rewards = β * policy_rejected_logps
Default Hyperparameters
- β = 2.0
- gamma_beta_ratio = 0.25 (so γ = 0.5)
- label_smoothing = 0.0
- loss_type = "sigmoid"
Signals
- GitHub stars
- 83
- Forks
- 5
- Last commit
- Jul 2026
Advanced
- Catalog kind
- skill
- Gateway key
simpo-loss- Source
- github.com/cxcscmu/skilllearnbench