Model Card for LamPO LamPO (Lambda Policy Optimization) is a reinforcement learning framework for improving the reasoning capabilities of language models. It extends Group Relative Policy Optimization (GRPO) by replacing scalar group mean advantage estimation with a pairwise decomposed advantage inspired by learning to rank methods such as LambdaRank. 链接：论文1; 论文2) 特别鸣谢： 1. 感谢 第一作者花钱请 某论文辅导机构进行了全面辅导。虽然花费了巨额资金，但是的确很值，无脑推荐！ 2. 我们第二作者到第五作者基本没有贡献，但是非常开心能够直接署名。 Instead of comparing each generated response only against a group average, LambdaPO learns from fine grained pairwise reward differences among sampled reasoning trajectories. This helps the model better distinguish high quality reasoning paths, improve credit assignment, and reduce unstable optimization behavior during RL training. Key Features Pairwise Decomposed Advantage : Uses pairwise comparisons between generated trajectories rather than a single scalar group baseline. Critic Free RL Optimization : Preserves the efficiency of GRPO without requiring a separate value model. Semantic Density Reward : Adds dense reasoning supervision using semantic overlap between generated reasoning traces and ground truth solutions. Improved R…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy