Blog Post··7 min read
GRPO vs PPO: Why Removing the Value Head Changes Everything
GRPO achieves competitive alignment results without a value function. Here's exactly what changes in the math and implementation, and why that matters for training efficiency and stability.