--- title: "GRPO & RLVR Training" type: skill id: "wshobson-agents/llm-engineering/grpo-rlvr-training" category: "AI & machine learning" publisher: "Seth Hobson" license: "MIT" source: "https://github.com/wshobson/agents/tree/7e2fb55" canonical: "https://labs.pho.chat/en/skills/wshobson-agents/llm-engineering/grpo-rlvr-training" open_in_pho: "https://pho.chat/customize?item=wshobson-agents%2Fllm-engineering%2Fgrpo-rlvr-training&utm_source=labs&utm_medium=skill-page&utm_campaign=skills-library" --- # GRPO & RLVR Training Train reasoning and verifiable-task behavior with GRPO and reinforcement learning from verifiable rewards (RLVR). Use when task success is algorithmically checkable (math, code, tool calls, structured output), when designing GRPO reward functions, or when a GRPO run diverges or reward-hacks. Source: Seth Hobson (MIT), copied at a pinned commit. Free to use. Part of the Phở Skills library (https://labs.pho.chat/en/skills). Skill set: LLM apps and fine tuning (https://labs.pho.chat/en/skills/wshobson-agents/llm-engineering.md). ## What it covers - When RL Applies - The Recipe - The Inspection Rule - Variant Selection - VLM RL Is Reference-Only - References ## Other skills in this set - [Embedding Strategies](https://labs.pho.chat/en/skills/wshobson-agents/llm-engineering/embedding-strategies.md): Select and optimize embedding models for semantic search and RAG applications. Use when choosing embedding models, implementing chunking strategies, or optimizing embedding quality for specific… - [Hybrid Search Implementation](https://labs.pho.chat/en/skills/wshobson-agents/llm-engineering/hybrid-search-implementation.md): Combine vector and keyword search for improved retrieval. Use when implementing RAG systems, building search engines, or when neither approach alone provides sufficient recall. - [LangChain & LangGraph Architecture](https://labs.pho.chat/en/skills/wshobson-agents/llm-engineering/langchain-architecture.md): Design LLM applications using LangChain 1.x and LangGraph for agents, memory, and tool integration. Use when building LangChain applications, implementing AI agents, or creating complex LLM workflows. - [LLM Evaluation](https://labs.pho.chat/en/skills/wshobson-agents/llm-engineering/llm-evaluation.md): Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or… - [RAG Implementation](https://labs.pho.chat/en/skills/wshobson-agents/llm-engineering/rag-implementation.md): Build Retrieval-Augmented Generation (RAG) systems for LLM applications with vector databases and semantic search. Use when implementing knowledge-grounded AI, building document Q&A systems, or… - [Similarity Search Patterns](https://labs.pho.chat/en/skills/wshobson-agents/llm-engineering/similarity-search-patterns.md): Implement efficient similarity search with vector databases. Use when building semantic search, implementing nearest neighbor queries, or optimizing retrieval performance. - [Vector Index Tuning](https://labs.pho.chat/en/skills/wshobson-agents/llm-engineering/vector-index-tuning.md): Optimize vector index performance for latency, recall, and memory. Use when tuning HNSW parameters, selecting quantization strategies, or scaling vector search infrastructure. - [Checkpoint Promotion](https://labs.pho.chat/en/skills/wshobson-agents/llm-engineering/checkpoint-promotion.md): Gate fine-tuned checkpoints with drift budgets, paired comparison, and forgetting checks before promotion. Use after a training run produces a checkpoint, when deciding whether a tuned model ships… - [Dataset Curation](https://labs.pho.chat/en/skills/wshobson-agents/llm-engineering/dataset-curation.md): Prepare, format, and validate datasets for supervised fine-tuning and preference training. Use when converting raw data into training format, applying chat templates, configuring sequence packing… - [Eval Harness First](https://labs.pho.chat/en/skills/wshobson-agents/llm-engineering/eval-harness-first.md): Build the evaluation harness that gates every fine-tuning run — golden sets, per-failure-mode graders, judge calibration, and base-model baselines. Use when starting a fine-tuning effort, when… - [Fine-Tuning Method Selection](https://labs.pho.chat/en/skills/wshobson-agents/llm-engineering/finetuning-method-selection.md): Decide whether to fine-tune at all, and route to the right method (SFT, DPO/ORPO/KTO, GRPO/RLVR, continued pretraining) and base model. Use when starting any fine-tuning effort, when unsure whether… - [LoRA & QLoRA Recipes](https://labs.pho.chat/en/skills/wshobson-agents/llm-engineering/lora-qlora-recipes.md): Configure LoRA and QLoRA supervised fine-tuning with current best-practice hyperparameters. Use when writing or reviewing a LoRA/QLoRA training configuration, choosing rank/alpha/target modules, or… - [Preference Optimization](https://labs.pho.chat/en/skills/wshobson-agents/llm-engineering/preference-optimization.md): Align a fine-tuned model with preference data using DPO, ORPO, KTO, or SimPO. Use when preference pairs or thumbs-up/down feedback exist, when choosing between preference-optimization methods, or… - [Quantized Export](https://labs.pho.chat/en/skills/wshobson-agents/llm-engineering/quantized-export.md): Export a promoted fine-tuned model in the right deployment format — merged safetensors, LoRA-only, GGUF with imatrix, or FP8. Use after a checkpoint passes promotion, when choosing a quantization… - [Trace To Training Data](https://labs.pho.chat/en/skills/wshobson-agents/llm-engineering/trace-to-training-data.md): Convert evaluation traces and production logs into SFT examples and preference pairs. Use when graded traces or failure examples exist and need to become training data, when applying rejection… - [Vision-Language SFT](https://labs.pho.chat/en/skills/wshobson-agents/llm-engineering/vision-sft.md): Fine-tune vision-language models (VLMs) with supervised learning on image+text data. Use when adapting a VLM to a visual domain or task, configuring frozen-vision-tower LoRA, or debugging a VLM… - [ML Pipeline Workflow](https://labs.pho.chat/en/skills/wshobson-agents/llm-engineering/ml-pipeline-workflow.md): Build end-to-end MLOps pipelines from data preparation through model training, validation, and production deployment. Use when creating ML pipelines, implementing MLOps practices, or automating… - [Recsys Pipeline Architect](https://labs.pho.chat/en/skills/wshobson-agents/llm-engineering/recsys-pipeline-architect.md): Design composable recommendation, ranking, and feed pipelines using the six-stage Source→Hydrator→Filter→Scorer→Selector→SideEffect framework popularized by xAI's open-sourced X For You algorithm… ## Use it - In Phở: open https://pho.chat/customize?item=wshobson-agents%2Fllm-engineering%2Fgrpo-rlvr-training&utm_source=labs&utm_medium=skill-page&utm_campaign=skills-library and press Try, or add it so Phở loads it whenever a request matches. Phở works with your own API key (OpenAI, Anthropic, Google and others). - Full instructions (original SKILL.md, unchanged): https://labs.pho.chat/skills/raw/wshobson-agents/llm-engineering/skills/grpo-rlvr-training/SKILL.md - Original files: https://github.com/wshobson/agents/tree/7e2fb55 (MIT). - Full machine-readable index: https://labs.pho.chat/skills.json