# Miles

> Miles is an open-source reinforcement learning framework for large-scale LLM post-training, pairing SGLang rollout with Megatron-LM training at trillion-parameter scale.

- [Welcome](https://miles.radixark.com/docs/index.md): Miles is an open-source RL framework for large-scale LLM post-training, pairing SGLang rollout with Megatron-LM training at trillion-parameter scale.
- [Getting Started](https://miles.radixark.com/docs/getting-started/index.md): Install Miles and run your first RL training job — the two pages you need to go from zero to a working loop.
- [Installation](https://miles.radixark.com/docs/getting-started/installation.md): Install Miles on NVIDIA or AMD GPUs. Docker is the recommended path.
- [Quick Start](https://miles.radixark.com/docs/getting-started/quick-start.md): Get an RL training job up and running in under an hour.
- [User Guide](https://miles.radixark.com/docs/user-guide/index.md): Concepts, launch scripts, customization hooks, and a complete CLI reference.
- [Core Concepts](https://miles.radixark.com/docs/user-guide/concepts.md): The four objects that make up every Miles job, and how data flows between them.
- [Launch Script](https://miles.radixark.com/docs/user-guide/launch-script.md): What a Miles launch script does when you run it, how it is structured, and the three ways to override a recipe.
- [Argument Groups](https://miles.radixark.com/docs/user-guide/argument-groups.md): The launch-script argument groups used by Miles recipes, with links to the flags that belong in each group.
- [Fully Async RL](https://miles.radixark.com/docs/user-guide/fully-async.md): How fully async rollout decouples generation from training, which flags control it, and how to evaluate without stalling the trainer.
- [Training Backends](https://miles.radixark.com/docs/user-guide/training-backend.md): The contract Megatron-LM and FSDP both implement, which one to pick, and how to configure parallelism, GPU layout, offload, and checkpoints for each.
- [Monitoring & Logging](https://miles.radixark.com/docs/user-guide/monitoring.md): wandb, structured logs, profiling, and what to look at when something looks off.
- [Customization](https://miles.radixark.com/docs/user-guide/customization.md): The plug-points where you can drop in your own Python without forking Miles.
- [Generate Endpoint](https://miles.radixark.com/docs/user-guide/generate-endpoint.md): Write a custom generate function against SGLang's raw /generate endpoint, owning prompt construction, tokens, and loss masks.
- [Agentic Rollout (TITO)](https://miles.radixark.com/docs/user-guide/agentic-rollout.md): Configure an OpenAI-compatible agent loop with Token-In-Token-Out trajectory assembly.
- [Agentic Environments](https://miles.radixark.com/docs/user-guide/environments.md): How Miles trains on RL environments — datasets with rewards, self-wired environments, and optional external ecosystems.
- [Harbor](https://miles.radixark.com/docs/user-guide/harbor.md): Train agents on mixed task suites (SWE-bench, Terminal-Bench, custom) through the Harbor framework.
- [OpenEnv](https://miles.radixark.com/docs/user-guide/openenv.md): Train on Hugging Face OpenEnv environments through the agent-function extension point.
- [NeMo Gym](https://miles.radixark.com/docs/user-guide/nemo-gym.md): Train on NVIDIA NeMo Gym environments through the agent-function extension point.
- [Verifiers (Prime Intellect)](https://miles.radixark.com/docs/user-guide/verifiers.md): Train on Prime Intellect Verifiers environments with Miles.
- [CLI Reference](https://miles.radixark.com/docs/user-guide/cli-reference.md): Every command-line flag Miles accepts, grouped by subsystem.
- [Supported Models](https://miles.radixark.com/docs/models/index.md): Per-family recipes covering weight conversion, launch flags, and parallelism choices.
- [DeepSeek](https://miles.radixark.com/docs/models/deepseek/index.md): Miles recipes for the DeepSeek family — V4 Flash, V4 Pro, and V3.2.
- [DeepSeek-V4 Flash](https://miles.radixark.com/docs/models/deepseek/deepseek-v4-flash.md): Launch recipe for DeepSeek-V4-Flash (284 B) — FP8 rollout / BF16 train, 8-node H200 (64 GPUs).
- [DeepSeek-V4 Pro](https://miles.radixark.com/docs/models/deepseek/deepseek-v4-pro.md): Launch recipe for DeepSeek-V4-Pro (1.6 T) — V4-family architecture at Pro scale.
- [DeepSeek-V3.2](https://miles.radixark.com/docs/models/deepseek/deepseek-v3-2.md): Launch recipe for DeepSeek-V3.2 (671 B total / 37 B active) — BF16 training, NSA rollout, 8 training nodes and up.
- [DeepSeek V3](https://miles.radixark.com/docs/models/deepseek/deepseek.md): Launch recipe for DeepSeek-V3 (671 B total / 37 B active) via scripts/run_deepseek.py.
- [Thinking Machines](https://miles.radixark.com/docs/models/thinkingmachines/index.md): Miles recipes for Thinking Machines Lab models — Inkling (975 B), a multimodal MoE with short convolution, relative attention, and a shared-expert sink.
- [Inkling](https://miles.radixark.com/docs/models/thinkingmachines/inkling.md): Launch recipe for Inkling (975 B), Thinking Machines' multimodal MoE with short convolution, relative attention, and a shared-expert sink.
- [Inkling-Small](https://miles.radixark.com/docs/models/thinkingmachines/inkling-small.md): Launch recipe for Inkling-Small (276 B), the compact sibling of Inkling — same architecture, 4-node H200 footprint.
- [Qwen](https://miles.radixark.com/docs/models/qwen/index.md): Miles recipes for the full Qwen3, Qwen3.5, and Next line — dense and MoE, through the Qwen4-architecture Qwen3.8-Flash-Next.
- [Qwen3.8](https://miles.radixark.com/docs/models/qwen/qwen3-8.md): Launch recipe for the dense Qwen3.8-27B, plus where the 2.4T-A95B MoE recipe lives.
- [Qwen3.6](https://miles.radixark.com/docs/models/qwen/qwen3-6.md): Launch recipe for the dense Qwen3.6-27B with attention-output-gate.
- [Qwen3.6 MoE](https://miles.radixark.com/docs/models/qwen/qwen3-6-moe.md): Launch recipe for Qwen3.6-35B-A3B with MTP training and EAGLE speculative rollout.
- [Qwen3.5](https://miles.radixark.com/docs/models/qwen/qwen3-5.md): Launch recipes for Qwen3.5-4B / 9B / 27B with attention-output-gate.
- [Qwen3.5 MoE](https://miles.radixark.com/docs/models/qwen/qwen3-5-moe.md): Launch recipe for Qwen3.5-35B-A3B with MTP training and EAGLE speculative rollout.
- [Qwen3.8-Flash-Next](https://miles.radixark.com/docs/models/qwen/qwen3-8-flash-next.md): RL recipe for Qwen3.8-Flash-Next, the GDN + QSA hybrid MoE preview of the Qwen4 architecture, with hyper-connections and a host-resident PLE table.
- [Qwen3-Next 80B-A3B](https://miles.radixark.com/docs/models/qwen/qwen3-next.md): Launch recipes for Qwen3-Next-80B-A3B-Thinking on the Megatron backend.
- [Qwen3](https://miles.radixark.com/docs/models/qwen/qwen3.md): Launch recipes for dense Qwen3 models (0.6 B – 32 B).
- [Qwen3 MoE](https://miles.radixark.com/docs/models/qwen/qwen3-moe.md): Launch recipes for Qwen3-30B-A3B (single node) and Qwen3-235B-A22B (multi-node).
- [GLM](https://miles.radixark.com/docs/models/glm/index.md): Miles recipes for the GLM4.5, GLM4.7 Flash, GLM5, and GLM5.2 families, plus the KDA + DSA hybrid GLM-5.3-Flash.
- [GLM-5.3-Flash](https://miles.radixark.com/docs/models/glm/glm5-3-flash.md): RL recipe for GLM-5.3-Flash, a KDA + DSA hybrid MoE with mHC hyper-connections and NoPE MLA.
- [GLM-5.2](https://miles.radixark.com/docs/models/glm/glm5-2.md): Launch recipe for GLM-5.2 (744 B / 40 B active) — FP8 KV cache, TIS, 16+ node config.
- [GLM-5 / GLM-5.1](https://miles.radixark.com/docs/models/glm/glm5.md): Launch recipe for GLM-5 and GLM-5.1 (744 B / 40 B active) — Python launcher, 16+ node config.
- [GLM4.7 Flash](https://miles.radixark.com/docs/models/glm/glm4-7-flash.md): Launch recipes for GLM-4.7-Flash — compact MLA + MoE with R3 enabled by default.
- [GLM4.5](https://miles.radixark.com/docs/models/glm/glm4-5.md): Launch recipes for GLM-4.5 (355B-A32B) — the 8-node launcher and the Blackwell launcher.
- [Kimi](https://miles.radixark.com/docs/models/kimi/index.md): Miles recipes for the Moonshot family — Kimi K2.6 / K2.5 (multimodal, 1 T / 32 B-A) and Kimi K2 / K2-Thinking.
- [Kimi-K3](https://miles.radixark.com/docs/models/kimi/kimi-k3.md): LoRA RL recipe for Kimi-K3, a KDA + MLA hybrid with an 896-expert latent MoE, trained colocated with SGLang.
- [Kimi K2.5 / K2.6](https://miles.radixark.com/docs/models/kimi/kimi-k2.5.md): Launch recipe for Kimi-K2.5, running full-parameter GRPO on 32 × 8 H200 with an INT4 actor and a BF16 reference.
- [Kimi K2](https://miles.radixark.com/docs/models/kimi/kimi-k2.md): Launch recipes for Kimi-K2-Instruct and Kimi-K2-Thinking — 32 nodes × 8 GPU.
- [Nemotron](https://miles.radixark.com/docs/models/nemotron/index.md): Miles recipes for NVIDIA's Nemotron-3 family — Mamba+Attention(+MoE) hybrids loaded via Megatron AutoBridge.
- [Nemotron-3-Nano](https://miles.radixark.com/docs/models/nemotron/nemotron-3-nano.md): Launch recipe for the dense NVIDIA Nemotron-3-Nano-4B (Mamba+Attention hybrid) via Megatron AutoBridge.
- [Nemotron-3-Nano MoE](https://miles.radixark.com/docs/models/nemotron/nemotron-3-nano-moe.md): Launch recipe for NVIDIA Nemotron-3-Nano-30B-A3B (Mamba+Attention+MoE hybrid) via Megatron AutoBridge.
- [Nemotron-3-Super](https://miles.radixark.com/docs/models/nemotron/nemotron-3-super.md): Launch recipe for NVIDIA Nemotron-3-Super-120B-A12B-FP8 (Mamba+Attention+MoE hybrid, FP8 native) via Megatron AutoBridge.
- [Nemotron-3-Ultra](https://miles.radixark.com/docs/models/nemotron/nemotron-3-ultra.md): Launch recipe for NVIDIA Nemotron-3-Ultra-550B-A55B (hybrid Mamba2 + Attention + latent-MoE) via Megatron AutoBridge.
- [Gemma](https://miles.radixark.com/docs/models/gemma/index.md): Miles recipes for Google's Gemma-4 line, trained on the base VLM checkpoint through the HF to Megatron bridge.
- [Gemma-4](https://miles.radixark.com/docs/models/gemma/gemma-4.md): Launch recipes for Gemma-4 26B-A4B-it (MoE) and 31B-it (dense) via the HF to Megatron bridge.
- [GPT-OSS 20B](https://miles.radixark.com/docs/models/gpt-oss/gpt-oss.md): Launch recipe for OpenAI's GPT-OSS 20B — Megatron BF16 on a single 8-GPU node, loading the MXFP4 HF checkpoint through mbridge.
- [Advanced Features](https://miles.radixark.com/docs/advanced/index.md): Systems-level features for large-scale and long-running RL.
- [Low Precision RL](https://miles.radixark.com/docs/advanced/low-precision.md): Unified low-precision pipelines for RL — block-wise FP8, MXFP8, and NVFP4 across rollout and training.
- [INT4 Quantization-Aware Training](https://miles.radixark.com/docs/advanced/int4-qat.md): Train MoE policies with fake-quantized expert weights in Megatron and packed W4A16 rollout weights in SGLang.
- [Speculative Decoding](https://miles.radixark.com/docs/advanced/speculative-decoding.md): Draft + target speculative rollout, with online SFT for MTP-style drafts.
- [LoRA Training and Serving](https://miles.radixark.com/docs/advanced/lora.md): Train LoRA adapters with miles, synchronize them to SGLang, and run dense, MoE, quantized-rollout, multi-LoRA, and agentic recipes.
- [On-Policy Distillation](https://miles.radixark.com/docs/advanced/on-policy-distillation.md): Train a student on its own rollouts with a teacher's token-level probabilities as a reverse-KL signal, composable with GRPO, PPO, and other estimators.
- [Fault Tolerance](https://miles.radixark.com/docs/advanced/fault-tolerance.md): Rollout-side health checks and engine recovery, gated by --use-fault-tolerance.
- [PD Disaggregation](https://miles.radixark.com/docs/advanced/pd-disaggregation.md): Separate prefill and decode pools so each is sized for its workload.
- [P2P Weight Transfer](https://miles.radixark.com/docs/advanced/p2p-weight-transfer.md): Direct rank-to-rank weight sync from actor to rollout via RDMA writes.
- [Mooncake Rollout Data Transfer](https://miles.radixark.com/docs/advanced/mooncake-rollout-transfer.md): Configure Mooncake Store as the rollout object-store backend in Miles.
- [Disaggregated RL Rollout](https://miles.radixark.com/docs/advanced/disaggregated-rollout.md): Scale rollout independently across clusters and regions while miles preserves policy publication and version attribution.
- [Disk Offload](https://miles.radixark.com/docs/advanced/disk-offload.md): Spill the paused training actor to node-local disk when host RAM cannot hold it.
- [Rollout Routing Replay (R3)](https://miles.radixark.com/docs/advanced/miles-router.md): Capture expert routing during inference and replay it during training to stabilize RL.
- [Examples](https://miles.radixark.com/docs/examples/index.md): These examples are runnable starting points for your own RL workflow.
- [VLM Single-Turn RL (FSDP & Megatron)](https://miles.radixark.com/docs/examples/geo3k-vlm.md): Training VLMs with FSDP using GRPO on the GEO3K dataset.
- [VLM Multi-Turn (geo3k dataset)](https://miles.radixark.com/docs/examples/geo3k-vlm/multi-turn.md): The same dataset over multiple turns, with the model cropping images through an interactive environment.
- [Multi-LoRA Training Example (fully-async)](https://miles.radixark.com/docs/examples/multi-lora.md): Fully-async multi-adapter LoRA training with a slot-keyed adapter page table.
- [On-Policy Distillation Examples](https://miles.radixark.com/docs/examples/on-policy-distillation.md): Teacher–student distillation on the student's own rollouts, run inside the on-policy training loop.
- [Qwen3.5-35B-A3B Self-Distillation on a Single Node (RLVR teacher → OPD)](https://miles.radixark.com/docs/examples/on-policy-distillation/qwen3-5-35b-selfdistill.md): Two-phase self-distillation of Qwen3.5-35B-A3B on one 8xH200 node, with an in-process Megatron teacher.
- [PPO Example](https://miles.radixark.com/docs/examples/ppo.md): Actor-critic PPO with GAE advantages, where the critic shares the actor's train GPUs.
- [Retool v2](https://miles.radixark.com/docs/examples/retool-v2.md): Tool-enabled language model generation with sandboxed Python code execution interleaved with thinking.
- [SWE-Agent training with Harbor on Docker sandboxes](https://miles.radixark.com/docs/examples/swe-agent-harbor-docker.md): Trains coding and terminal agents with Harbor-managed local Docker sandboxes and verifier rewards.
- [Infra Features](https://miles.radixark.com/docs/examples/infra-features.md): These examples exercise runtime and infrastructure behaviour rather than a training recipe.
- [Fully Asynchronous Rollout Example](https://miles.radixark.com/docs/examples/infra-features/fully-async.md): Demonstrates fully asynchronous rollout generation for higher efficiency.
- [FP8 training examples](https://miles.radixark.com/docs/examples/infra-features/low-precision.md): Examples of FP8 training and inference, plus INT4 QAT, for improved throughput and stability.
- [P2P Weight Transfer Examples](https://miles.radixark.com/docs/examples/infra-features/p2p-weight-transfer.md): Point-to-point weight transfer between training and rollout engines.
- [Random fully-async example](https://miles.radixark.com/docs/examples/infra-features/random-async.md): Dataset-free stress test of the async rollout ↔ trainer loop.
- [Rollout Correction Methods](https://miles.radixark.com/docs/examples/infra-features/train-infer-mismatch-helper.md): Algorithmic methods for rollout correction (e.g., TIS, MIS).
- [True On-Policy between Training and Inference](https://miles.radixark.com/docs/examples/infra-features/true-on-policy.md): Ensures strictly equal log probabilities between inference (SGLang) and training engines.
- [Developer Guide](https://miles.radixark.com/docs/developer/index.md): Contribution conventions, internal architecture, dependency versions, and debugging.
- [Contributing](https://miles.radixark.com/docs/developer/contributor-guide.md): Repository layout, the local loop, what enforces code style, what lives in .claude, and how to drive CI from a PR.
- [Architecture Overview](https://miles.radixark.com/docs/developer/architecture.md): The 30-minute tour of how Miles is organized internally.
- [Versions and Images](https://miles.radixark.com/docs/developer/versions.md): How the Miles, SGLang, and Megatron-LM trees fit together, which artifact owns each release identity, and what the Docker images pin.
- [Debugging](https://miles.radixark.com/docs/developer/debug.md): Split a misbehaving run into rollout and training halves, then debug the side that is actually wrong.
- [CI Contributor Guide](https://miles.radixark.com/docs/ci/contributor-guide.md): For community contributors — add a CI test and confirm it runs, tell an infra failure from your own, and report a machine issue or a flaky test.
- [Stage](https://miles.radixark.com/docs/ci/00-stage.md): How CI stages are defined, how a test's suite maps to a stage, and what each stage does.
- [Labels](https://miles.radixark.com/docs/ci/01-label.md): The three kinds of CI label — domain labels that gate tests, scope labels that broaden selection, and bypass-fastfail.
- [Docker build](https://miles.radixark.com/docs/ci/02-docker-build.md): The Dockerfiles, the build script, rolling and release build workflows, and how to build and push manually.
- [Metric history & regression gate](https://miles.radixark.com/docs/ci/03-metric-history-gate.md): How CI keeps per-test training metrics across runs, gates new numbers against that history, and how to add gate specs or repair hosted history.
- [Release a Version](https://miles.radixark.com/docs/ci/04-release.md): Cut a versioned Miles release branch, run release CI, tag an exact release, and publish the official Docker images.
- [Command Identity](https://miles.radixark.com/docs/ci/05-command-identity.md): Who may run each PR-comment CI command — the identity bindings, access groups, tier constraints, and token identities behind the comment gateway.
- [Miles-Diffusion Documentation](https://miles.radixark.com/docs/diffusion/index.md)
- [Installation](https://miles.radixark.com/docs/diffusion/getting-started/installation.md): Get a working miles-diffusion environment — Docker (recommended) or from source.
- [Quick Start](https://miles.radixark.com/docs/diffusion/getting-started/quick-start.md): A working Flow-GRPO training job on SD3.5 + OCR — default script uses 2 GPUs, using the Flow-GRPO OCR dataset (`flowgrpo_ocr`).
- [CLI Reference](https://miles.radixark.com/docs/diffusion/user-guide/cli-reference.md): Every flag train_diffusion.py accepts, grouped by subsystem.
- [Core Concepts](https://miles.radixark.com/docs/diffusion/user-guide/concepts.md): The four objects that make up every miles-diffusion job and the trajectory-level training loop.
- [Customization](https://miles.radixark.com/docs/diffusion/user-guide/customization.md): Plug-points where you can drop in your own Python without forking miles-diffusion.
- [Launch Scripts](https://miles.radixark.com/docs/diffusion/user-guide/launch-script.md): What a miles-diffusion launch script does when you run it, how it is structured, and the ways to override a recipe.
- [Recipe Verification Levels](https://miles.radixark.com/docs/diffusion/user-guide/recipe-verification.md): Evidence levels for documented training recipes.
- [Rewards](https://miles.radixark.com/docs/diffusion/user-guide/rewards.md): Built-in reward models (PickScore, OCR), rm_hub dispatch, and prompt data format.
- [Cosmos3](https://miles.radixark.com/docs/diffusion/models/cosmos/cosmos3.md): Cosmos3-Nano Flow-GRPO and PickScore recipe with token-level conditioning and packed single-sample forwards.
- [MiniMax H3](https://miles.radixark.com/docs/diffusion/models/h3/h3.md): Text-to-audio-video Flow-GRPO + PickScore on MiniMax H3 — 2-GPU recipe (not merged yet).
- [LTX-2](https://miles.radixark.com/docs/diffusion/models/ltx/ltx2.md): Video GRPO on LTX-2.3 — native model package, CPS-SDE, unguided forward.
- [Qwen-Image](https://miles.radixark.com/docs/diffusion/models/qwen-image/qwen-image.md): Flow-GRPO with PickScore on Qwen-Image — the flow_grpo-aligned 5-GPU recipe.
- [Stable Diffusion 3 / 3.5](https://miles.radixark.com/docs/diffusion/models/sd3/sd3.md): SD3 family config, LoRA targets, and launch recipes for Flow-GRPO and DiffusionNFT.
- [Wan2.2-T2V-A14B](https://miles.radixark.com/docs/diffusion/models/wan/wan2-2.md): Dual-expert MoE video model — Flow-GRPO + PickScore recipe, LoRA SFT recipe, and the high/low-noise expert boundary.
- [Deterministic Training](https://miles.radixark.com/docs/diffusion/advanced/deterministic.md): What --deterministic-mode covers, which attention backends it accepts, and what it deliberately does not fix.
- [Dtype Control](https://miles.radixark.com/docs/diffusion/advanced/dtype-control.md): The global dtype flags, the model-boundary input cast policy, and per-parameter dtype overrides under FSDP2.
- [LoRA Training and Weight Sync](https://miles.radixark.com/docs/diffusion/advanced/lora.md): PEFT LoRA on FSDP diffusion actors and CUDA-IPC weight sync to sglang-diffusion rollout engines.
- [SDE Step Backend](https://miles.radixark.com/docs/diffusion/advanced/sde-backend.md): Train-side SDE dynamics for Flow-GRPO — when it applies, which flags to set, and how to check train/rollout alignment.
- [Single-Prompt Multi-Generation](https://miles.radixark.com/docs/diffusion/advanced/single-prompt-multi-gen.md): One rollout request, N outputs — engine-side conditioning expansion, microgroup mechanics, and seed layout.
- [Streaming Reward and Deserialization](https://miles.radixark.com/docs/diffusion/advanced/streaming-reward.md): How rollout responses stream through msgpack transport, a Ray parser-actor pool, and per-microgroup async reward scoring.
- [Contributing](https://miles.radixark.com/docs/diffusion/developer/contributor-guide.md): Repository layout, the test suites, CI labels, and PR conventions for miles-diffusion.
- [Blog](https://miles.radixark.com/docs/blog/index.md): Miles engineering write-ups on RL systems, post-training recipes, and performance results, published on the LMSYS blog.
- [Subscribe](https://miles.radixark.com/docs/subscribe.md): Stay up to date with Miles releases and announcements.