Skip to main content
Miles is a high-performance, enterprise-ready reinforcement learning framework for large-scale model post-training. It pairs SGLang for high-throughput rollout with Megatron-LM for scalable training, and ships the precision, stability and observability features an RL run needs at trillion-parameter scale. A PyTorch FSDP2 backend is available for runs that would rather train the HuggingFace implementation as-is, though the recipes, the parallelism and the largest models all live on Megatron-LM. See Training Backends.
“A journey of a thousand miles begins with a single rollout.”

​
Core features

​
Performance

  • Fully async RL. Rollout and training workers are decoupled, with configurable on- and off-policy schedules, a pipeline tuned for fewer bubbles, and customizable async rollout and eval modes. See Fully Async RL.
  • Fast agentic rollout. Generation runs on SGLang behind a router that spreads requests across engines, preserves per-request metadata and health-checks the fleet. Tuned for multi-turn agentic workloads.
  • Fast weight updates. New weights reach the engines in-loop in seconds, even on a trillion-parameter model such as Kimi-K2.6, with P2P RDMA as the fast path for disaggregated setups.
  • Low-precision training. MXFP8 and NVFP4 training with a numerically stable RL recipe that reduces precision-induced divergence. FP8, INT4 QAT, BF16 and FP16 are also supported.
  • LoRA and multi-LoRA. Low-rank adapters train frontier-scale models on a fraction of the GPUs, and the same adapters load straight into SGLang for rollout.

​
Correctness and resilience

​
What Miles runs

  • Day-0 model support. DeepSeek-V4, Kimi-K3, GLM-5.2, Inkling and Nemotron landed on release day. Beyond day 0, nearly every frontier model runs on Miles, including Kimi-K2.6 and Qwen3.5. See Supported models.
  • Extensive hardware support. NVIDIA from H100 through GB300, and AMD MI300X through MI355X via ROCm. See Supported hardware.
  • Wide recipe support. GRPO, GSPO, PPO and REINFORCE++ for RL, plus SFT and on-policy distillation.
  • Agentic environments. Train coding and computer-use agents through connectors for Harbor, HUD, NeMo Gym, OpenEnv, Verifiers and more, each plugging into the rollout layer that fits it, with task sandboxes on AgentENV, Daytona, E2B or Modal. See Agentic Environments.
  • Diffusion models. Flow-GRPO, DiffusionNFT and SFT on an sglang-diffusion rollout engine and an FSDP2 trainer, in Miles-diffusion.
  • Comprehensive CI. Unit suites run on every pull request, and tag-triggered end-to-end GPU training tests cover the supported model families on both NVIDIA and AMD runners.

​
Supported models

Each model name links to its recipe page or launch script. The table is not exhaustive — it highlights recent releases; many more models run on Miles out of the box, including older generations of the families below. See Models for LLM family guides and Diffusion for diffusion recipes and validation details.

​
Supported hardware

  • NVIDIA: GB300, GB200, B300, B200, H200, H100, A100.
  • AMD: MI300X, MI325, MI350, MI355X (via ROCm).
See Installation for per-GPU status and the container images for each.

​
News

  • [2026/08] 🔥 Miles v0.1 is released! Read the blog post here: Miles v0.1: Production-level Post-training.
  • [2026/07] Towards Blackwell-Native 8-bit and 4-bit RL: End-to-End MXFP8 and NVFP4 RL in Miles (blog).
  • [2026/07] 🔥 SGLang and Miles add day-0 support for Kimi K3 (blog).
  • [2026/07] On-policy distillation lands in Miles (blog).
  • [2026/07] 🔥 SGLang and Miles add day-0 support for Inkling, a frontier multimodal model (blog).
  • [2026/07] DeepSeek-V4 Flash RL training comes to AMD Instinct MI355X with Miles (blog).
  • [2026/06] SGLang and Miles add day-0 support for NVIDIA Nemotron 3 Ultra (blog).
  • [2026/05] No token left behind: token-in-token-out in Miles (blog).
  • [2026/04] Updating 1 T parameters in seconds: P2P weight transfer in large-scale distributed RL (blog).
  • [2026/04] 🔥 DeepSeek-V4 on day 0: from fast inference to verified RL with SGLang and Miles (blog).

​
Start here

  1. Installation — Docker, bare metal, AMD.
  2. Quick Start — a training job up and running in under an hour.
  3. Core concepts — the four objects in every Miles job.
  4. Launch script — what python scripts/run_*.py does and how to override a recipe.
  5. Training backends — Megatron-LM and FSDP: parallelism, checkpoints, and hooks.

​
Acknowledgment

Miles was forked from slime, and integrates SGLang, Megatron-LM and torch_memory_saver. Miles is shaped by the teams that build on it and support its development, from hardware and cloud to model labs, agent infrastructure, and academia:

​
Contribute