AReaL: Large-Scale Asynchronous Reinforcement Learning for Reasoning and Agentic LLMs
Meet AReaL, the open-source asynchronous RL system from Tsinghua IIIS and Ant Group for training reasoning and agentic LLMs at scale.
AReaL is a large-scale asynchronous reinforcement learning (RL) system designed to bridge foundation model training with modern agent-based applications. Originally built by researchers and engineers from Tsinghua IIIS and the AReaL Team at Ant Group, it is now the go-to open-source infrastructure for training reasoning and agentic models efficiently.
🚀 Want to deploy AReaL yourself?
Docker configs, system requirements, and installation guides — all on one page.
View AReaL Tool Page →Why asynchronous RL?
Traditional synchronous RL training wastes GPU cycles while the policy waits for rollouts. AReaL decouples generation, training, and weight updates so that all stages run in parallel. The result is stable, industry-leading throughput that scales from a single 1.5B model to frontier reasoning models.
Key highlights
| Feature | Why it matters |
|---|---|
| ⚡ Fully asynchronous training | No GPU idle time between rollout and update |
| 🔌 Agentic RL by design | Plug in black-box agent apps by swapping base_url |
| 📈 Cutting-edge performance | SOTA math, coding, and search agents out of the box |
Scalability proven in the wild
AReaL's asynchronous pipeline consistently outperforms synchronous baselines in throughput, as shown in the training comparisons below — making it a favorite for large-scale RL workloads.
Released under the permissive Apache-2.0 license with 5.6K+ GitHub stars, AReaL is production-ready: Docker images, example configs for math, coding, and SWE agent training, and full documentation are all included.
🚀 Ready to run AReaL?
Get the Docker setup, requirements, and deployment guide.
View AReaL Tool Page →