AReaL Tutorial: Train Your First AI Agent with Asynchronous Reinforcement Learning
A hands-on guide to training agents with AReaL: pull the runtime, point it at any agent app, and launch async RL training.
In this tutorial, we walk through training your first agent with AReaL — the large-scale asynchronous RL system from Tsinghua IIIS and Ant Group. If you can run Docker and a GPU, you can follow along.
🚀 Want to deploy AReaL yourself?
Docker configs, system requirements, and installation guides — all on one page.
View AReaL Tool Page →Step 1: Pull the runtime
AReaL ships a Dockerfile with sglang and vllm variants. Build or pull the image, then mount a data directory:
docker build -t areal-runtime:dev-sglang .
Step 2: Point AReaL at your agent
The killer feature: AReaL works with black-box agent applications. You don't need to rewrite your agent runtime — just replace the base_url with AReaL's RL service. The project ships complete examples (math, coding, SWE, even an OpenClaw agent) that follow exactly this pattern.
Step 3: Launch asynchronous training
Start with a ready-made config like examples/math/gsm8k_kpop.yaml. AReaL's fully asynchronous paradigm keeps generation, training, and weight updates running in parallel — watch the training curve climb:
Review: strengths and trade-offs
| Strengths | Trade-offs |
|---|---|
| Industry-leading throughput | Requires GPU hardware for real training runs |
| Agent-agnostic by design | Steeper learning curve than simple fine-tuning |
| Active community & docs | Official Docker Hub image still maturing |
For teams serious about reasoning and agentic models, AReaL is one of the most complete open-source RL stacks available today.
🚀 Ready to run AReaL?
Get the Docker setup, requirements, and deployment guide.
View AReaL Tool Page →