AudioCraft: Meta's Open-Source AI for Music, Sound Effects & Audio Compression

What Is AudioCraft?

AudioCraft is Meta's open-source AI audio generation library — a unified framework for creating music, sound effects, and compressed audio using deep learning. With over 23,000 GitHub stars, it's the most comprehensive open-source audio generation toolkit available, featuring three powerful models in one package.

Developed by Meta's Fundamental AI Research (FAIR) team, AudioCraft includes MusicGen for text-to-music generation, AudioGen for text-to-sound effects, and EnCodec for high-fidelity neural audio compression. Together, these models let you generate full songs from text descriptions, create sound effects for games and videos, and compress audio efficiently with minimal quality loss.

The Three Models Inside AudioCraft

1. MusicGen — Text-to-Music

MusicGen generates high-quality music from text descriptions. Describe what you want in natural language — "a relaxing jazz piano trio with soft drums" — and MusicGen creates a complete musical piece. It uses a single-stage auto-reggressive transformer trained on 20,000 hours of licensed music, supporting both mono and stereo output. The model produces 32kHz audio and can generate prompts up to 30 seconds in length.

2. AudioGen — Text-to-Sound

AudioGen creates environmental sounds from text prompts. Need a dog barking, rain on a roof, footsteps on gravel, or a car engine starting? AudioGen generates realistic sound effects from text descriptions. It's trained on 2,000+ hours of sound data across hundreds of categories, making it invaluable for game development, video production, and audio prototyping.

3. EnCodec — Neural Audio Compression

EnCodec is Meta's state-of-the-art neural audio codec that compresses audio at multiple bitrates while maintaining high fidelity. It uses a convolutional encoder-decoder network with a residual vector quantizer, achieving transparent quality at just 1.5 kbps. This is the same model powering Meta's end-to-end audio transmission systems.

Why AudioCraft Matters

1. Unified Framework for Audio Generation

AudioCraft provides a single codebase for training and deploying multiple audio generation models. Same data loading, same training loop, same inference pipeline — regardless of whether you're generating music, sound effects, or compressing audio. This dramatically reduces the learning curve compared to using separate tools for each task.

2. Custom Model Training

Beyond using pre-trained models, AudioCraft supports custom training and fine-tuning. You can train MusicGen on your own music dataset to generate music in a specific style or genre. The framework handles data preparation, tokenization, and training automatically with well-documented configuration files.

3. State-of-the-Art Quality

In objective and subjective evaluations, MusicGen consistently ranks among the best open-source music generation models. Its music output has been described as "surprisingly coherent" by reviewers, with proper structure, harmony, and rhythm — not just random notes.

4. Stereo Output

MusicGen supports stereo audio generation, producing full stereo mixes rather than mono output. This is a significant advantage over many other text-to-music models that only generate mono audio.

5. Active Research Foundation

AudioCraft is an active Meta FAIR research project with regular updates and improvements. It's not an abandoned experiment — it's a living codebase that has evolved through multiple versions, with ongoing research into better models and capabilities.

Getting Started

Installation

pip install 'torch>=2.0' torchaudio
pip install 'audiocraft>=1.1.0'

Generate Music from Text

from audiocraft.models import MusicGen
from audiocraft.utils import export
import soundfile as sf

model = MusicGen.get_pretrained("facebook/musicgen-medium")
model.set_generation_params(duration=8)

wav = model.generate([
    "A cheerful acoustic guitar melody with soft percussion",
    "Atmospheric synth pad with gentle rain sounds"
])

sf.write("music_output.wav", wav[0].cpu().numpy().T, 32000)

Generate Sound Effects

from audiocraft.models import AudioGen

model = AudioGen.get_pretrained("facebook/audiogen-medium")
model.set_generation_params(duration=5)

wav = model.generate(["Dog barking in the distance", "Rain on a metal roof"])
sf.write("sfx_output.wav", wav[0].cpu().numpy().T, 16000)

Self-Hosting with Docker

docker pull dustynv/audiocraft:latest

docker run -d \
  --name audiocraft \
  --gpus all \
  -p 8888:8888 \
  -v ./output:/output \
  dustynv/audiocraft:latest

AudioCraft vs Other Audio Tools

Best Use Cases

Music Prototyping

Quickly generate musical ideas for songs, commercials, and media projects. MusicGen's output can serve as starting points for full compositions.

Game Audio

Generate sound effects on demand for game development. AudioGen can create any environmental sound from text, eliminating the need for expensive sound libraries.

Video Production

Create custom background music and sound effects for videos, podcasts, and social media content without licensing issues.

Research & Education

AudioCraft's modular architecture makes it ideal for studying and experimenting with audio generation techniques.

Tips for Best Results

• Use facebook/musicgen-medium for the best quality/speed balance — large is better but much slower
• Keep prompts descriptive — include genre, instruments, mood, and tempo for better results
• Generate at 8 seconds as a starting point, then extend with continuations
• Use stereo mode (model.lm.stereo=True) for fuller sound
• Use GPU with 8GB+ VRAM for the medium model, 12GB+ for large

Conclusion

AudioCraft is more than just a music generator — it's a complete audio AI toolkit. MusicGen for music, AudioGen for sound effects, and EnCodec for compression, all in one unified framework from Meta FAIR. With 23k GitHub stars, MIT licensing, and support for custom training, it's the most versatile open-source audio generation platform available.