AudioCraft MusicGen Tutorial: Generate Music from Text with Meta's AI

AudioCraft MusicGen Tutorial: Generate Music from Text Prompts

This hands-on tutorial shows you how to generate music and sound effects using AudioCraft's MusicGen and AudioGen models — from installation to custom generation techniques and Docker deployment.

Prerequisites

• Python 3.9+ with pip
• NVIDIA GPU with 8GB+ VRAM recommended (medium model)
• 12GB+ VRAM for the large model
• 5GB free disk space for model weights
• Docker (optional, for containerized deployment)

Step 1: Install AudioCraft

pip install 'torch>=2.0' torchaudio --index-url https://download.pytorch.org/whl/cu118
pip install 'audiocraft>=1.1.0'

Step 2: Generate Your First Music Track

from audiocraft.models import MusicGen
import soundfile as sf

# Load the medium model (3GB VRAM)
model = MusicGen.get_pretrained("facebook/musicgen-medium")

# Configure generation: 8 seconds, temperature 0.9
model.set_generation_params(
    duration=8,
    temperature=0.9,
    top_k=250,
    top_p=0.0
)

# Generate from text
wav = model.generate([
    "Upbeat electronic dance music with synth bass and house beat, 130 BPM"
])

# Save
sf.write("edm_track.wav", wav[0].cpu().numpy().T, 32000)

On first run, the model downloads ~3GB of weights. Subsequent runs are instant.

Step 3: Advanced Generation Techniques

Multi-Prompt Generation

Generate multiple variations from different prompts in one call:

prompts = [
    "Soulful R&B with smooth vocals and piano, 90 BPM",
    "Dark ambient drone with subtle textures and bass",
    "Acoustic folk guitar fingerpicking with soft percussion"
]
wavs = model.generate(prompts)

for i, prompt in enumerate(prompts):
    sf.write(f"track_{i}.wav", wavs[i].cpu().numpy().T, 32000)

Melody Conditioning

Guide MusicGen with a reference melody:

import torchaudio
from audiocraft.utils import export

# Load a melody to condition on
melody, sr = torchaudio.load("reference_melody.wav")
melody = torchaudio.functional.resample(melody, sr, 32000)

# Generate with melody + text
wav = model.generate_with_chroma(
    ["Jazz quartet improvisation over this melody"],
    melody[None].expand(1, -1, -1),
    sample_rate=32000
)

sf.write("jazz_improvisation.wav", wav[0].cpu().numpy().T, 32000)

Stereo Generation

model.lm.stereo = True  # Enable stereo
model.set_generation_params(duration=10)

wav = model.generate(["Wide ambient soundscape with stereo panning effects"])

# wav shape: (channels, samples) — now 2 channels
sf.write("stereo_track.wav", wav[0].cpu().numpy().T, 32000)

Step 4: Generate Sound Effects with AudioGen

from audiocraft.models import AudioGen

model = AudioGen.get_pretrained("facebook/audiogen-medium")
model.set_generation_params(duration=5)

sounds = model.generate([
    "Thunderstorm with heavy rain and wind",
    "Coffee shop ambience with people chatting",
    "Car engine starting and driving away"
])

for i, s in enumerate(sounds):
    sf.write(f"sfx_{i}.wav", s.cpu().numpy().T, 16000)

Step 5: Docker Deployment

docker pull dustynv/audiocraft:latest

docker run -d \
  --name audiocraft \
  --gpus all \
  -p 8888:8888 \
  -v ./output:/output \
  dustynv/audiocraft:latest

# Run generation inside container
docker exec audiocraft python3 << 'EOF'
from audiocraft.models import MusicGen
import soundfile as sf

model = MusicGen.get_pretrained("facebook/musicgen-medium")
model.set_generation_params(duration=8)

wav = model.generate(["Lofi hip hop with warm vinyl crackle and piano"])
sf.write("/output/lofi_track.wav", wav[0].cpu().numpy().T, 32000)
print("Done!")
EOF

Generation Parameters Guide

Parameter Recommendations

• Music: temperature=0.9, top_k=250, cfg_coef=3.0
• Sound effects: temperature=0.7, top_k=100, cfg_coef=4.0
• Melody following: temperature=0.5, top_k=100, cfg_coef=2.0
• Experimental: temperature=1.5, top_k=500, cfg_coef=1.5

Troubleshooting

Problem: "CUDA out of memory" with medium model

Solution: Use facebook/musicgen-small instead (requires only 2GB VRAM). Or enable CPU offloading: model = MusicGen.get_pretrained("facebook/musicgen-medium", device="cuda") with model.lm.to(device).

Problem: Generated audio has artifacts

Solution: Lower temperature to 0.7-0.8 and reduce top_k to 150-200. Also ensure you're using the correct sample rate (32000 for MusicGen, 16000 for AudioGen) when saving.

Problem: Stereo output is mono

Solution: Set model.lm.stereo = True BEFORE calling set_generation_params(). Not all models support stereo.

Problem: Download hangs on first run

Solution: Pre-download from HuggingFace: huggingface-cli download facebook/musicgen-medium or use a VPN if in a region with restricted access.

Building an API Server

# save as musicgen_api.py
from audiocraft.models import MusicGen
from flask import Flask, request, send_file
import tempfile, soundfile as sf, os

app = Flask(__name__)
model = MusicGen.get_pretrained("facebook/musicgen-medium")
model.set_generation_params(duration=8)

@app.route("/generate", methods=["POST"])
def generate():
    prompt = request.json.get("prompt", "Piano melody")
    duration = request.json.get("duration", 8)
    model.set_generation_params(duration=duration)
    wav = model.generate([prompt])
    tmp = tempfile.NamedTemporaryFile(suffix=".wav", delete=False)
    sf.write(tmp.name, wav[0].cpu().numpy().T, 32000)
    return send_file(tmp.name, mimetype="audio/wav")

app.run(host="0.0.0.0", port=8888)

Conclusion

AudioCraft makes AI music generation accessible to everyone. MusicGen's text-to-music, AudioGen's text-to-sound, and EnCodec's compression — all in one MIT-licensed framework from Meta. Whether you're a musician prototyping ideas, a game developer generating sound effects, or a researcher exploring audio AI, AudioCraft has the tools you need.