A Practical Guide to Generative Music AI for Developers April 2025 hero

Creative Coding

A Practical Guide to Generative Music AI for Developers April 2025

Level

Beginner

Duration

36h 9m of video content

Format

Self-paced video

Certified course — cohort places coming soon

Register interest

Watch a preview

An Introduction to AI Music (50')

Course overview

Course content

Session 1: An Introduction to AI Music

2 videos, 1 resource, 2 lessons

+
  • Session 1 Recording
    Checking access...
  • Main Topics: AI Music Case Studies, Course Roadmap
  • An Introduction to AI Music (50')
    Checking access...
  • Session 1 PDF handout
  • Open Discussion

Session 2: Setting up your environment

6 videos, 3 lessons

+
  • Session 2 Recording
    Checking access...
  • Main Topics: Environment Setup
  • GitHub Repository
  • Setting up your environment - Cloning the class repository (10')
    Checking access...
  • Hands On
  • Setting up your environment - Hands On 1.1: Loading, visualizing, playing audio (10')
    Checking access...
  • Setting up your environment - Hands On 1.2: Extracting Audio Features, RMS and ZCR (13')
    Checking access...
  • Setting up your environment - Hands On 1.2: Extracting Audio Features, Spectrograms (13')
    Checking access...
  • Setting up your environment - Hands On 2: Manipulating MIDI Data (20')
    Checking access...

Session 3: Core Machine Learning Concepts for Music and Audio

6 videos, 2 lessons

+
  • Session 3 Recording
    Checking access...
  • Main Topics: Audio vs Symbolic Music, Basics of Generative AI, Data Acquisition and Ethics
  • Statistical Basics of Generative Modeling in Artificial Intelligence (10')
    Checking access...
  • Variational Autoencoders (16')
    Checking access...
  • Hands On
  • Hands On 1.1: Lakh MIDI Dataset (12')
    Checking access...
  • Hands On 1.2: Free Music Archive (6')
    Checking access...
  • Hands On 2: Using RAVE
    Checking access...

Session 4: Real-Life Collaborations between Artists and Engineers with Guest Speaker Jordan Rudess

2 videos, 2 lessons

+
  • Session 4 Recording
    Checking access...
  • Main Topics: Human-Computer Interaction, Iterative Design, Continuous Deployment
  • Human-Computer Interaction & User-Centered Design
    Checking access...
  • Open Discussion with Jordan Rudess

Session 5: Representation Learning for Music

4 videos, 2 lessons

+
  • Session 5 Recording
    Checking access...
  • Deep Dive into MIDI & Spectrograms
  • Comparing Musical Representations & Encodec Deep Dive (14')
    Checking access...
  • Understanding RVQ in Encodec (12')
    Checking access...
  • Hands On
  • Hands On: Encodec (29')
    Checking access...

Session 6: Autoregressive Music Generation

5 videos, 2 lessons

+
  • Session 6 Recording
    Checking access...
  • Main Topics: Autoregressive modeling, the Transformer architecture, HuggingFace Hub
  • The Transformer architecture (15')
    Checking access...
  • Understanding Anticipatory Music Transformers (13')
    Checking access...
  • Hands On
  • Hands On: Using AMT to generate MIDI data (Part 1) (18')
    Checking access...
  • Hands On: Using AMT to generate MIDI data (Part 2) (17')
    Checking access...

Session 7: Autoregressive Music Generation (Part 2)

3 videos, 2 lessons

+
  • Session 7 Recording
    Checking access...
  • Main Topics: MusicGen & Audio Generation with Transformers
  • Understanding MusicGen (8')
    Checking access...
  • Hands On
  • Hands On: Using MusicGen to generate audio (38')
    Checking access...

Session 8: Diffusion Models for Music Generation

8 videos, 2 lessons

+
  • Session 8 Recording
    Checking access...
  • Main Topics: Diffusion Models, Latent Diffusion Models
  • Intro to Diffusion Models Part 1 (11')
    Checking access...
  • Intro to Diffusion Models Part 2 (14')
    Checking access...
  • Conditioning & Classifier-Free Guidance (10')
    Checking access...
  • The UNet Architecture (6')
    Checking access...
  • Inference-Time Optimization: DITTO (6')
    Checking access...
  • Hands On
  • Hands On: Using Stable Audio Part 1 (15')
    Checking access...
  • Hands On: Using Stable Audio Part 2 (18')
    Checking access...

Session 9: Real-Time Generative AI & Commercial Applications of Generative AI in Music [Guest: Christian Steinmetz]

4 videos, 2 lessons

+
  • Session 9 Recording
    Checking access...
  • Introduction to TorchScript (18')
    Checking access...
  • 8-bit Linear Quantization (9')
    Checking access...
  • ONNX and Graph Optimizations (15')
    Checking access...
  • Main Topics: Landscape of companies in AI and Music, Available Commercial Products
  • Demo

Session 10: Final Project Planning

6 videos, 2 lessons

+
  • Session 10 Recording 1
    Checking access...
  • Session 10 Recording 2
    Checking access...
  • Session 10 Recording 3
    Checking access...
  • Main Topics: Setting up a project specification, timeline, and scope
  • Training Part 1 (8')
    Checking access...
  • Training Part 2 (13')
    Checking access...
  • Training Part 3 (16')
    Checking access...
  • Peer Review & Feedback

Session 11: Final Project Lab

1 video, 2 lessons

+
  • Session 11
    Checking access...
  • Lab Session: Guided Coding & Troubleshooting
  • Milestone Check-Ins

Session 12: Project Showcase & Next Steps

1 video, 2 lessons

+
  • Session 12 Recording
    Checking access...
  • Final Presentations
  • Next Steps

Instructors

Lancelot Blanchard

Lancelot Blanchard

Instructor

Lancelot Blanchard is a musician, engineer, and AI researcher at the MIT Media Lab's Responsive Environments group. His research focuses on generative AI systems for human-AI co-created live musical performances, pioneering the concept of Symbiotic Virtuosity in AI-human music collaboration. He has collaborated with Grammy-winning keyboardist Jordan Rudess and brings eight years of software engineering experience to his teaching.

Frequently asked questions

Interested in this course?

Register your interest and we'll let you know when the next cohort opens.

Certified course — cohort places coming soon

Register interest