Creative Coding
Course page reconstructed from Postgres and Bunny.
Session 1 Recording
Session 1 Recording • Main Topics: AI Music Case Studies, Course Roadmap • An Introduction to AI Music (50') • Open Discussion
Session 2 Recording • Main Topics: Environment Setup • GitHub Repository • Setting up your environment - Cloning the class repository (10') • Hands On • Setting up your environment - Hands On 1.1: Loading, visualizing, playing audio (10') • Setting up your environment - Hands On 1.2: Extracting Audio Features, RMS and ZCR (13') • Setting up your environment - Hands On 1.2: Extracting Audio Features, Spectrograms (13') • Setting up your environment - Hands On 2: Manipulating MIDI Data (20')
Session 3 Recording • Main Topics: Audio vs Symbolic Music, Basics of Generative AI, Data Acquisition and Ethics • Statistical Basics of Generative Modeling in Artificial Intelligence (10') • Variational Autoencoders (16') • Hands On • Hands On 1.1: Lakh MIDI Dataset (12') • Hands On 1.2: Free Music Archive (6') • Hands On 2: Using RAVE
Session 4 Recording • Main Topics: Human-Computer Interaction, Iterative Design, Continuous Deployment • Human-Computer Interaction & User-Centered Design • Open Discussion with Jordan Rudess
Session 5 Recording • Deep Dive into MIDI & Spectrograms • Comparing Musical Representations & Encodec Deep Dive (14') • Understanding RVQ in Encodec (12') • Hands On • Hands On: Encodec (29')
Session 6 Recording • Main Topics: Autoregressive modeling, the Transformer architecture, HuggingFace Hub • The Transformer architecture (15') • Understanding Anticipatory Music Transformers (13') • Hands On • Hands On: Using AMT to generate MIDI data (Part 1) (18') • Hands On: Using AMT to generate MIDI data (Part 2) (17')
Session 7 Recording • Main Topics: MusicGen & Audio Generation with Transformers • Understanding MusicGen (8') • Hands On • Hands On: Using MusicGen to generate audio (38')
Session 8 Recording • Main Topics: Diffusion Models, Latent Diffusion Models • Intro to Diffusion Models Part 1 (11') • Intro to Diffusion Models Part 2 (14') • Conditioning & Classifier-Free Guidance (10') • The UNet Architecture (6') • Inference-Time Optimization: DITTO (6') • Hands On • Hands On: Using Stable Audio Part 1 (15') • Hands On: Using Stable Audio Part 2 (18')
Session 9 Recording • Introduction to TorchScript (18') • 8-bit Linear Quantization (9') • ONNX and Graph Optimizations (15') • Main Topics: Landscape of companies in AI and Music, Available Commercial Products • Demo
Session 10 Recording 1 • Session 10 Recording 2 • Session 10 Recording 3 • Main Topics: Setting up a project specification, timeline, and scope • Training Part 1 (8') • Training Part 2 (13') • Training Part 3 (16') • Peer Review & Feedback
Session 11 • Lab Session: Guided Coding & Troubleshooting • Milestone Check-Ins
Session 12 Recording • Final Presentations • Next Steps
Lancelot Blanchard is a musician, engineer, and AI researcher at the MIT Media Lab's Responsive Environments group. His research focuses on generative AI systems for human-AI co-created live musical performances, pioneering the concept of Symbiotic Virtuosity in AI-human music collaboration. He has collaborated with Grammy-winning keyboardist Jordan Rudess and brings eight years of software engineering experience to his teaching.