Visual Music Performance with Machine Learning - On demand hero

Course template / hosted architecture draft

Visual Music Performance with Machine Learning - On demand

Learn to build a real-time audiovisual instrument in openFrameworks.

Level

Level 3

Duration

—

Format

—

Course overview

Learn to build a real-time audiovisual instrument in openFrameworks.

Who is this course for?

  • • In this workshop you will use openFrameworks to build a real-time audiovisual instrument. You will generate dynamic abstract visuals within openFrameworks and procedural audio using the ofxMaxim addon. You will then learn how to control the audiovisual material by mapping controller input to audio and visual parameters using the ofxRapid Lib add on.

Requirements

  • • A computer and internet connection
  • • A web cam and mic
  • • A Zoom account
  • • Installed version of openFrameworks
  • • Downloaded addons ofxMaxim, ofxRapidLib
  • • Access to MIDI/OSC controller (optional - mouse/trackpad will also suffice)

Course content

Session 1

What you will learn in this course

Course Overview · Requirements · Pre-course preparation

+

Session 2

Visual Music Performance with Machine Learning - On demand

Part 1 - Sphere setup · Part 2 - Phong lighting · Part 3 - Camera + Normal matrix

+

Session materials

What you will learn in this course

Pre-course preparation

Open resource

What you will learn in this course

Work sheet with exercises

Open resource

Visual Music Performance with Machine Learning - On demand

Part 1 - Sphere setup

Open resource

Visual Music Performance with Machine Learning - On demand

Part 2 - Phong lighting

Open resource

Visual Music Performance with Machine Learning - On demand

Part 3 - Camera + Normal matrix

Open resource

Visual Music Performance with Machine Learning - On demand

Part 4 - Vertex displacement

Open resource

Visual Music Performance with Machine Learning - On demand

Part 5 - ofxMaxim setup

Open resource

Visual Music Performance with Machine Learning - On demand

Part 6 - Simple FM synth

Open resource

Visual Music Performance with Machine Learning - On demand

Part 7 - Machine Learning - Data collection

Open resource

Visual Music Performance with Machine Learning - On demand

Part 8 - Machine Learning - Train + Run model

Open resource

Visual Music Performance with Machine Learning - On demand

Part 9 - OSC controller

Open resource

Visual Music Performance with Machine Learning - On demand

Finished Project on Github

Open resource

Instructors

Bryan Dunphy

Bryan Dunphy

Instructor

Bryan Dunphy graduated in 2021 from a PhD at Goldsmiths University. He specialises in audio-visual, immersive performances and creations. Most of his work uses Machine Learning.

Frequently asked questions