Level: Intermediate

In this workshop you will use openFrameworks to build a real-time audiovisual instrument. You will generate dynamic abstract visuals within openFrameworks and procedural audio using the ofxMaxim addon. You will then learn how to control the audiovisual material by mapping controller input to audio and visual parameters using the ofxRapid Lib add on.

Session Learning Outcomes

By the end of this session a successful student will be able to:

• Create generative visual art in openFrameworks

• Create procedural audio in openFrameworks using ofxMaxim

• Discuss interactive machine learning techniques

• Use a neural network to control audiovisual parameters simultaneously in real-time

Session Study Topics

• 3D primitives and perlin noise

• FM synthesis

• Regression analysis using multilayer perceptron neural networks

• Real-time controller integration

Requirements

• A computer and internet connection

• A web cam and mic

• A Zoom account

• Installed version of openFrameworks

• Downloaded addons ofxMaxim, ofxRapidLib

• Access to MIDI/OSC controller (optional – mouse/trackpad will also suffice)

About the workshop leader 

Bryan Dunphy is an audiovisual composer, musician and researcher interested in generative approaches to creating audiovisual art. His work explores the interaction of abstract visual shapes, textures and synthesised sounds. He is interested in exploring strategies for creating, mapping and controlling audiovisual material in real time. He has recently completed his PhD in Arts and Computational Technology at Goldsmiths, University of London.

In this workshop, you will learn to build incredible live videos with particles systems, using Max and Jitter.

Cycling’74 has recently released GL3, which ties together more closely Jitter with Open GL, and optimises use of the GPU. With this recent update available in the package manager, you can build highly performance videos without having to code them in C++.

Requirements

• Latest version of Max 8 installed on Mac or Windows
• A good working knowledge of Max is expected
• Understanding of how the GEN environment works in Jitter
• Some familiarity with textual programming languages
• A knowledge of basic calculus is a bonus
• The GL3 package installed
• To install this package open the “Package Manager” from within Max, look for the GL3 package and click “install”.

What you will learn

Session 1, 20th October, 6pm UK / 10am PDT / 1pm EST:

– Introduction to GL3 features

– Quick overview of most of the examples in the GL3 package

– Build a simple particle system from scratch

– Explorations with gravity/wind

– Exploration with target attraction

Session 2, 27th October, 6pm UK / 10am PDT / 1pm EST:

– Improve particle system with rendering billboard shader

– Creation of a “snow” or “falling leaves” like effect

– Starting to introduce interactivity in the system

– Using the camera input

– Connecting sound to your patches

Session 3, 3rd November, 6pm UK / 10am PDT / 1pm EST:

– Improve the system interactivity

– Particles emitting from object/person outline taken from camera

– Create a particle system using 3D models and the instancing technique

– Transforming an image or a video stream into particles

Session 4, 10th November, 6pm UK / 10am PDT / 1pm EST:

– Introduction to flocking behaviours and how to achieve them in GL3

– Create a 3D generative landscape and modify it using the techniques from previous sessions

– Apply post-processing effects

About the workshop leader:

Federico Foderaro is an audiovisual composer, teacher and designer for interactive multimedia installations, author of the YouTube channel Amazing Max Stuff.
Graduated in Electroacoustic Musical Composition at the Licinio Refice Conservatory in Frosinone cum laude, he has lived and worked in Berlin since 2016.

His main interest is the creation of audiovisual works and fragments, where the technical research is deeply linked with the artistic output.
The main tool used in his production is the software Max/MSP from Cycling74, which allows for real-time programming and execution of both audio and video, and represents a perfect mix between problem-solving and artistic expression.

Beside his artistic work, Federico teaches the software Max/MSP, both online and in workshops in different venues. The creation of commercial audio-visual interactive installations is also a big part of his work life, having led in the years to satisfactory collaborations and professional achievements.