A guide to seven powerful programs for music and visuals
Dom Aversano
What should I learn? A guide to seven powerful programs for music and visuals.
The British saxophonist Shabaka Hutchings described an approach to learning music that reduces it down to two tasks: the first is to know what to practise, and the second is to practise it. The same approach works for coding, and though it is a simple philosophy that does not necessarily make it easy. Knowing what to practise can feel daunting amid such a huge array of tools and approaches, making it all the more important to be clear about what you wish to learn so you can then devote yourself without doubt or distraction to the task of studying.
As ever the most important thing is not the tool but the skills, knowledge, and imagination of the person using it. However, nobody wants to attempt to hammer a nail into the wall with a screwdriver. Some programs are more suited to certain tasks than others, so it is important to have a sense of their strengths and weaknesses before taking serious steps into learning them.
What follows is a summary and description of some popular programs to help you navigate your way to what inspires you most, so you can learn with passion and energy.
Pure Data
Pure Data is an open-source programming language for audio and visual (GEM) coding that was developed by Miller Puckette in the mid-1990s. It is a dataflow language where objects are patched together using cords, in a manner appealing to those who like to conceptualise programs as a network of physical objects.
Getting started in Pure Data is not especially difficult even without any programming experience, since it has good documentation and plenty of tutorials. You can build interesting and simple programs within days or weeks, and with experience, it is possible to build complex and professional programs.
The tactile and playful process of patching things together also represents a weakness of Pure Data, since once your programs become more advanced you need increasing numbers of patch cables, and dragging hundreds – or even thousands – of them from one place to another becomes monotonous work.
Cost: free
Max/MSP/Jitter and Max for Live
Max/MSP is Pure Data’s sibling, which makes it quite easy to migrate from one program to the other, but there are significant and important differences too. The graphical user interface (GUI) for Max is more refined and allows for organising patching chords in elegant ways that help mental clarity. With Max for Live you have Max built into Ableton – bringing together two powerful programs.
Max has a big community surrounding it in which you can find plenty of tutorials, Discord channels, and a vast library of instruments to pull apart. Just as Pure Data has GEM for visualisation Max has Jitter, in which you can create highly sophisticated visuals. All in all, this represents an incredibly powerful setup for music and visuals.
The potential downsides are that Max is paid, so if you’re on a small budget Pure Data might be better suited. It also suffers from the same patch cord fatigue as Pure Data, where you can end up attaching cords from one place to another in a repetitive manner.
Cost: $9.99 per month / $399 permanent licence or $250 for students and teachers
SuperCollider
SuperCollider is an open-source language developed by James McCartney that was released in 1996, and a more traditional programming language than either Pure Data or Max. If you enjoy coding it is an immensely powerful tool where your imagination is the limit when it comes to sound design, since with as little as a single line of code you are capable of creating stunning musical outputs.
However, SuperCollider is difficult, so if you have no programming experience expect to put in many hours before you feel comfortable. Its documentation is inconsistent and written in a way that sometimes assumes a high level of technical understanding. Thankfully, there is a generous and helpful online forum that is very welcoming to newcomers, so if you are determined to learn, do not be put off by the challenge.
An area that SuperCollider is lacking in comparison to Max and Pure Data is a sophisticated built-in environment for visuals, and although you can use it to create GUIs, they do not have the same elegance as in Max.
Cost: free
TidalCycles
Though built from SuperCollider, TidalCycles is nevertheless much easier to learn. Designed for the creation of algorithmic music, it is popular in live coding or algorave music. The language is intuitive and uses music terminology in its syntax, giving people with an existing understanding of music an easy way into coding. There is a community built around it complete with Discord channels and an active community blog.
The downsides to TidalCycles are the installation is difficult, and it is a somewhat specialist tool that does not have as broad capabilities as the aforementioned programs.
Cost: free
P5JS
P5JS is an open-source Javascript library that is a tool of choice for generative visual artists. The combination of a gentle learning curve and the ease of being able to run it straight from your browser makes it something easy to incorporate into one’s life, either as a simple tool for sketching out visual ideas or as something much more powerful that is capable of generating world-class works of art.
It is hard to mention P5JS without also mentioning Daniel Shiffmen, one of the most charismatic, humorous, and engaging programming teachers, who has rightly earned himself a reputation as such. He is the authour of a fascinating book called The Nature of Code which takes inspiration from natural systems, and like P5JS is open-source and freely available.
Cost: free
Tone.js
Like P5JS, Tone.js is also a Javascript library, and one that opens the door to a whole world of musical possibilities in the web browser. In the words of its creators it ‘offers common DAW (digital audio workstation) features like a global transport for synchronizing and scheduling events as well as prebuilt synths and effects’ while allowing for ‘high-performance building blocks to create your own synthesizers, effects, and complex control signals.’
Since it is web based one can get a feel for it by delving into some of the examples on offer.
Cost: free
TouchDesigner
In TouchDesigner you can create magnificent live 3D visuals without the need for coding. Its visual modular environment allows you to patch together modules in intuitive and creative ways, and it is easy to input midi or OSC if you want to incorporate a new visual dimension to your music. To help learn there is an active forum, live meetups, and many tutorial videos on this site. While the initial stages of using TouchDesigner are not difficult, one can become virtuosic with the option of even writing your own code in the programming language Python.
There is a showcase of work made using TouchDesigner on their website which gives you a sense of what it is capable of.
Cost: All features $2200 / pro version $600 / free for personal and non-commercial use.
Getting Started with TouchDesigner – September series
Dates & Times: Thursdays: 9th / 16th / 23rd / 30th September 6pm UK / 7pm Berlin / 10am LA / 1pm NYC
Length: 2-hours each session
Level: Beginners curious about creating immersive visuals with TouchDesigner
TouchDesigner is a powerful tool for creating live performances, installations, real time visuals and complex digital systems. In this series of workshops you’ll learn the basic functioning of TD’s node-based workflow and how to use them to create media, use the data to manipulate content and how to organize and navigate your TouchDesigner networks.
Series Learning Outcomes
By the end of this series a successful student will be able to create content by:
-
Becoming familiar with the User Interface
-
Exploring TouchDesigner’s building blocks
-
Constructing dynamic visual effects
-
Applying basic audio reactivity and other techniques
See below for all 4 weeks
Series Study Topics
-
TouchDesigner User Interface and patching
-
How to use building blocks (operators)
-
Controlling parameters for dynamic visuals
-
Audio reactivity and visual effects techniques
See below for all 4 weeks
Session 1
Topics
-
Overview, Interface, Mouse & Shortcuts
-
Operator Families & Operator Structure
-
TOPs – Texture Operators
-
CHOPs – Channel Operators
Learning Outcomes
By the end of this session a successful student will be able to:
1: Navigate the TouchDesigner User Interface and utilise mouse and shortcuts controls
2: Become familiar with operator families and their structure
3: Construct basic networks via TOPs
4: Export CHOPs to create dynamic networks
Session 2
Topics
-
SOPs, MATs and Rendering
-
DATs
-
COMPs and Network Structure
-
Palette & Additional Info
Learning Outcomes
By the end of this session a successful student will be able to:
1: Build rendering networks via SOPs and MATs
2: Utilise and apply data via DAT operators
3: Construct custom networks via COMPs
4: Locate examples and info from resources such as the Palette
Session 3
Topics
-
Audio Overview
-
Instancing
-
Building a Project 1
Learning Outcomes
By the end of this session a successful student will be able to:
1: Utilise audio CHOPs within a network
2: Apply audio CHOPs as modulation sources
3: Build basic Instancing networks
4: Begin to construct a bespoke self directed project
Project Overview
During Session 3 and 4, you will build a self directed and bespoke project in TouchDesigner. You will be supported by the workshop leader and the teaching assistants during these sessions. You can incorporate any techniques you have learnt from the workshop series into the project.
Examples include:
-
TOPs driven from CHOPs
-
Dynamic / generative art
-
Audio reactive content
-
Basic Instancing network
-
3D geometry via SOPs
In session 4 you can share your project to your peers in your Zoom break-out room. You will also be asked to provide feedback on your peers project’s.
Session 4
Topics
-
Building a Project 2
Learning Outcomes
By the end of this session a successful student will be able to:
1: Work independently, with support, within the TD environment
2: Apply various techniques from the workshop series to construct dynamic networks
3: Continue to construct a bespoke self directed project
4: Share their work with peers and provide feedback to peers
Requirements
-
A computer with internet connection
-
A web cam and mic
-
A Zoom account
-
A three button mouse or to configure Apple Track Pad appropriately
-
TouchDesigner (free version suffices https://derivative.ca/download)
-
If you’re on Mac please check TouchDesigner can run on your system (i.e. has basic GPU requirements such as Intel HD4000 or better)
About the workshop leader
Bileam Tschepe aka elekktronaut is a Berlin based artist and educator who creates audio-reactive, interactive and organic digital artworks, systems and installations in TouchDesigner, collaborating with and teaching people worldwide
Immersive AV Composition -On demand / 2 Sessions
Level: Advanced
These workshops will introduce you to the ImmersAV toolkit. The toolkit brings together Csound and OpenGL shaders to provide a native C++ environment where you can create abstract audiovisual art. You will learn how to generate material and map parameters using ImmersAV’s Studio() class. You will also learn how to render your work on a SteamVR compatible headset using OpenVR. Your fully immersive creations will then become interactive using integrated machine learning through the rapidLib library.
Session Learning Outcomes
By the end of this session a successful student will be able to:
-
Setup and use the ImmersAV toolkit
-
Discuss techniques for rendering material on VR headsets
-
Implement the Csound API within a C++ application
-
Create mixed raymarched and raster based graphics
-
Create an interactive visual scene using a single fragment shader
-
Generate the mandelbulb fractal
-
Generate procedural audio using Csound
-
Map controller position and rotation to audiovisual parameters using machine learning
Session Study Topics
-
Native C++ development for VR
-
VR rendering techniques
-
Csound API integration
-
Real-time graphics rendering techniques
-
GLSL shaders
-
3D fractals
-
Audio synthesis
-
Machine learning
Requirements
-
A computer and internet connection
-
A web cam and mic
-
A Zoom account
-
Cloned copy of the ImmersAV toolkit plus dependencies
-
VR headset capable of connecting to SteamVR
About the workshop leader
Bryan Dunphy is an audiovisual composer, musician and researcher interested in generative approaches to creating audiovisual art in performance and immersive contexts. His work explores the interaction of abstract visual shapes, textures and synthesised sounds. He is interested in exploring strategies for creating, mapping and controlling audiovisual material in real time. He has recently completed his PhD in Arts and Computational Technology at Goldsmiths, University of London.
