Move slow and create things

Dom Aversano

Over Christmas I took a week off, and no sooner had I begun to relax than an inspiring idea came to mind for a generative art piece for an album cover. The algorithm needed to make it was clear in my mind, but I did not want to take precious time away from family and friends to work on it. Then a thought occurred — could I build it quickly using ChatGPT?

I had previously resisted using Large Language Models (LLMs) in my projects for a variety of reasons. Would outsourcing coding gradually deskill me? Whose data was the system trained on and was I participating in their exploitation? Is the environmental effect of using such computationally intense technology justifiable?

Despite my reservations I decided to try it, treating it as an experiment that I could stop at any point. Shortly prior to this, I had read a thought-provoking online comment questioning whether manual coding might seem as peculiar and antiquated to the future as programming in binary does now. Could LLMs help make computers less rigid and fixed, opening up the world of programming to anyone?

While I had previously used ChatGPT to create some simple code for Supercollider, I had been unimpressed by the results. For this project, however, the quality of the code was different. Every prompt returned P5JS code that did exactly what I intended, without the need for clarification. I made precisely what I envisioned in less than 30 minutes. I was astonished. It was not the most advanced program, but neither was it basic.

Despite the success, I felt slightly uneasy. The computer scientist Grady Booch wrote that ‘every line of code represents an ethical and moral decision.’ It is tempting to lose sight of this amid a technological culture steeped in a philosophy of ‘move fast and break things’ and ‘it’s better to ask for forgiveness than permission’. So what specifically felt odd?

I arrived at what I wanted without much of a journey, learning little more than how to clarify my ideas to a machine. This is a stark contrast to the slow and meticulous manner of creation that gradually develops our skills and thinking, which is generally considered quintessential to artistic activity. Furthermore, although the arrival is quicker the destination is not exactly the same, since handcrafted code can offer a representation of a person’s worldview, whereas LLM code is standardised.

However, I am aware that historically many people — not least of all in the Arts and Crafts movement — expressed similar concerns, and one can argue that if machines dramatically reduce labourious work it could free up time for creativity. Removing the technical barrier to entry could allow many more people’s creative ideas to be realised. Yet efficiency is not synonymous with improvement, as anyone who has scanned a QR-code menu at a restaurant can attest.

The idea that LLMs could degrade code is plausible given that they frequently produce poor or unusable code. While they will surely improve, to what degree is unknown. A complicated project built from layers of machine-generated code may create layers of problems: short-term and long-term. Like pollution, its effects might not be obvious until they accumulate and compound over time. If LLMs are trained on LLM-generated code it could have a degradative effect, leading to a Model Collapse.

The ethics of this technology are equally complicated. The current lack of legislation around consent on training LLMs means many people are discovering that their books, music, or code has been used to train a model without their knowledge or permission. Beyond legislating, a promising idea has been proposed by programmer and composer Ed Newton-Rex, who has founded a company called Fairly Trained, which offers to monitor and certify different LLMs, providing transparency on how they were trained.

Finally, while it is hard to find accurate assessments of how much electricity these systems use, some experts predict they could soon consume as much electricity as entire countries, which should not be difficult to imagine given that the Bitcoin blockchain is estimated to consume more electricity than the whole of Argentina.

To return to Grady Booch’s idea that ‘every line of code represents an ethical and moral decision’ one could extend this to every interaction with a computer represents an ethical and moral decision. As the power of computers increases so should our responsibility, but given the rapid increases in computing power, it may be unrealistic to expect our responsibility to keep pace. Taking a step back to reflect does not make one a Luddite, and might be the most technically insightful thing to do. Only from a thoughtful perspective can we hope to understand the deep transformations occurring, and how to harness them to improve the world.

A guide to seven powerful programs for music and visuals

Dom Aversano

What should I learn? A guide to seven powerful programs for music and visuals.

The British saxophonist Shabaka Hutchings described an approach to learning music that reduces it down to two tasks: the first is to know what to practise, and the second is to practise it. The same approach works for coding, and though it is a simple philosophy that does not necessarily make it easy. Knowing what to practise can feel daunting amid such a huge array of tools and approaches, making it all the more important to be clear about what you wish to learn so you can then devote yourself without doubt or distraction to the task of studying.

As ever the most important thing is not the tool but the skills, knowledge, and imagination of the person using it. However, nobody wants to attempt to hammer a nail into the wall with a screwdriver. Some programs are more suited to certain tasks than others, so it is important to have a sense of their strengths and weaknesses before taking serious steps into learning them.

What follows is a summary and description of some popular programs to help you navigate your way to what inspires you most, so you can learn with passion and energy.

Pure Data

Pure Data is an open-source programming language for audio and visual (GEM) coding that was developed by Miller Puckette in the mid-1990s. It is a dataflow language where objects are patched together using cords, in a manner appealing to those who like to conceptualise programs as a network of physical objects. 

Getting started in Pure Data is not especially difficult even without any programming experience, since it has good documentation and plenty of tutorials. You can build interesting and simple programs within days or weeks, and with experience, it is possible to build complex and professional programs.

The tactile and playful process of patching things together also represents a weakness of Pure Data, since once your programs become more advanced you need increasing numbers of patch cables, and dragging hundreds – or even thousands – of them from one place to another becomes monotonous work.

Cost: free

Introductory Tutorial 

Official Website

Max/MSP/Jitter and Max for Live

Max/MSP is Pure Data’s sibling, which makes it quite easy to migrate from one program to the other, but there are significant and important differences too. The graphical user interface (GUI) for Max is more refined and allows for organising patching chords in elegant ways that help mental clarity. With Max for Live you have Max built into Ableton – bringing together two powerful programs.

Max has a big community surrounding it in which you can find plenty of tutorials, Discord channels, and a vast library of instruments to pull apart. Just as Pure Data has GEM for visualisation Max has Jitter, in which you can create highly sophisticated visuals. All in all, this represents an incredibly powerful setup for music and visuals.

The potential downsides are that Max is paid, so if you’re on a small budget Pure Data might be better suited. It also suffers from the same patch cord fatigue as Pure Data, where you can end up attaching cords from one place to another in a repetitive manner.

Cost: $9.99 per month / $399 permanent licence or $250 for students and teachers

Introductory Tutorial

Official Website

SuperCollider

SuperCollider is an open-source language developed by James McCartney that was released in 1996, and a more traditional programming language than either Pure Data or Max. If you enjoy coding it is an immensely powerful tool where your imagination is the limit when it comes to sound design, since with as little as a single line of code you are capable of creating stunning musical outputs. 

However, SuperCollider is difficult, so if you have no programming experience expect to put in many hours before you feel comfortable. Its documentation is inconsistent and written in a way that sometimes assumes a high level of technical understanding. Thankfully, there is a generous and helpful online forum that is very welcoming to newcomers, so if you are determined to learn, do not be put off by the challenge.

An area that SuperCollider is lacking in comparison to Max and Pure Data is a sophisticated built-in environment for visuals, and although you can use it to create GUIs, they do not have the same elegance as in Max.

Cost: free

Introductory Tutorial 

Official website

TidalCycles

Though built from SuperCollider, TidalCycles is nevertheless much easier to learn. Designed for the creation of algorithmic music, it is popular in live coding or algorave music. The language is intuitive and uses music terminology in its syntax, giving people with an existing understanding of music an easy way into coding. There is a community built around it complete with Discord channels and an active community blog.

The downsides to TidalCycles are the installation is difficult, and it is a somewhat specialist tool that does not have as broad capabilities as the aforementioned programs.

Cost: free

Introductory Tutorial 

Official Websit

P5JS

P5JS is an open-source Javascript library that is a tool of choice for generative visual artists. The combination of a gentle learning curve and the ease of being able to run it straight from your browser makes it something easy to incorporate into one’s life, either as a simple tool for sketching out visual ideas or as something much more powerful that is capable of generating world-class works of art.

It is hard to mention P5JS without also mentioning Daniel Shiffmen, one of the most charismatic, humorous, and engaging programming teachers, who has rightly earned himself a reputation as such. He is the authour of a fascinating book called The Nature of Code which takes inspiration from natural systems, and like P5JS is open-source and freely available. 

Cost: free

Introductory Tutorial

Official Website

Tone.js

Like P5JS, Tone.js is also a Javascript library, and one that opens the door to a whole world of musical possibilities in the web browser. In the words of its creators it ‘offers common DAW (digital audio workstation) features like a global transport for synchronizing and scheduling events as well as prebuilt synths and effects’ while allowing for ‘high-performance building blocks to create your own synthesizers, effects, and complex control signals.’

Since it is web based one can get a feel for it by delving into some of the examples on offer

Cost: free

Introductory Tutorial

Official website

TouchDesigner

In TouchDesigner you can create magnificent live 3D visuals without the need for coding. Its visual modular environment allows you to patch together modules in intuitive and creative ways, and it is easy to input midi or OSC if you want to incorporate a new visual dimension to your music. To help learn there is an active forum, live meetups, and many tutorial videos on this site. While the initial stages of using TouchDesigner are not difficult, one can become virtuosic with the option of even writing your own code in the programming language Python. 

There is a showcase of work made using TouchDesigner on their website which gives you a sense of what it is capable of.

Cost: All features $2200 / pro version $600 / free for personal and non-commercial use. 

Introductory Tutorial

Official Website