How an image led to an album

Dom Aversano

The Code / Album by Dom Aversano

Curiosity takes you to new places. I arrived in such a place after contemplating what a highly complex polyrhythm might look like. As an instrumentalist, I am accustomed to placing limits on my thinking based on what is physically possible, but since the digital realm essentially removes most physical constraints, we enter a new world of disembodied possibilities. The following image — created in P5JS — is one such example.

 

Visualisation of 750 polyrhythms

This image depicts 750 polyrhythms juxtaposed one on top of another. The x-axis represents time, and the y-axis is the increasing division of that time. At the very top left of the image, there is a single point. The line beneath has two equidistant points — one at the top left and one at the top centre. The line beneath this has three equidistant points: then four, five, six etc. all the way to 750 divisions. To create this by hand would be painstaking, if not impossible, but coding it is simple.

When I saw the image I was astonished — it contained star-like shapes, mirror symmetry, curved lines at either edge, and a series of interesting patterns across the top. Despite finding the image fascinating, I could not find a use for it, so I shelved it and moved on.

A little while later I decided to share these images on Substack, hoping they might be of interest to someone. To bring the images to life I decided to sonify them, by building a simple custom program in Supercollider. The program quickly morphed into a rabbit hole, as when I tinkered with it I heard new sound worlds awakening. It wasn’t long before I realised I was halfway into creating an album that I had never intended to make.

What captured me about the music was the same as the images: they were humanly impossible. Performing 750 rhythms is completely beyond the capabilities of the human mind, but effortless for a computer. The result was music that was temporally organised, but with no meter or ‘one’ to resolve on. There was a logical flow of patterns, but nothing to tap one’s foot to. Using the harmonic series as a scale allowed vast clusters of tones that the chromatic scale could not accommodate. With this vast number of tones, the distinction between timbre, chords, and notes started to break down.

The idea that computers can unlock forms of artistic expression which lie beyond the capabilities of the human body was described eloquently by the late computer artist Vera Molnar.

Without the aid of a computer, it would not be possible to materialize quite so faithfully an image that previously existed only in the artist’s mind. This may sound paradoxical, but the machine, which is thought to be cold and inhuman, can help to realize what is most subjective, unattainable, and profound in a human being.

Molnar’s proposal that machines can provide access to unattainable realms of artistic expression seemed a strong counter-argument to the romantic notion that machines degrade the subtleties of human expression. Rather than having machines imitate human expression, in Molnar’s interpretation, they could express facets of the human experience that the limits of physicality prevented. The machine rather than deromanticising human expression could be a tool used to express subtle aspects of ourselves.

With this idea in mind, I delved deeper into the visual dimension. One day, it occurred to me that the original polyrhythm image could be visualised circularly. In this case, the rhythms would be represented as rotating divisions in space that could be layered one on top of another. The result was an image distinct from the previous one.

 

Visualisation of 104 polyrhythms

The process for generating this image: Draw one radial line at 0°. Then add two lines equidistant in rotation. Then add three lines equidistant in rotation. Then add four lines, and so on.

The new image looked organic, almost botanical. The big division at the top was matched by half the size at 180° and two more half the size again at 90° and 270°. The dark lines represented points of convergence. The lighter areas are spaces of less density.

I chose the image as the cover for the album since having artwork and music derived from the same algorithm felt satisfying and aesthetically appropriate. Had I not made the initial image I would not have made the music, or at least I would have arrived at it in another way at another time. That this process occurred at all remains a surprise to me, which I treat as a testament to the capacity for curiosity to take us to unknown places.

An interview with Interaction Designer Arthur Carabott Part II

Dom Aversano

The Coca-Cola Beatbox Pavilion from the 2012 London Olympic Games

This is Part II of an interview with interaction designer Arthur Carabott. In Part I Arthur discussed how after studying music technology at Sussex University he found a job working on the Coca-Cola Beatbox Pavilion in the 2012 Olympic Games. What follows is his description of how the work evolved. 

Did you conceive that project in isolation or collaboration?

The idea had already been sold and the architects had won the competition. What was known was there would be something musical because Mark Ronson was going to be making a song. So the idea was to build a giant instrument from a building, which everyone could play by waving their hands over giant pads. They wanted to use sports sounds and turn them into music while having a heartbeat play throughout the building, tying everything together.

Then it came down to me playing with ideas, trying things out, and them liking things or not liking things. We knew that we had five or six athletes and a certain number of interactive points on the building.

So it was like, okay, let’s break it down into sections. We can start with running or with archery or table tennis. That was the broad structure, which helped a lot because we could say we have 40 interactive points, and therefore roughly eight interactions per sport.

Did you feel you were capable of doing this? How would you advise someone in a similar position?

Yeah, I was 25 when this started. While it’s difficult to give career advice, one thing I hold onto is saying yes to things that you’ve never done before but you kind of feel that you could probably do. If someone said we want you to work on a spaceship I’d say that’s probably a bad idea, but this felt like a much bigger version of things that I’d already done.

There were new things I had to learn, especially working at that scale. For instance, making the system run fast enough and building a backup system. I’d never done a backup system. I had just used my laptop in front of my class or for an installation. So I definitely learning things.

If I have any natural talent it’s for being pretty stubborn about solving problems and sticking at it like a dog with a bone. Knowing that I can, if I work hard at this thing, pull it off. That was the feeling.

 

Arthur Carabott rehearing at the Apple shop with Chagall van den Berg

How did you get in contact with Apple?

I was a resident in the Music Hackspace then and rented a desk in Somerset House. Apple approached Music Hackspace about doing a talk for their Today at Apple series.

I already had a concept for a guerrilla art piece, where the idea was to make a piece of software where I could play music in sync across lots of physical devices. The idea was to go around the Apple store and get a bunch of people to load up this page on as many devices as we could, and then play a big choir piece by treating each device as a voice.

Kind of like a flash mob?

Yeah, sort of. It was inspired by an artist who used to be based in New York called Kyle McDonald, who made a piece called People Staring at Computers. His program would detect faces and then take a photo of them and email it to him. He installed this in the New York Apple stores and got them to send him photos. He ended up being investigated by the Secret Service, who came to his house and took away his computers.

However, for my thing, I wanted to bring a musician into it. Chagall was a very natural choice for the Hackspace. For the music I made an app where people could play with the timbre parameters of a synth, but with a quite playful interface which had faces on it.

How did you end up working with the composer Anna Meredith? You built an app with her, right?

Yes, an augmented reality app. It came about through a conversation with my friend, Marek Bereza, who founded Elf Audio and makes the Koala sampler app. We met up for a coffee and talked about the new AR stuff for iPhones. The SDK had just come to the iPhones and it had this spatial audio component. We were just knocking around ideas of what could be done with it.

I got excited about the fact that it could give people a cheap surround sound system by placing virtual objects in their space. Then you have — for free, or for the cost of an app — a surround sound system.

There was this weekly tea and biscuits event at Somerset House where I saw Anna Meredith and said, ‘Hey, you know, I like your music and I’ve got this idea. Could I show it to you and see what you think?’ So I came to her studio and showed her the prototype and we talked it through. It was good timing because she had her album FIBS in the works. She sent me a few songs and we talked back and forth about what might work for this medium. We settled on the piece Moon Moons, which was going to be one of the singles.

It all came together quite quickly. The objects in it are actual ceramic sculptures that her sister Eleanor made for the album. So I had to teach myself how to do photogrammetry and 3D scan them, before that technology was good on phones.

Augmented reality app build for Anna Merediths album FIBS

You moved to LA. What has that been like?

It was the first time I moved to another country without a leaving date. London’s a great city. I could have stayed, and that would have been the default setting, but I felt like I took myself off the default setting.

So, I took a trip to LA to find work and I was trying to pull every connection I could. Finding people I could present work to, knocking on doors, trying to find people to meet. Then I found this company Output and I was like, ‘Oh, they seem like a really good match’. They’re in LA and they have two job openings. They had one software developer job and one product designer job.

I wrote an email and an application to both of these and a cover letter which said: Look, I’m not this job and I’m not that job. I’m somewhere in the middle. Do you want me to be doing your pixel-perfect UI? That’s not me. Do you want me to be writing optimized audio code? That’s not me either. However, here’s a bunch of my work and you can hear all these things that I can do.

I got nothing. Then I asked Jean Baptise from Music Hackspace if he knew any companies. He wrote an email to Output introducing me and I got a meeting.

I showed my work. The interviewer wrote my name on a notebook and underlined it. When I finished the presentation I looked at his notebook and he hadn’t written anything else. I was like, ‘Okay, that’s a very good sign or very bad sign’. But I got the job.

How do you define what you do?

One of the themes of my career is that has been a double-edged sword is it not being specifically one thing. In the recruitment process what they do is say we have a hole in our ship, and we need someone who can plug it. And very rarely are companies in a place where they think, we could take someone on who’s interesting, but we don’t have an explicit problem for them to solve right now, but we think they could benefit what we’re doing.

The good thing is I find myself doing interesting work without fitting neatly into a box that people can understand. My parents have no idea what I do really.

However, I do have a term I like, but it’s very out of fashion, which is interaction designer. What that means is to play around with interaction, almost like behaviour design.

You can’t do it well without having something to play with and test behaviours with. You can try and simulate it in your head, but generally, you’re limited to what you already know. For instance, you can imagine how a button works in your head, but if you imagine what would happen if I were to control this MIDI parameter using magnets, you can’t know what that’s like until you do it.

What are your thoughts on machine learning and AI? How that will affect music technology?

It’s getting good at doing things. I feel like people will still do music and will keep doing music. I go to a chess club and chess had a boom in popularity, especially during the pandemic. In terms of beating the best human player that has been solved for decades now, but people still play because people want to play chess, and they still play professionally. So it hasn’t killed humans wanting to play chess, but it’s definitely changed the game.

There is now a generation who have grown up playing against AIs and it’s changed how they play, and that’s an interesting dynamic. The interesting thing with music is, it has already been devalued. People barely pay anything for recorded music, but people still go to concerts though concert tickets are more expensive than ever people are willing to pay.

I think the thing that people are mostly interested in with music is the connection, the people, the personal aspect of it. Seeing someone play music, seeing someone very good at an instrument or singing is just amazing. It boosts your spirits. You see this in the world of guitar. A new guitarist comes along and does something and everyone goes, ‘Holy shit, why has no one done that before’?

Then you have artists like Squarepusher and Apex Twin who their own patches to cut up their drum breaks. But they’re still using their own aesthetic choice of what they use. I’m not in the camp that if it’s not 100% played by a human on an instrument, then it’s not real music.

The problem with the word creativity is it has the word create in it. So I think a lot of the focus goes on the creation of materials, whereas a lot of creativity is about listening and the framing of what’s good. It’s not just about creating artefacts. The editorial part is an important part of creativity. Part of what someone like Miles Davis did is to hear the future.

You can find out more about Arthur Carabott on his websiteInstagram, and X

Dom Aversano is a British-American composer, percussionist, and writer. You can discover more of his work on his website, Liner Notes, X, and Instagram.

An interview with Interaction Designer Arthur Carabott Part I

Dom Aversano

If ever evidence was needed of the power of DNA it was demonstrated to me just over a decade ago, when I walked into a room in a dingy industrial estate in Hoxton, East London, to attend one of the early Music Hackspace meet-ups, and much to my surprise saw my cousin, Arthur Carabott, twirling on an office chair, listening to a presentation on ultrasonic sound.

The population in London at that point was roughly 8 million, and there were fewer than 20 people in that room — the odds of us both being there were minuscule. Although we both grew up in an extended family that was very musical, we came to the Music Hackspace by entirely distinct routes, at a time when it was little more than a charming and eccentric fringe group.

Having known Arthur since childhood, it’s not surprising to me that he ended up working in a field that combines artistic and technical skills. He always approached technical problems with a rare tenacity and single-mindedness. Several times I saw Arthur receive a Mechano toy for a birthday or Christmas, only to sit quietly for hours on end working on it until it was finally built.

The Music Hackspace played a significant part in both our formations, so I was curious to know about his experience of this. What surprised me was how much I did not know about Arthur’s journey through music.

What follows is a transcript of that conversation — Part II will follow shortly.

What drew you to music?

There was always music playing in the house. In my family, there was the expectation that you’d play an instrument. I did violin lessons at 7, which I didn’t enjoy and then piano aged 10. I remember being 10 or 11 and there was a group, there were a bunch of us that liked Queen. They are an interesting band because they appeal to kids. They’re theatrical, and some of it is quite clown-like. Then I remember songs like Cotton Eye Joe and singers like Natalie Imbruglia, you know, quite corny music — I’ve definitely got a corny streak. But there was this significant moment one summer when I was probably 11 or so, and I discovered this CD with a symbol on it that was all reflective. It was OK Computer, by Radiohead. That summer made a big musical impact. It’s an album I still listen to.

How does music affect you?

I think music, food, and comedy are quite similar in that when it’s good, there’s no denying it. Of course, with all three, you can probably be a bit pretentious and be like, ‘Oh no, I am enjoying this’ when you’re not. But those are three of my favourite things in the world.

I heard a comedian talking about bombing recently. They said if a musician has an off night, and they get on stage, they don’t play well it’s still music. Whereas if a comedian goes up and they bomb, and no one laughs, it’s not comedy.

You became a very accomplished guitarist. Why did you not choose that as a career?

I went to guitar school and there was a point in my teens when my goal was to become the best guitarist in the world. I remember something Squarepusher had on his website once, where he wrote about being a teenager and giving up on the idea of trying to be like his classmate Guthrie Govan, who is now one of the world’s best guitarists. I resonated with that as there’s a point where you’re like, okay, I’m never gonna do that.

Part of my problem was being hypermobile and therefore prone to injuries, which stopped me from practising as much as I wanted to. Yet, there was still this idea that when I went to Sussex University and studied music informatics with Nick Collins I was going to go there, learn Supercollider, and discover the secrets that Squarepusher and Aphex Twin used. Someone told me they don’t even cut up their drum loops, they’ve got algorithms to do it!

I was actually signed up to do the standard music degree but my friend Alex Churchill said to change it to music informatics as it will change your life. That was a real turning point.

In what way?

What clicked was knowing I enjoyed programming and I wasn’t just going to use music software — I was going to make it.

The course was rooted in academia and experimental art practice rather than commercial things like building plugins. We were looking at interactive music systems and generative music from 2006 – 2009, way before this wave of hype had blown up. We doing some straight-up computer science stuff, and early bits of neural networks and genetic algorithms. Back then we were told, that no one’s really found practical uses for this yet.

We studied people like David Cope, who was an early pioneer who spent decades working on AI music. All these things helped me think outside conventional ways when it came to traditional music tech stuff, and the paradigms of plug-ins, DAWs, and so on.

Today at Apple event where over 100 iPhones and iPads were synchronised for a live performance with singer and producer Chagall

What did you do with this training and how did it help you?

I had no idea what I was going to do afterwards. I was offered a position in the first year of the Queen Mary M.A .T. Media, Art and Technology PhD, but I was a bit burnt out on academia and wanted to do the straight music thing.

I volunteered at The Vortex in London as a sound engineer. I had done paid work at university in Brighton but mostly for teenage punk bands. The Vortex being unpaid worked better because it meant that I only did it for gigs I wanted to see. I was already into Acoustic Ladyland, but there I discovered bands like Polar Bear and got to know people like Seb Rochford and Tom Skinner. I admired their music and also got to interact with and get to know them.

How did you come across Music Hackspace and how did it influence you?

I’d heard there was this thing on Hackney Road. I remember going on a bit of a tour because they would do open evenings and I went with a group of people. It felt like the underground. The best music tech minds in London. A bit of a fringe thing, slightly anarchist and non-mainstream. Music Hackspace was for me mostly about connecting to other people and a community.

What led you to more technical, installation-type work?

I remember seeing Thor Magnussen who had been doing his PhD at Sussex while I was in my undergrad and he taught one of our classes. He was talking about doing an installation and I remember thinking, I don’t really know what an installation is. How do I get one?

Then came the opportunity to work on the 2012 Olympics which came through my sister Juno, and her boyfriend at the time Tim Chave who introduced me to the architects Asif Khan and Pernilla Ohrstedt. I met them and showed them a bunch of like fun things that I’d made, like an app which took Lana Del Rey’s single Video Game and let you remix it in real time. You could type in every word contained in the song, hit enter, and she would sing it, remixed, in time with the beat.

They asked me various technical questions but after the meeting, I didn’t hear anything for a while. Then got a call in December 2011 from Asif. He asked, ‘Can you go to Switzerland next week?’ And I’m like, ‘Wait, am I doing this project? Have I got the job?’ He responded, ‘Look, can you go to Switzerland next week?’ So I said ‘Okay, yeah’.

So then it became official. It was six days a week for six months to get it done in time for the Olympics.

 

The Coca-Cola Beatbox Pavilion from the 2012 London Olympic Games

Part II of this interview will follow shortly. 

You can find out more about Arthur Carabott on his website, Instagram, and X

Dom Aversano is a British-American composer, percussionist, and writer. You can discover more of his work at Liner Notes.

A Q&A with AI regulator Ed Newton-Rex

Dom Aversano

Ed Newton-Rex - photo by Jinnan Wang

In November last year, Ed Newton-Rex, the head of audio at Stability AI, left the company citing a small but significant difference in his philosophy towards training large language models (LLMs). Stability AI was one of several companies that responded to an invitation from the US Copyright Office for comments on generative AI and copyright, submitting an argument that training their models on copyrighted artistic works fell under the definition of fair use: a law which permits the use of copyrighted works for a limited number of purposes, one of which is education. This argument has been pushed by the AI industry more widely, who contest that much like a student who learns to compose music by studying renowned composers, their machine learning algorithms are conducting a similar learning process.

Newton-Rex did not buy the industry’s arguments, and while you can read his full arguments for resigning in his X/Twitter post, central to his argument was the following passage:

(…) since ‘fair use’ wasn’t designed with generative AI in mind — training generative AI models in this way is, to me, wrong. Companies worth billions of dollars are, without permission, training generative AI models on creators’ works, which are then being used to create new content that in many cases can compete with the original works. I don’t see how this can be acceptable in a society that has set up the economics of the creative arts such that creators rely on copyright.

It is important to make clear that Newton-Rex is not a critic of AI; he is an enthusiast who has worked in the machine learning field for more than a decade; his contention is narrowly focused on the ethics surrounding the training of AI models.

Newton-Rex’s response to this was to set up a non-profit called Fairly Trained, which awards certificates to AI companies whose training data they consider ethical.

Their mission statement contains the following passage:

There is a divide emerging between two types of generative AI companies: those who get the consent of training data providers, and those who don’t, claiming they have no legal obligation to do so.

In an attempt to gain a better understanding of Newton-Rex’s thinking on this subject, I conducted a Q&A by email. Perhaps the most revealing admission is that Newton-Rex desires to eliminate his company. What follows is the unedited text. 

Fairly Trained is a non-profit founded by Ed Newton-Rex that award certificates to AI companies who train their models in a manner that is deemed ethical.

Do you think generative artificial intelligence is an accurate description of the technology Fairly Trained certifies?

Yes!

Having worked inside Stability AI and the machine learning community, can you provide a sense of the culture and the degree to which the companies consider artists’ concerns?

I certainly think generative AI companies are aware of and consider artists’ concerns. But I think we need to measure companies by their actions. In my view, if a company trains generative AI models on artists’ work without permission, in order to create a product that can compete with those artists, it doesn’t matter whether or not they’re considering artists’ concerns – through their actions, they’re exploiting artists.

Many LLM companies present a fair use argument that compares machine learning to a student learning. Could you describe why you disagree with this?

I think the fair use argument and the student learning arguments are different.

I don’t think generative AI training falls under the fair use copyright exception because one of the factors that is taken into account when assessing whether a copy is a fair use is the effect of the copy on the potential market for, and value of, the work that is copied. Generative AI involves copying during the training stage, and it’s clear that many generative AI models can and do compete with the work they’re trained on.

I don’t think we should treat machine learning the same as human learning for two reasons. First, AI scales in a way no human can: if you train an AI model on all the production music in the world, that model will be able to replace the demand for pretty much all of that music. No human can do this. Second, humans create within an implicit social contract – they know that people will learn from their work. This is priced in, and has been for hundreds of years. We don’t create work with the understanding that billion-dollar corporations will use it to build products that compete with us. This sits outside of the long-established social contract. 

Do you think that legislators around the world are moving quickly enough to protect the rights of artists?

No. We need legislators to move faster. On current timetables, there is a serious risk that any solutions – such as enforcing existing copyright law, requiring companies to reveal their training data, etc. – will be too late, and these tools will be so widespread that it will be very hard to roll them back.

At Fairly Trained you provide a certification that signifies that a company trains their models on ‘data provided with the consent of its creators’. How do you acquire an accurate and transparent knowledge of the data each company is using?

They share their data with us confidentially.

For Fairly Trained to be successful it must earn people’s trust. What makes your organisation trustworthy?

We are a non-profit, and we have no financial backing from anyone on either side of this debate (or anyone at all, in fact). We have no hidden motives and no vested interests. I hope that makes us trustworthy.

If your ideal legislation existed, would a company like Fairly Trained be necessary? 

No, Fairly Trained would not be necessary. I very much hope to be able to close it down one day!

To learn more about what you have read in this article you can visit the Fairly Trained website or Ed Newton-Rex’s website

Dom Aversano is a British-American composer, percussionist, and writer. You can discover more of his work at the Liner Notes.

Music in the browser or app?

Dom Aversano

As The Bard famously put it, ‘The app, or the browser, that is the question.’

At some point, your inspirational idea for digital music will have to travel from the platonic realm of your thoughts, into either an app or browser. Unless you can luxuriate in doing both, this represents a stark choice. The most appropriate choice depends on weighing up the advantages and disadvantages of both. The graphic above is designed to help categorise what you are creating, thereby providing a better sense of its ideal home.

The most traditional category is recorded music, as it predates the proliferation and miniaturisation of personal computing. In the 20th Century, radio transformed music, and then television transformed it again. In this regard, Spotify and YouTube are quite traditional, as the former imitates radio while the latter mimics TV. This might help explain why Spotify is almost entirely an app, sitting in the background like a radio, and YouTube is most commonly used in the browser, fixing your gaze as if it were a TV. Whether a person is likely to be tethered to a computer or walking around with a phone, may help in deciding between browsers and apps.

Turning to generative music, a successful example of this in the browser is Generative FM, created by Alex Bainter, which hosts more than 50 generative music compositions that you can easily dip into. It is funded by donations, as well as an online course on designing generative systems. The compositions are interesting, varied, and engaging, but as a platform it’s easy to tune out of it. This might be because we are not in the habit of listening to music in the browser without a visual component. The sustainability of this method is also questionable since, despite there still being a good number of daily listeners, the project appears to have been somewhat abandoned, with the last composition having been uploaded in 2021.

Perhaps Generative FM was more suited to an app form, and there are many examples of projects that have chosen this medium. Artists such as Bjork, Brian Eno, and Jean-Michel Jarre have released music as apps. There are obvious benefits to this, such as the fact that an app feels more like a thing than a web page, as well as the commitment that comes from installing an app, especially one you have paid for — in the case of Brian Eno’s generative Reflection app, it comes at the not inconsiderable costs £29.99.

Yet, more than a decade since Bjork released her app Biophilia, the medium is still exceedingly niche and struggling to become established. Bjork has not released any apps since Biophilia, which would have been time-consuming and expensive to create. Despite Bjork’s app not having beckoned in a new digital era for music, this may be a case of a false start rather than a nonstarter. As app building gets easier and more people learn to program, there may be a breakthrough artist who creates a new form of digital music that captures people’s imaginations.

To turn the attention to music-making, and music programming in particular, there is a much clearer migratory pattern. Javascript has allowed programming language to work seamlessly in the browser. In graphical languages, this has led to P5JS superseding Processing. In music programming languages Strudel looks likely to supersede TidalCycles. Of the many ways in which having a programming language in the browser is helpful, one of the greatest is that it allows group workshops to run much more smoothly, removing the tedium and delays caused by faulty software. If you have not yet tried Strudel, it’s worth having a go, as you can get started with music-making in minutes by running and editing some of its patches.

The final category of AI — or large language models — is the hardest to evaluate. Since there is massive investment in this technology, most of the major companies are building their software for both browsers and apps. Given the gold rush mentality, there is a strong incentive to get people to open up a browser and start using the software as quickly as possible. Suno is an example of this, where you can listen to music produced with it instantly. If you sign it only takes a couple of clicks and a prompt to generate a song. However, given the huge running costs of training LLMs, this culture of openness will likely reduce in the coming years, as the companies seek to recuperate their backers’ money.

The question of whether to build something for the browser or an app is not a simple one. As technology offers us increasingly large numbers of possibilities, it becomes more difficult to choose the ideal one. However, the benefit of this huge array of options is that we have the potential to invent new ways of creating and presenting music that may not yet have been imagined, whether that’s in an app or browser.

Please feel free to share your thoughts and insights on creating for the browser or apps in the comment section below!

Dom Aversano is a British-American composer, percussionist, and writer. You can discover more of his work at Liner Notes.

Book Review: Supercollider for the Creative Musician

Dom Aversano

Supercollider for the creative musician.

Several years ago a professor of electronic music at a London University advised me not to learn Supercollider as it was ‘too much of a headache’ and it would be better just to learn Max. I nevertheless took a weekend course, but not long after my enthusiasm for the language petered out. I did not have the time to devote to learning and was put off by Supercollider’s patchy documentation. It felt like a programming language for experienced programmers more than an approachable tool for musicians and composers. So instead I learned Pure Data, working with that until I reached a point where my ideas diverged from anything that resembled patching cords, at which point, I knew I needed to give Supercollider a second chance.

A lot had changed in the ensuing years, and not least of all with the emergence of Eli Fieldsteel’s excellent YouTube tutorials. Eli did for SuperCollider what Daniel Shiffman did for Processing/P5JS by making the language accessible and approachable to someone with no previous programming experience. Just read the comments for Eli’s videos and you’ll find glowing praise for their clarity and organisation. This might not come as a complete surprise as he is an associate professor of composition at the University of Illinois. In addition to his teaching abilities, Eli’s sound design and composition skills are right up there. His tutorial example code involves usable sounds, rather than simply abstract archetypes of various synthesis and sampling techniques. When I heard Eli was publishing a book I was excited to experience his teaching practice through a new medium, and curious to know how he would approach this.

The title of the book ‘SuperCollider for the Creative Musician: A Practical Guide’ does not give a great deal away, and is somewhat tautological. The book is divided into three sections: Fundamentals, Creative Techniques, and Large-Scale Projects.

The Fundamentals section is the best-written introduction to the language yet. The language is broken down into its elements and explained with clarity and precision making it perfectly suited for a beginner, or as a refresher for people who might not have used the language in a while. In a sense, this section represents the concise manual Supercollider has always lacked. For programmers with more experience, it might clarify the basics but not represent any real challenge or introduce new ideas.

The second section, Creative Techniques, is more advanced. Familiar topics like synthesis, sampling, and sequencing, are covered, as well as more neglected topics such as GUI design. There are plenty of diagrams, code examples, and helpful tips that anyone would benefit from to improve their sound design and programming skills. The code is clear, readable, and well-crafted, in a manner that encourages a structured and interactive form of learning and makes for a good reference book. At this point, the book could have dissembled into vagueness and structural incoherence, but it holds together sharply.

The final section, Large-Scale Projects, is the most esoteric and advanced. Its focus is project designs that are event-based, state-based, or live-coded. Here Eli steps into a more philosophical and compositional terrain, showcasing the possibilities that coding environments offer, such as non-linear and generative composition. This short and dense section covers the topics well, providing insights into Eli’s idiosyncratic approach to coding and composition.

Overall, it is an excellent book that every Supercollider should own. It is clearer and more focused than The Supercollider Book, which with multiple authours is fascinating, but makes it less suitable for a beginner. Eli’s book makes the language feel friendlier and more approachable. The ideal would be to own both, but given a choice, I would recommend Eli’s as the best standard introduction.

My one criticism — if it is a criticism at all — is that I was hoping for something more personal to the authour’s style and composing practice, whereas this is perhaps closer to a learning guide or highly-sophisticated manual. Given the aforementioned lack of this in the Supercollider community Eli has done the right thing to opt to plug this hole. However, I hope that this represents the first book in a series in which he delves deeper into Supercollider and his unique approach to composition and sound design.

 

 

Eli Fieldsteel - authour of Supercollider for the Creative Musician
Eli Fieldsteel / authour of Supercollider for the Creative Musician

Click here to order a copy of Supercollider for the Creative Musician: A Practical Guide

Dom Aversano is a British-American composer, percussionist, and writer. You can discover more of his work at the Liner Notes.

An interview with Blockhead creator Chris Penrose

Dom Aversano

A screenshot from Blockhead

Blockhead is an unusual sequencer with an unlikely beginning. In early 2020, as the pandemic struck, Chris Penrose was let go from his job in the graphics industry. After receiving a small settlement package, he combined this with his life savings and used it to develop a music sequencer that operated in a distinctively different manner from anything else available. In October 2023, three years after starting the project, he was working full-time on Blockhead, supporting the project through a Patreon page even though the software was still in alpha mode.

The sequencer has gained a cult following made up of fans as much as users, enthusiastic to approach music-making from a different angle. It is not hard to see why, as in Blockhead everything is easily malleable, interactive, and modulatable. The software works in a cascade-like manner, with automation, instruments, and effects at the top of the sequencer affecting those beneath them. These can be shifted, expanded, and contracted easily.

When I speak to Chris, I encounter someone honest and self-deprecating, all of which I imagine contributes to people’s trust in the project. After all, you don’t find many promotional videos that contain the line ‘Obviously, this is all bullshit’. There is something refreshingly DIY and brave about what he is doing, and I am curious to know more about what motivated him, so arranged to talk with Chris via Zoom to discuss what set him off on this path.

What led you to approach music sequencing from this angle? There must be some quite specific thinking behind it.

I always had this feeling that if you have a canvas and you’re painting, there’s an almost direct cognitive connection between whatever you intend in your mind for this piece of art and the actual actions that you’re performing. You can imagine a line going from the top right to the bottom left of the canvas and there is a connection between this action that you’re taking with a paintbrush pressing against the canvas, moving from top right down to left.

Do you think that your time in the graphics industry helped shape your thinking on music?

When it comes to taking the idea of painting on a canvas and bringing it into the digital world, I think programs like Photoshop have fared very well in maintaining that cognitive mapping between what’s going on in your mind and what’s happening in front of you in the user interface. It’s a pretty close mapping between what’s going on physically with painting on a canvas and what’s going on with the computer screen, keyboard and mouse.

How do you see this compared to audio software?

It doesn’t feel like anything similar is possible in the world of audio. With painting, you can represent the canvas with this two-dimensional grid of pixels that you’re manipulating. With audio, it’s more abstract, as it’s essentially a timeline from one point to another, and how that is represented on the screen never really maps with the mind. Blockhead is an attempt to get a little closer to the kind of cognitive mapping between computer and mind, which I don’t think has ever really existed in audio programs.

Do you think other people feel similarly to you? There’s a lot of enthusiasm for what you doing, which suggests you tapped into something that might have been felt by others.

I have a suspicion that people think about audio and sound in quite different ways. For many the way that digital audio software currently works is very close to the way that they think about sound, and that’s why it works so well for them. They would look at Blockhead and think, well, what’s the point? But I have a suspicion that there’s a whole other group of people who think about audio in a slightly different way and maybe don’t even realise as there has never been a piece of software that represents things this way.

What would you like to achieve with Blockhead? When would you consider it complete?

Part of the reason for Blockhead is completely selfish. I want to make music again but I don’t want to make electronic music because it pains me to use the existing software as I’ve lost patience with it. So I decided to make a piece of audio software that worked the way I wanted it. I don’t want to use Blockhead to make music right now because it’s not done and whenever I try to make music with Blockhead, I’m just like, no, this is not done. My brain fills with reasons why I need to be working on Blockhead rather than working with Blockhead. So the point of Blockhead is just for me to make music again.

Can you describe your approach to music?

The kind of music that I make tends to vary from the start. I rarely make music that is just layers of things. I like adding little moments in the middle of these pieces that are one-off moments. For instance, a half-second filter sweep in one part of the track. To do that in a traditional DAW, you need to add a filter plugin to the track. Then that filter plugin exists for the entire duration of the track, even if you’re just using it for one moment. It’s silly that it has to exist in bypass mode or 0% wet for the entire track, except in this little part where I want it. The same is true of synthesizers. Sometimes I want to write just one note from a synthesizer at one point in time in the track.

Is it possible for you to complete the software yourself?

At the current rate, it’s literally never going to be finished. The original goal with Patreon was to make enough money to pay rent and food. Now I’m in an awkward position where I’m no longer worrying about paying rent, but it’s nowhere near the point of hiring a second developer. So I guess my second goal with funding would be to make enough money to hire a second person. I think one extra developer on the project would make a huge difference.

It is hard not to admire what Chris is doing. It is a giant project, and to have reached the stage that it has with only one person working on it is impressive. Whether the project continues to grow, and whether he can hire other people remains to be seen, but it is a testament to the importance of imagination in software design. What is perhaps most attractive of all, is how it is one person’s clear and undiluted vision of what this software should be, which has resonated with so many people across the world.

If you would like to find out more about the Blockhead or support the project you can visit its Patreon Page.

Dom Aversano is a British-American composer, percussionist, and writer. You can discover more of his work at Liner Notes.

The persistence of misogyny in music technology

Dom Aversano

DJ Isis photographed by Vera de Kok
DJ Isis photographed by Vera de Kok

Last week the British House of Commons Women and Equalities Committee released their report into misogyny in music. It reached a stark and clear conclusion.

In this Report we have focused on improving protections and reporting mechanisms, and on necessary structural and legislative reforms. The main problem at the heart of the music industry is none of these; it is the behaviour of men—and it is almost always men.

Although the report is specific to the United Kingdom many of its findings could apply to other countries. One universal problem is the tendency for some men to view misogyny as a woman’s problem, even though men have greater power to eliminate it. For those of us working with music technology, this needs to be taken to heart, as the field comes out very badly in the report, especially concerning the gender imbalance for music producers, engineers, and songwriters.

In 2022, just 187 women and non-binary people were credited as either producer or engineer on the top 50 streamed tracks in 14 genres, compared to 3,781 men. Of all songwriters and composers who received a royalty in 2020 from their music being streamed, downloaded, broadcast, or performed, only one in six (16.7%) were women.

Music technology education does not fare better.

Participation rates show that music technology courses still show a stark gender imbalance, reflecting the lack of female representation in the production workforce, despite the technology’s increasing importance to modern musicians.

After reading this I was curious to know how Music Hackspace shaped up in this regard. While far from a comprehensive analysis, I decided to count the number of female and male teachers on the Music Hackspace Website and discovered 32 female teachers (35%) and 58 male teachers (65%). This is far from equal, but at least better than the ‘stark gender imbalance’ mentioned in the report. However, until it is equal, it is not good enough.

On a personal note, when writing this blog I try to keep bias and discrimination at the front of my mind, but I am aware I interview more men than women. This is more complicated than simply my intentions. When invited for an interview men have generally been more forthcoming than women and tend to be easier to locate and contact, especially given they often have more prominence within the musical world. It is not hard to imagine why women might be more reluctant to subject themselves to public attention, as they are criticised more than men and taken less seriously. In the government report, many female artists and managers were regularly mistaken for girlfriends.

The misogyny women experience in the public eye was grotesquely demonstrated recently when X/Twitter was flooded with deepfakes porn images of singer Taylor Swift just a few days before this year’s Grammy Awards. One does not have to be a music superstar to be subjected to such abuse. Last year in the Spanish town of Almendralejo more than 28 girls aged from 11 to 17 had AI-generated naked images created of them, with 11 local boys having been involved in the creation and circulation of the images, demonstrating that such threats now exist across all levels of society.

This is to say nothing of the wider patriarchal socio-political forces at work. This year the world will again be subjected to a presidential run by the convicted sex offender Donald Trump, who has bragged about sexually assaulting women and described his daughter as “voluptuous”. He is not alone, with social media-savvy men like Jordan Peterson and Andrew Tate promoting their misogynistic ideas to mass audiences of boys and men. These misogynistic ideas have been demonstrated to be algorithmically amplified by platforms such as TikTok such that Gen Z boys are more likely than Baby Boomers to believe that feminism is harmful.

Music should set a better example and act as a counter-cultural force against these movements. Historically, music has been a driver of social change, as one can create and share ideas with millions of people across the world rapidly. Women’s participation in this artistic activity should be equal to that of men, and for as long as it is not, it is men’s responsibility to help redress the power imbalance. In this respect, I will finish with the same quote from the House of Commons report, which lays out the root of the problem starkly.

The main problem at the heart of the music industry (…) is the behaviour of men—and it is almost always men.

Click here for the full report Misogyny in Music by the British House of Commons Women and Equaliti

Move slow and create things

Dom Aversano

Over Christmas I took a week off, and no sooner had I begun to relax than an inspiring idea came to mind for a generative art piece for an album cover. The algorithm needed to make it was clear in my mind, but I did not want to take precious time away from family and friends to work on it. Then a thought occurred — could I build it quickly using ChatGPT?

I had previously resisted using Large Language Models (LLMs) in my projects for a variety of reasons. Would outsourcing coding gradually deskill me? Whose data was the system trained on and was I participating in their exploitation? Is the environmental effect of using such computationally intense technology justifiable?

Despite my reservations I decided to try it, treating it as an experiment that I could stop at any point. Shortly prior to this, I had read a thought-provoking online comment questioning whether manual coding might seem as peculiar and antiquated to the future as programming in binary does now. Could LLMs help make computers less rigid and fixed, opening up the world of programming to anyone?

While I had previously used ChatGPT to create some simple code for Supercollider, I had been unimpressed by the results. For this project, however, the quality of the code was different. Every prompt returned P5JS code that did exactly what I intended, without the need for clarification. I made precisely what I envisioned in less than 30 minutes. I was astonished. It was not the most advanced program, but neither was it basic.

Despite the success, I felt slightly uneasy. The computer scientist Grady Booch wrote that ‘every line of code represents an ethical and moral decision.’ It is tempting to lose sight of this amid a technological culture steeped in a philosophy of ‘move fast and break things’ and ‘it’s better to ask for forgiveness than permission’. So what specifically felt odd?

I arrived at what I wanted without much of a journey, learning little more than how to clarify my ideas to a machine. This is a stark contrast to the slow and meticulous manner of creation that gradually develops our skills and thinking, which is generally considered quintessential to artistic activity. Furthermore, although the arrival is quicker the destination is not exactly the same, since handcrafted code can offer a representation of a person’s worldview, whereas LLM code is standardised.

However, I am aware that historically many people — not least of all in the Arts and Crafts movement — expressed similar concerns, and one can argue that if machines dramatically reduce labourious work it could free up time for creativity. Removing the technical barrier to entry could allow many more people’s creative ideas to be realised. Yet efficiency is not synonymous with improvement, as anyone who has scanned a QR-code menu at a restaurant can attest.

The idea that LLMs could degrade code is plausible given that they frequently produce poor or unusable code. While they will surely improve, to what degree is unknown. A complicated project built from layers of machine-generated code may create layers of problems: short-term and long-term. Like pollution, its effects might not be obvious until they accumulate and compound over time. If LLMs are trained on LLM-generated code it could have a degradative effect, leading to a Model Collapse.

The ethics of this technology are equally complicated. The current lack of legislation around consent on training LLMs means many people are discovering that their books, music, or code has been used to train a model without their knowledge or permission. Beyond legislating, a promising idea has been proposed by programmer and composer Ed Newton-Rex, who has founded a company called Fairly Trained, which offers to monitor and certify different LLMs, providing transparency on how they were trained.

Finally, while it is hard to find accurate assessments of how much electricity these systems use, some experts predict they could soon consume as much electricity as entire countries, which should not be difficult to imagine given that the Bitcoin blockchain is estimated to consume more electricity than the whole of Argentina.

To return to Grady Booch’s idea that ‘every line of code represents an ethical and moral decision’ one could extend this to every interaction with a computer represents an ethical and moral decision. As the power of computers increases so should our responsibility, but given the rapid increases in computing power, it may be unrealistic to expect our responsibility to keep pace. Taking a step back to reflect does not make one a Luddite, and might be the most technically insightful thing to do. Only from a thoughtful perspective can we hope to understand the deep transformations occurring, and how to harness them to improve the world.

Steve Reich’s exploration of technology through music

Dom Aversano

Photo by Peter Aidu

New York composer Steve Reich did not just participate in the creation of a new style of classical music, he helped establish a new kind of composer. Previously, the word composer evoked an archetype of a quill-wielding child prodigy who had composed several symphonies before adulthood — finding perhaps its purest embodiment in the example of Amadeus Mozart — whereas Reich represented a composer who gradually and determinedly developed their talent in a more relatable manner. At the same age that Mozart was on his deathbed composing his Requiem, Reich was struggling to establish himself in New York, driving taxis to make ends meet.

A key source of Reich’s inspiration was atypical of the classical music tradition, in which composers tended to draw inspiration from nature, religion, romantic love, classical literature, and other art forms; by contrast, Reich’s career was ignited by ideas he derived from electronic machines.

In what is now musical folklore, the young composer set up two tape recorders in his home studio with identical recordings of the Pentecostal preacher Brother Walter proclaiming ‘It’s gonna rain’. Reich pressed play on both machines and to his astonishment found the loops were perfectly synchronised. That initial synchronisation then began to drift as one machine played slightly faster than the other, causing the loops to gradually move out of time, thereby giving rise to a panoply of fascinating acoustic and melodic effects that would be impossible to anticipate or imagine without the use of a machine. The experiment formed the basis for Reich’s famous composition It’s Gonna Rain and established the technique of phasing (I have written a short guide to Reich’s three forms of phasing beneath this article).

While most composers would have considered this a curious home experiment and moved on, Reich, ever the visionary, sensed something deeper that formed the basis for an intense period of musical experimentation lasting almost a decade. In a video explaining the creation of the composition, It’s Gonna Rain, he describes the statistical improbability of the two tape loops having been aligned.

And miraculously, you could say by chance, you could say by divine gift, I would say the latter, but you know I’m not going to argue about that, the sound was exactly in the centre of my head. They were exactly lined up.

To the best of my knowledge, it is the first time in classical music that someone attributed intense or divine musical inspiration to an interaction with an electronic machine. How one interprets the claim of divinity is irrelevant, the significant point is it demonstrates the influence of machines on modern music not simply as a tool, but as a fountain of ideas and profound inspiration.

In a 1970 interview with fellow composer Michael Nyman, Reich described his attitude and approach to the influence of machines on music.

People imitating machines was always considered a sickly trip; I don’t feel that way at all, emotionally (…) the kind of attention that kind of mechanical playing asks for is something we could do with more of, and the “human expressive quality” that is assumed to be innately human is what we could do with less of now.

While phasing became Reich’s signature technique, his philosophy was summed up in a short and fragmentary essay called Music as a Gradual Process. It contained insights into how he perceived his music as a deterministic process, revealed slowly and wholly to the listener.

I don’t know any secrets of structure that you can’t hear. We all listen to the process together since it’s quite audible, and one of the reasons it’s quite audible is because it’s happening extremely gradually.

Despite the clear influence of technology on Reich’s work, there also exists an intense criticism of technology that clearly distinguishes his thinking from any kind of technological utopianism. For instance, Reich has consistently been dismissive of electronic sounds and made the following prediction in 1970.

Electronic music as such will gradually die and be absorbed into the ongoing music of people singing and playing instruments.

His disinterest in electronic sounds remains to this day, and with the exception of the early work Pulse Music (1969), he has never used electronically synthesised sounds. However, this should not be confused with a sweeping rejection of modern technology or a purist attitude towards traditional instruments. Far from it.

Reich was an early adopter of audio samplers, using them to inset short snippets of speech and sounds into his music from the 1980s onwards. A clear demonstration of this can be found in his celebrated work Different Trains (1988). The composition documents the long train journeys Reich took between New York and Los Angeles from 1938 to 1941 when travelling between his divorced parents. He then harrowingly juxtaposed this with the train journeys happening at the same time in Europe, where Jews were being transported to death camps.

For the composition, Reich recorded samples of his governess who accompanied him on these journeys, a retired pullman porter who worked on the same train line, and three holocaust survivors. He transcribed their natural voice melodies and used them to derive melodic material for the string quartet that accompanies the sampled voices. This technique employs technology to draw attention to minute details of the human voice, that are easily missed without this fragmentary and repetitive treatment. As with Reich’s early composition, It’s Gonna Rain, it is a use of technology that emphasises and magnifies the humanity in music, rather than seeking to replace it.

Having trains act as a source of musical and thematic inspiration demonstrates, once again, Reich’s willingness to be inspired by machines, though he was by no means alone in this specific regard. There is a rich 20th-century musical tradition of compositions inspired by trains, including works such as jazz composer Duke Ellington’s Take the A Train, Brazilian composer Heitor Villa Lobos’s The Little Train of the Caipira, and the Finnish composer Kaija Saariaho’s Stilleben.

Reich’s interrogation of technology finally reaches its zenith in his large-scale work Three Tales — an audio-film collaboration with visual artist Beryl Korot. It examines three technologically significant moments of the 20th century: The Hindenburg disaster, the atom bomb testing at Bikini, and the cloning of Dolly the sheep. In Reich’s words, they concern ‘the physical, ethical, and religious nature of the expanding technological environment.’ As with Different Trains, Reich recorded audio samples of speech to help compose the music, this time using the voices of scientists and technologists such as Richard Dawkins, Jaron Lanier, and Marvin Minsky.

These later works have an ominous, somewhat apocalyptic feel, hinting at the possibility of a dehumanised and violent future, yet while maintaining a sense of the beauty and affection humanity contains. Throughout his career, Reich has used technology as both a source of inspiration and a tool for creation in a complicated relationship that is irreducible to sweeping terms like optimistic or pessimistic. Instead, Reich uses music to reflect upon some of the fundamental questions of our age, challenging us to ask ourselves what it means to be human in a hi-tech world.

 


A short guide to three phasing techniques Reich uses

There are three phasing techniques that I detect in Steve Reich’s early music which I will briefly outline.

First is a continuous form of phasing. A clear demonstration of this is in the composition It’s Gonna Rain (1965). With this phasing technique, the phase relationship between the two voices is not measurable in normal musical terms (e.g., ‘16th notes apart’ etc) but exists in a state of continuous change making it difficult to measure at any moment. An additional example of this technique can be heard in the composition Pendulum Music.

The second is a discrete form of phasing. A clear demonstration of this is the composition Clapping Music (1972). With this phasing technique, musicians jump from one exact phase position to another without any intermediary steps, making the move discrete rather than gradual. Since the piece is in a time cycle of 12 there are the same number of possible permutations, each of which is explored in the composition, thereby completing the full phase cycle.

The third is a combination of continuous and discrete phasing. A clear demonstration of this is Piano Phase (1967). With this phasing technique, musicians shift gradually from one position to another, settling in the new position for some time. In Piano Phase one musician plays slightly faster than the other until they reach their new phase position which they settle into for some time before making another gradual shift to another phase position. An additional example of this technique can be heard in the composition Drumming.

Music Hackspace is running an online workshop Making Generative Phase Music with Max/MSP Wednesday January 17th 17:00 GMT 

Dom Aversano is a British-American composer, percussionist, and writer. You can discover more of his work in his Substack publication, Liner Notes.

About
Privacy