Can AI help us make humane and imaginative music?
Dom Aversano
There is a spectrum upon which AI music software exists. On one end are programs which create entire compositions, and on the other are programs that help people create music. In this post I will focus on the latter part of the spectrum, and ask the question, can AI help us compose and produce music in humane and imaginative ways? I will explore this question through a few different AI music tools.
Tone Transfer / Google
For decades the dominance of keyboard interaction has constrained computer music. Keyboards elegantly arrange a large number of notes but limit the control of musical parameters beyond volume and duration. Furthermore, with the idiosyncratic arrangement of a keyboard’s notes, it is hard to work — or even think — outside of the 12-note chromatic scale. Even with the welcome addition of pitch modulation wheels and microtonal pressure-sensitive keyboards such as Roli’s fascinating Seaboard, keyboards still struggle to express the nuanced pitch and amplitude modulations quintessential to many musical cultures.
For this reason, Magenta’s Tone Transfer may represent a potentially revolutionary change in computer music interaction. It allows you to take a sound or melody from one instrument and transform it into a completely different-sounding instrument while preserving the subtleties and nuances of the original performance. A cello melody can be transformed into a trumpet melody, the sound of birdsong into fluttering flute sounds, or a sung melody converted into a number of traditional concert instruments. It feels like the antidote to autotune, a tool that captures the nuance, subtly, and humanity of the voice, while offering the potential to transform it into something quite different.
In practice, the technology falls short of its ambitions. I sang in a melody and transformed it into a flute sound, and while my singing ability is unlikely to threaten the reputation of Ella FitzGerald, the flute melody that emerged sounded like the flautist was drunk. However, given the pace at which machine learning is progressing, one can expect it to be much more sophisticated in the coming years, and I essentially regard this technology as an early prototype.
Google has admirably made the code open source and the musicians who helped train the machine learning algorithms are prominently credited for their work. You can listen to audio snippets of the machine learning process, and hear the instrument evolve in complexity after 1 hour, 3 hours, and 10 hours of learning.
It is not just Google developing this type of technology — groups like Harmonai and Neutone doing similar things and any one of them stands to transform computer music interaction, by anchoring us back into the most universal instrument, the human voice.
Mastering / LANDR
Although understanding how mastering works is relatively straightforward, understanding how a mastering engineer perceives music and uses their technology is far from simple since there is as much art as there is science to their craft. Therefore, is this a process that can be devolved to AI?
That is the assumption behind LANDR’s online mastering service which allows you to upload a finished track for mastering. Once it is processed, you are given the option to choose from three style settings (Warm, Balanced, Open) and three levels of loudness (Low, Medium, High), with a master/original toggle to compare the changes made.
I uploaded a recent composition to test it. The result was an improvement on the unmastered track, but the limited options to modify it gave the feeling of a one-size-fits-all approach, inadequate for those who intend to carefully shape their musical creations at every stage of production. However, this might not be an issue for people on lower-budget projects, or those who intend to simply and quickly improve their tracks for quick release.
In a desire to understand the AI technology I searched for more precise details, and while the company says that ‘AI isn’t just a buzzword for us’ I could only find a quote that does little to describe how the technology actually works.
Our legendary, patented mastering algorithm thoroughly analyzes tracks and customizes the processing to create results that sound incredible on any speaker.
While LANDR’s tool is useful for quick and cheap mastering, it feels constrained and artistically unrewarding if you want something more specific. The interface also feels like it limits the potential of the technology. Why not allow text prompts such as: “cut the low-end rumble, brighten the high end, and apply some subtle vintage reverb and limiting”.
Fastverb / Focusrite
Unlike mastering, reverb is an effect rather than a general skill or profession, making it potentially simpler to devolve aspects of it to AI. Focusrite’s Fastverb reverb effect uses AI to analyse your audio before prescribing certain settings for you based on this, which you can then go on to tweak. The company is vague about how their AI technology works, simply stating.
FAST Verb’s AI is trained on over half a million real samples, so you’ll never need to use presets again.
I use the plugin on a recent composition. The results were subtle but an improvement. I adjusted some of the settings and it sounded better. Overall, I had the impression of a tasteful reverb that would work with many styles of music.
Did the AI help significantly in arriving at the desired effect? It is hard to say. I would assume for someone with very limited experience using such tools, yes, but without someone confident with an effect, I doubt it saves much time at all.
I am aware however there is the potential for snobbery here. After all, if a podcaster can add a decent reverb to their show or a guitarist can add some presence to their recording easily, that’s no bad thing. They can if they want go on to learn more about these effects and fine-tune them themselves. For this reason purpose, it represents a useful tool.
Overview
LANDR’s Mastering service and Focusrite’s Fastverb are professional tools that I hope readers of this article will be tempted to try. However, while there is clearly automation at work, how the AI technology works is unclear. If the term AI is used to market tools, there should be clarification of what exactly it is — otherwise one might as well just write ‘digital magic’. By contrast, Google’s Tone Transfer have made their code open source, as well as describing in detail how they use machine learning, and the people involved in training the models.
I expect that the tools that attempt to speed up or improve existing processes, such as mastering and applying reverb, will have the effect of lowering the barrier to entry into audio engineering, but I have yet to see evidence it will improve it. In fact, it could degrade and homogenise audio engineering by encouraging people to work faster but with less skill and care.
By contrast, the machine learning algorithms that Googe, Harmonai, Neutone, and others are working on, could create meaningful change. They are not mature technologies, but there is the seed of something profound in them. The ability to completely transform the sounds of music while preserving the performance and the potential to bring the voice to the forefront of computer music could prove to be genuinely revolutionary.
What follows from the collapse of NFTs?
Dom Aversano
Almost a quarter of a century after Napster fired a torpedo into the record industry one might have expected stability to have returned, but the turmoil continues well into the new century without any signs of resolution.
The story is familiar. MP3 collections never felt like record collections, making them ripe to be superseded by full-catalogue music streaming. Streaming is unprofitable for the companies selling it and unsustainable for the musicians on it, so in a bid to save themselves, not music, the platforms are now transforming into rivers of algorithmically recommended muzak. Ironically, the oldest medium is in the healthiest state, vinyl, and while it is inspiring to know people still go out and buy records, it does not help solve the problem of digital music.
Given this context, it was always tempting to see NFTs — or non-fungible tokens — as the saviour of digital music. But with Sam Bankman-Fried now standing on trial and 95% of NFTs estimated to be worthless we should be asking, what went wrong?
It is beyond the scope of this article to explain what NFTs are — which has been done well elsewhere — but what can be said is the heavy nomenclature they carry can make it feel impenetrable and confusing: you have blockchain, minting, wallets, cryptocurrency, drops, Bitcoin, Metaverse, Web 3, smart contracts etc. The time required to make sense of this — much like an NFT — is a luxury few can afford, providing a wall of obscurantism that imbues the culture with an aura of mystique and intellectualism.
My experience took me down a winding path. Initially, I found NFTs interesting, as they seemed like an innovative method for digital ownership that could help fund the creation of new music and provide fans with a strong connection to their favourite artists, but as my research accumulated their appeal steadily diminished. A combination of too-good-to-be-true promises and scammy behaviour made it seem murky, if not at times actively sinister.
While I am not closed off to the possibility of something valuable emerging from this world (for instance, smart contracts seem genuinely interesting) based on the evidence, NFTs were always doomed to fail.
Here is why.
The torrent of terminology in this culture makes it easy to be blinded by the science and lose sight of the obvious — for instance, cryptocurrencies, despite the name, are not currencies. There is barely a thing on Earth you can buy with crypto. It is actually an asset untethered to economic activity, or simpler yet, an elaborate gambling token. Just as nobody wants to appear a philistine for not appreciating a certain art form, nobody wants to feel like a Luddite for not understanding a particular technology, but spend your evenings and weekends dispassionately breaking down the terminology and you’ll find little of substance remains.
Most people try to understand cryptocurrency in a purely technical sense and ignore the sociological of its emergence. Bitcoin arose shortly after the 2008 financial crisis when mistrust of banking was at an all-time high. At this time having a so-called currency circumventing banks was music to people’s ears, and the Hollywood superhero manner in which Bitcoin entered the world through a mysterious unknown figure called Satoshi Nakamoto only added to its anarcho-utopian appeal.
Blockchain sounds cooler than it is. Some blockchains create huge environmental damage, have very long transaction times, and are vulnerable to privacy breaches and theft. If you lose your password to your digital wallet or if it falls into someone else’s hands you may lose everything, without any recourse to institutional support or insurance. Most concerning of all, far from being a tool for honesty and transparency, cryptocurrency is regularly used by organised criminals as a tool for money laundering. For these reasons, blockchain has been referred to at various points as ‘a solution in search of a problem’.
Experts have much less faith in cryptocurrency than the public. An economist who famously predicted the 2007–08 subprime mortgage crisis, Nouriel Roubini, called crypto ‘a scam’ and a ‘Ponzi scheme’ that preys on young people, people on lower income, and minorities, and advises people to ‘stay away’, referring to those who run the industry as ‘crooks’ that ‘literally belong in jail’.
Even if none of the above really dents your belief in the validity of cryptocurrencies/NFTs/blockchains, there is a gaping flaw that is impossible to ignore.
NFTs have no intrinsic value.
I can put a photo of the Taj Mahal on a blockchain and link it to you, but that doesn’t mean you own a brick of it.
Writer and programmer Stephen Dhiel, who is a vociferous critic of cryptocurrencies, offered the following analogy about NFTs in a Twitter/X thread.
There is one comparable market to NFTs: The Star Naming Market (…) Back in the 90s some entrepreneurs found you could convince the public to buy “rights” to name yet-unnamed stars after their loved ones by selling entries in an unofficial register (…) You’d buy the “rights” to a name [sic] the star and they’d send you a piece of paper claiming that you were now the owner of said star. Nothing was actually done in this transaction, you simply paid someone to update a register about a ball of plasma millions of light years away. (…) NFTs are the evolution of this grift in a more convoluted form. Instead of allegedly buying a star, you’re allegedly buying a JPEG from an artist. Except you’re not buying the image, you’re buying a digitally signed URL to the image.
With NFTs now largely worthless, it’s hard to argue with Dhiel’s analysis. So where does this leave us?
Few genuinely innovative ideas remain, but a company called JKBX has proposed that people can buy royalty shares of their favourite musicians’ songs. The problem is, even if it worked, would it be healthy to have fans treating their favourite artists’ songs as investments? Would listening to All You Need is Love feel the same if you were waiting for your share of a royalty payment to come through? Is turning music into a weird stock market for royalties really the best thing we can dream up?
After nearly a quarter of a century of unsuccessfully trying to resurrect the 20th-century music recording industry for the 21st-century, perhaps it is time to ask, was this ever the right goal? MP3s, streaming, and NFTs, did not balance the boat, which still rocks about aimlessly on stormy seas.
Perhaps the original goal was never ambitious or imaginative enough, after all, why resurrect an old method of distributing music when you could create a new one? NFTs were attractive to people for many reasons, but a major one was they promised a new internet culture — Web 3, metaverse etc. — that could offer ordinary people economic dignity. That people found this appealing is grounds for hope, as it demonstrates there is an appetite for a radical departure from the stagnant and centralised world of the social media empires.
The question that remains is: can we imagine it and build it? And if not now, when? If music wishes to remain a relevant art form, it can’t afford another quarter-century of floundering.
Do you have thoughts on what you have read? If so, please leave your comments below.
Further information on cryptocurrency/NFTs/blockchain
The Missing Crypto Queen — Podcast by investigative journalist Jamie Bartlett
The Case Against Crypto — Essay by programmer Stephen Diehl
Crypto is dead — Debate between Yanis Varoufakis & Viktor Tábori
Visualizing Sound with TouchDesigner
Visualizing Sound with TouchDesigner
What do recent trends in generative art mean for music?
Dom Aversano
In his provocative and fascinating book, Noise, the French musicologist and economist Jacques Attali wrote the following about the prophetic power of music.
Music is prophecy. Its styles and economic organization are ahead of the rest of society because it explores, much faster than material reality can, the entire range of possibilities in a given code.
For a long time I considered this flattering statement about music to be true, but the more I learned about visual arts the more I saw that at various points in history it seemed to push ahead of music. Decades before Brian Eno used the term Generative Music the term Generative Art was being used, which does not mean there were no generative processes in music before then — there certainly were — but the terminology helped articulate a theoretical framework through which the art could be understood and developed.
In the last few years a big shift occurred in visual generative arts, somewhat obscured by the huge attention given to advances in machine learning and large language models, but worthy of examination for anyone interested in digital arts.
This innovation was fuelled by NFTs or non-fungible tokens (you can read more about them here). Putting aside the controversial ethical and technological aspects of cryptocurrency and NFTs — which I hope to cover in a future post — the economy it produced provided many generative artists with a living, during which the technical aspects of the art grew more sophisticated and publications like Right Click Save emerged to document the movement. This year the NFT economy fundamentally collapsed, making for an opportune moment to review what happened during its boom, and its relevance to musicians and composers.
In 2021 the generative artist and writer Tyler Hobbs wrote an important essay called The Rise of Long-Form Generative Art, which helps make sense of the recent changes to generative art. Within it, he describes two broad categories of generative art: short-form and long-form.
Generative art has traditionally favoured short-form, which he describes as follows.
First, there was almost always a “curation” step. The artist could generate as many outputs as they pleased and then filter those down to a small set of favorites. Only this curated set of output would be presented to the public.
The result of this is often small collections ranging from a single image to about a dozen. The artist is still largely in control, creating art in a manner that does not radically deviate from tradition.
In a jargon-dense paragraph Hobbs describes long-form art, with the last sentence being especially significant.
The artist creates a generative script (e.g. Fidenza) that is written to the Ethereum blockchain, making it permanent, immutable, and verifiable. Next, the artist specifies how many iterations will be available to be minted by the script. A typical choice is in the 500 to 1000 range. When a collector mints an iteration (i.e. they make a purchase), the script is run to generate a new output, and that output is wrapped in an NFT and transferred directly to the collector. Nobody, including the collector, the platform, or the artist, knows precisely what will be generated when the script is run, so the full range of outputs is a surprise to everyone.
This constitutes a fundamental change. The artist no longer directly creates the art but an algorithm to create art, renouncing control over what the algorithm produces from the moment it is published. It is a significant shift in the relationship between the artist, the artwork, and the audience, that calls into question the definition of art.
Within the same essay, Hobbs describes a concept for analysing long-form art that he calls “output space”.
Fundamentally, with long-form, collectors and viewers become much more familiar with the “output space” of the program. In other words, they have a clear idea of exactly what the program is capable of generating, and how likely it is to generate one output versus another. This was not the case with short-form works, where the output space was either very narrow (sometimes singular) or cherry-picked for the best highlights.
This concept of an algorithm’s spectrum of variation is valuable. After all, scale without meaningful variation is decorated repetition. Paradoxically — in a superficial sense at least — algorithms can simultaneously have infinite permutations and a great sense of predictability and monotony. The notion of output space is perhaps a more accurate way to evaluate generative works than their literal number of iterations or other quantifiable measures.
In reflecting on how the concept of long-form might exist in music, two works sprung to mind.
The first is Jem Finer’s Longplayer. The composition was created with the intention of being played for a millennium and is currently installed at Trinity Buoy Wharf in East London. For almost a year I worked part-time at the Longplayer and had the opportunity to listen to the installation for hours on end. It struck me as a novel and ambitious idea with an attractive sound, but I was not able to detect any noticeable variation or development from one hour, week, or month to the next. To use the language of generative art, its output space felt narrow — at least over a duration that is short in comparison to its intended length.
I should point out this might well miss the point of the composition, designed as it is to make one reflect on vast time scales and to invite intergenerational collaboration.
The second example is Brian Eno’s composition Reflections, released both as an app and a series of musical excerpts. Eno describes it using the metaphor of a river.
It’s always the same river, but it’s always changing.
Having discovered this piece relatively recently I have not listened sufficiently to have an opinion, although there are many glowing online reviews about its ability to transform and change mood, and people listening to it extensively.
The requirement of extensive listening highlights an important difference between music and visual art. It is much quicker to scan over a collection of 1,000 images than to spend, hours, weeks, or even months attentively listening to an algorithm unfold, which helps explain why long-form generative art is currently more popular than long-form generative music, though there may be another reason too.
You might ask why has long-form generative art become so popular recently as it is by no means a new concept. In 1949 the abstract artist Joseph Albers began a 25-year project working on an iconic and influential series called A Homage to Squares, comprising over 100 paintings that combine squares of different sizes and colours in a variety of ways. By contrast, you now have artists developing algorithms in a couple of months to create ten times more images than Albers’s series. Is this meaningful art, or a hi-tech example of the philosophical more is more?
While it might be cynical to reduce an art movement to a single economic factor, it would also be naive, to ignore it. A significant number of people were made wealthy in a very short time by the boom of NFTs, and the supply and demand relationship was transformed as digital art can be produced with dramatically less time and cost than traditional art. Huge demand could be met with huge supply with little more effort than adding a couple of zeros to the number of iterations.
The rates that certain pieces sold for at the height of the hype are astonishing. A single image in a collection of Cellular Automaton sold for 1,000,000 Tezos (£537,000). I do not know whether this was motivated by some murky financial practice or credulity on the part of the collector, but to have a single work in a collection of 1,000 — composed from an 80-year-old mathematical concept — selling for such a huge price indicates that money significantly shapes the culture. Despite the rot, some art that emerged from this movement is genuinely inspiring and thought-provoking.
Take Dreaming of Le Corbusier, by the Norwegian artist Andreas Rau. It is an impressive algorithm that generates a new ‘architectural’ abstract artwork each time you click on it. Some works have the appearance of having been designed deliberately, with the consistent quality of the compositions being remarkable.
There is also the work of Rich Poole which is featured in this piece. The series feels musical in its composition — reminiscent of a beautiful music sequencer, where colour, height, and length correspond to some musical parameters. The owners of the NFTs choose from iterations of an algorithm what work they would like, meaning the series is ‘collector-curated’.
What happens to generative art now that NFTs have collapsed? That is anyone’s guess. It is hard to envision the sudden emergence of an economy remotely comparable to the over-hyped NFT market. Yet there has been a shift and a new potential glanced at, not just by the artists involved, but by all of us.
The artworks featured in this article are shared with kind permission by the artist Rich Poole. You can view his entire series for Manu here.
How to design a music installation – an interview with Tim Murray-Browne (part 2)
Dom Aversano
How to design a music installation - an interview with Tim Murray-Browne (part 2)
In the first part of this interview, artist Tim Murray-Browne discussed his approach to creating interactive installations, and the importance of allowing space for the agency of the audience with a philosophy that blurs the traditional artist/audience dichotomy in favour of a larger-scale collaboration.
In the second part of this interview, we discuss how artificial intelligence and generative processes could influence music in the near future and the potential social and political implications of this, before returning to the practical matters of advice on how to build an interactive music installation and get it seen and heard.
I recently interviewed the composer and programmer Robert Thomas who envisions a future in which music behaves in a more responsive and indeterminate manner, more resemblant to software than the wax cylinder recording that helped define 20th-century music. In this scenario, fixed recording could become obsolete. Is this how you see the future?
I think the concept of the recorded song is here to stay. In the same way, I think the idea of the gig and concert is here to stay. There are other things being added on top and it may become less and less relevant as time goes on. Just in the way that buying singles has become less relevant even though we still listen to songs.
I think the most important thing is having a sense of personal connection and ownership. This comes back to agency, where I feel I’m expressing myself through the relationship with this music or belonging to a particular group or community. What I think a lot of musicians and people who make interactive music can get wrong is since they take such joy and pleasure in being creatively expressive, they think they can somehow give that joy to someone else without figuring out how to give them some kind of personal ownership of what they’re doing.
As musicians it’s tempting to think we can make a track and then create an interactive version, and that someone’s going to listen to that interactive version of my track and remix it live or change aspects of it, and have this personalised experience that it is going to be even better because they had creative agency over it.
I think there’s a problem with that because you’re asking people to do some of the creative work but without the sense of authorship or ownership. I may be wrong about this because in video games you definitely come as an audience and explore the game and develop skill and a personal style that gives you a really personal connection to it. But games and music are very different things. Games have measurable goals to progress through, and often with metrics. Music isn’t like that. Music is like an expanse of openness. There isn’t an aim to make the perfect music. You can’t say this music is 85% good.
How do you see the future?
I agree with Robert in some sense, but where I think we’re going to see the song decline in relevance has less to do with artists creating interactive versions of their work and more to do with people using AI to completely appropriate and remix existing musical works. When those tools become very quick and easy to use I think we will see the song transform into a meme space instead. I don’t see any way to avoid that. I think there will be resistance, but it is inevitable.
In the AI space, there are some artists who are seeing this coming and trying to make the most of it. So instead of trying to stop people from using AI to rip off their work, they’re trying to get a cut of it. Like say, okay you can use my voice but you’ll give me royalties. I’ve done all of this work to make this voice, it’s become like a kind of recognizable cultural asset and I know I’m going to lose control of it, but I want some royalties and to own the quality of this vocal timbre
Is there a risk in deskilling, or even populism, in a future where anyone can make profound changes to another person’s creative work? The original intention of copyright law was to protect artists’ work from falling out of their hands financially and aesthetically. The supposed democratisation of journalism has largely defunded and deskilled an important profession and created an economy for much less skilled influencers and provocateurs. Might not the same happen to music?
The question of democratisation is problematic. For instance, democracy is good, but there are consequences when you democratise the means of production, particularly in the arts where a big part of what we’re doing is essentially showing off. Once the means of production are democratised, then those who have invested in the skills previously needed lose that capacity to define themselves through them. Instead, everyone can do everything and for this short while, because we’re used to these things being scarce, it suddenly seems like we’ve all become richer. Then pretty soon, we find we’re all in a very crowded room trying to shout louder and louder. It’s like we were in a gig and we took away the stage and now we’re all expecting to have the same status that the musician on the stage had.
I can see your concerns with that, but when it comes to music transforming from being a produced thing to being very quickly made with AI tools by people who aren’t professional. If you’re a professional musician there will still be winners and losers, and those winners and losers will in part be those who are good at using the tools. There will be those with some kind of artistic vision. And there’ll be those who are good at social media and networking, and good at understanding how to make things go viral.
It’s not that different from how music is now. It takes more than musical talent to become a successful artist as a musician, you’ve got to build relationships with your fans, you have to do all of these other things which maybe you could get away with not doing so much in the past.
Let’s return to the original theme of what makes for a good installation. What advice would you give to someone in the same position now that you were in just over a decade ago when starting Cave of Sounds?
In 2012 when we started building Cave of Sounds Music Hackspace was a place for people to build things. This was fundamental for me. People there were making software and hardware and there was this sort of default attitude of ‘we built it, now we’re going to show somebody’. We’re going to get up in the front of the room and I’m going to talk to you about this thing, and maybe I’ll play some music on it.
I find the term installation problematic because it comes from this world of the art gallery and of having a space and doing something inside the space where it can’t necessarily just be reduced to a sculpture or something. Whereas, for me, it was just a useful word to describe a musical device where the audience is going to be actively interacting with it, rather than sitting down and watching a professional interact with it. So that shift from a musician on a stage to an audience participating in the work.
I don’t think it necessarily has to begin with a space. It needs a curiosity of interaction. Maybe I’m just projecting what I feel, but what I observed at Music Hackspace is people taking so much enjoyment in building things, and less time spent performing them. Some people really want to get up and perform as musicians. Some people really want to build stuff for the pleasure of building.
How do you get an installation out into the world?
How to get exhibited is still an ongoing mystery to me, but I will say that having past work that has succeeded means people are more likely to accept new work based on a diagram and description. Generally, having a video of a piece makes it much more likely for people to want to show it. The main place things are shown is in festivals, more than galleries or museums. Getting work into a festival is a question of practical logistics: How many people are going to experience it and how much space and resources does it demand? And then festivals tend to conform to bigger trends – sometimes a bit too much I think as then they end up all showing quite similar works. When we made Cave of Sounds, DIY hacker culture and its connection to grassroots activism was in the air. Today, the focus is the environment, decolonisation, and social justice. Tomorrow there will be other things.
Then, there’s a lot of graft, and a lot of that graft is much easier when you’re younger than when you’re older. I don’t think I could go through the Cave of Sounds process today like I did back then. I’m very happy I did it back then.
What specifically about the Cave of Sounds do you think made it work?
The first shocking success of Cave of Sounds is that when we built it we had like a team of eight, and I had a very small fee because I was doing this artist residency, but everyone else was a volunteer on that project or collaborating artists, but unpaid. And we worked together for eight months to bring it together.
A lot of people came to the first meeting but from the second meeting, the people who turned up from that point forward were the eight people making the work who stuck through to the end. I think there’s something remarkable about that. Something about the core idea of the work really resonated with those people, and I think we got really lucky with them. And there was a community that they were embedded in as well. But the fact that everyone might made it to the end, just like shows that there was something kind of magical in the nature of the work and the context of that combination of people.
So a work like Cave Sounds was possible because we had a lot of people who were very passionate, and we had a diversity of skills, but we also had like a bit of an institutional name behind us. We had a small budget as well, but the budget was very small, and most of the budget did not pay for the work. The budget covered some of the materials, really, but a significant amount of labour went into that piece, and it came from people working for passion.
Do you have a dream project or a desire for something you would like to do in the future?
For the past few years I’ve been exploring how to use AI to interpret the moving body so that I can create physical interaction without introducing any assumptions about what kind of movement the body can make. So if I’m making an instrument by mapping movement sensors to sound, I’m not thinking ‘OK this kind of hand movement should make that kind of sound’ but instead training an AI on many hours of sensor data where I’m just moving in my own natural way and asking it ‘What are the most significant movements here?’
I’m slightly obsessed with this process. It’s giving me a completely different feeling when I interact with the machine, like my actions are no longer mediated by the hand of an interaction designer. Of course, I’m still there as a designer, but it’s like I’m designing an open space for someone rather than boxes of tools. I think there’s something profoundly political about this shift, and I’m drawn to that because it reveals a way of applying AI to liberate people to be individually themselves, rather than using it to make existing systems even more efficient at being controlling and manipulative which seems to be the main AI risk I think we’re facing right now. I could go on more as well – moving from the symbolic to the embodied, from the rational to the intuitive. Computers before AI were like humans with only the left side of the brain. I think they make humans lose touch with their embodied nature. AI adds in the right side, and some of the most exciting shifts I think will be in how we interact with computers as much as what those computers can do autonomously.
So far, I’ve been exploring this with dancers, having them control sounds in real-time but still being able to dance as they dance rather than dancing like they’re trapped inside a land of invisible switches and trigger zones. And in my latest interactive installation Self Absorbed I’ve been using it to explore the latent space of other AI models, so people can morph through different images by moving their bodies. But the dream project is to expand this into a larger multi-person space, a combined virtual and physical realm that lets people influence their surroundings in all kinds of inexplicable ways by using the body. I want to make this and see how far people can feel a sense of connection with each other through full-body interfaces that are too complicated to understand rationally but are so rich and sensitive to the body that you can still find ways to express yourself.
Cave of Sounds was created by Tim Murray-Browne, Dom Aversano, Sus Garcia, Wallace Hobbes, Daniel Lopez, Tadeo Sendon, Panagiotis Tigas, and Kacper Ziemianin with support from Music Hackspace, Sound and Music, Esmée Fairbairne Foundation, Arts Council England and British Council.
To find out more about Tim Murray-Browne you can visit his website or follow him on Substack, Instagram, Mastodon, or X.
Complete Guide to Max for Live
Max for Live is the most complete software for audio/visual professionals to build patches suitable for installation, experimental music, live lighting, and the web. If you can imagine it, you can build it.
Ask Me Anything about Max
In this Ask Me Anything session, Massi Cerioni answers questions about Max MSP gen~ and rnbo~, which bridge the gap between Max prototyping and development, with powerful code export features.
How to design a music installation – an interview with Tim Murray-Browne (part 1)
Dom Aversano
How to design a music installation - an interview with Tim Murray-Browne (part 1)
I met artist and coder Tim Murray-Browne just over a decade ago, briefly after he was made artist in residence for Music Hackspace. Tall, thin, with a deep yet softly-spoken voice, he stood up and gave a presentation to an audience of programmers, academics, musicians, and builders, in a room buzzing with anticipation. The setting was a dingy studio in Hoxton, East London, prior to the full-on gentrification of the artistic neighbourhood.
Tim’s idea for a project was bold: He had no idea. Or to be more precise, his idea was to have no idea. Instead, the idea would emerge from a group. There were quizzical looks in the audience and questions to confirm indeed the idea was to have no idea. For an artistically audacious idea, this was a good audience, comprised as it was of open-minded, radical, and burningly curious people. By the meeting’s end an unspoken consensus of ‘let’s give this a go’ seemed to have quietly been reached.
Tim’s faith in his concept was ultimately vindicated since the installation that emerged from this process, Cave of Sounds, still tours to this day. Created by a core group of eight people — myself one of them — it has managed to stay relevant amid a slew of socio-political and technological changes. As an artist, Tim has continued to make installations, many focusing on dance, movement, and the human body, as well as more recently, AI.
I wanted to reflect back on this last decade, to see what had been learned, what had changed, what the future might hold, and above all else, how one goes about creating an installation.
What do you think are the most important things to consider when building an interactive installation?
First, you need some kind of development over time. I used to say narrative though I’m not sure if that is the right word anymore, but something needs to emerge within that musical experience. A pattern or structure that grows. Let’s say someone arrives by themselves, maybe alone in a room, and is confronted with something physical, material, or technological, and the journey to discover what patterns emerge has begun. Even though an installation is not considered a narrative form, any interaction is always temporal.
Second, has to do with agency. It’s very tempting as an artist to create a work and have figured out exactly what experience you want your audience to have and to think that that’s going to be an interactive experience even though you’ve already decided it. Then you spend all your time locking down everything that could happen in the space to make sure the experience you envisaged happens. I think if you do this you may as well have made a non-interactive artwork, as I believe the power of interactivity in art lies in the receiver having agency over what unfolds.
Therefore, I think the question of agency in music is fundamental. When we are in the audience watching music a lot of what we get out of it is witnessing someone express themselves skillfully. Take virtuosity, that comes down to witnessing someone have agency in a space and really do something with it.
How exactly do you think about agency in relation to installations?
In an interactive installation, it’s important to consider the agency of the person coming in. You want to ask, how much freedom are we going to give this person? How broad is the span of possible outcomes? If we’re doing something with rhythm and step sequencing are we going to quantise those rhythms so everything sounds like a techno track? Or are we going to rely on the person’s own sense of rhythm and allow them to decide whether to make it sound like a techno track or not?
It all comes down to the question of what is the point of it being interactive. While it is important to have some things be controllable, a lot of the pleasure and fun of interactive stuff is allowing for the unexpected, and therefore I find the best approach when building an installation is to get it in front of unknown people as soon as possible. Being open to the unexpected does not mean you cannot fail. An important reason for getting a work in front of fresh people is to understand how far they are getting into the work. If they don’t understand how to affect and influence the work then they don’t have any agency, and there won’t be any sense of emergence.
Can you describe music in your childhood? You say you sang in choirs from the age of six to twelve. What was your experience of that?
At the time it burnt me out a little but I’m very thankful for it today. It was very much tied to an institution. It was very institutional music and it was obligatory. I was singing in two to three masses a week and learning piano and percussion. I stopped when I was about 13. I had a few changes in life, we moved country for a little bit and I went to a totally different kind of school and environment. It wasn’t until a few years later that I picked up the piano again, and only really in the last couple of years have I reconnected with my voice.
Your PhD seemed to be a turning point for you and a point of re-entry into music. Can you describe your PhD, and how that influenced your life?
I began doing a PhD looking at generative music, and as I was trying to figure out what the PhD would be I had an opportunity to do a sound installation in these underground vaults in London Bridge Station with a random bunch of people in my research group. They were doing an installation there and someone had some proximity sensors I could use. There was an artist who had some projections which were going up and I made a generative soundscape for it. Being in the space and seeing the impact of that work in a spatial context really shifted my focus. I felt quite strongly that I wanted to make installations rather than just music, and I reoriented my PhD to figure out how to make it about that. I was also confronted with the gulf of expectation and reality in interactive art. I thought the interactivity was too obvious if anything, but then as I sat and watched people enter the space, most did not even realise the piece was interactive.
How do these questions sit with you today?
From an academic perspective, it was a really terrible idea because a PhD is supposed to be quite focused, and I was questioning how can you make interactive music more captivating. I had this sense in my head of what an interactive music experience could be, and it was as immersive, durational and gripping as a musical experience. Nearly every interactive sound work I was finding ended up being quite a brief experience – you kind of just work out all the things you can do and then you’re done.
I saw this pattern in my own work too. My experience in making interactive sound works was much more limited back then, but I saw a common pattern of taking processes from recorded music and making it interactive. My approach was to ask ‘Well what is music really? why do we like it?’ and all kinds of answers come up about emerging structures, belonging, and self-expression, so then the question was how can we create interactive works that embody those qualities within the interactivity itself.
What it left me with was not such a clear pathway into academia, because I hadn’t arrived at some clear and completed research finding, but what I had done was immersed myself so fundamentally in trying to answer this question, how can I make captivating interactive music experiences?f
What did you find?
On the question of interaction with technology, I think the most fundamental quality of technology is interaction, human-computer interaction. How is it affecting us? How are we affecting it? How does that ongoing relationship develop?
There is so much within those questions, and yet interactivity is often just tacked on to an existing artwork or introduced in a conventional way because that is how things are done. In fact, the way you do interactivity says a lot about who you are and how you see the world. How you design interaction is similar to how you make music, there are many ways, and each has a political interpretation that can be valuable in different contexts.
Who has influenced you in this respect?
The biggest influence on me at the point where I’d finished my PhD and commenced Cave of Sounds was the book Musicking by Christopher Small.
The shift in mindset goes from thinking that music is something being done by musicians on a stage and being received by everyone else around them, to being a collective act that everybody’s participating in together, and that if there weren’t an audience there to receive it the musician couldn’t be participating in the same music.
What I found informative is to take a relativist view on different musical cultures. Whether it is a rock concert, classical concert, folk session, or jazz jam, you can think of them as being different forms of this same thing, just with different parameters of where the agency is.
For instance, if you’re jamming with friends in a circle around a table there is space for improvisation and for everybody to create sound. This has an egalitarian nature to it. Whereas with an orchestra there is little scope for the musicians to choose what notes they play, but a huge scope for them to demonstrate technical virtuosity and skill, and I don’t think there’s anything wrong with that. I love orchestral music. I think there is beauty to the coordination and power. I can see how it could be abused politically, but it’s still a thing that I feel in my body when I experience it, and I want to be able to access that feeling.
What I’m most suspicious about are stadium-level concerts. The idolisation of one individual on a stage with everyone in the crowd going emotionally out of control. It is kind of this demagogue/mob relationship. People talk about these Trump rallies as if they’re like rock concerts, and it’s that kind of relationship that is abused politically.
Cave of Sounds was created by Tim Murray-Browne, Dom Aversano, Sus Garcia, Wallace Hobbes, Daniel Lopez, Tadeo Sendon, Panagiotis Tigas, and Kacper Ziemianin with support from Music Hackspace, Sound and Music, Esmée Fairbairne Foundation, Arts Council England and British Council.
You can read more of this interview in Part 2 which will follow shortly, where we discuss the future of music as well as practical advice for building installations. To find out more about Tim Murray-Browne you can visit his website or follow him on Substack, Instagram, Mastodon, or X.
Exploring the 2023 MIDI Innovation Awards
Dom Aversano
In Jaron Lanier’s cult classic technology manifesto, You Are Not a Gadget, the writer outlines a concept he calls lock-in, which he defines as when via mass adoption a technology becomes so deeply embedded into a culture that it becomes difficult to either improve or remove it without massive effort, even if its design is fundamentally flawed. The British road system exemplifies this, with its twisting and turning narrow lanes designed for horse-drawn carts, which while somewhat charming relics of a bygone era, can make it impossible to create separated bike lanes without bulldozing entire sections of cities. Lanier provides another example, MIDI, which he perceives as a reductive and delimiting music language that shrinks our conception of music to the functioning of keyboards, yet nevertheless, that he predicts will persist as a language well into the future, due to the huge work it would take to extract it from our musical infrastructure.
More than a decade after Lanier’s book was published his prediction that MIDI would persist is vindicated, however, Lanier may have underestimated the extent to which, unlike the British road system, MIDI has the capacity to transform itself without a major uprooting, which is the intention of MIDI 2.0. Anyone who has followed the non-starter of Web 3.0 will know that technological advancement requires more than adding a number and a decimal place to an existing technology. However, this new version of MIDI offers genuinely new capabilities, such as bidirectionally, backwards compatibility, a finer resolution of detail, and the capacity for instruments to communicate with greater sophistication.
Browsing through the entrants and finalists on the MIDI Association website reminded me of the show-and-tell-type events Music Hackspace put on in its early days. There is a nice balance between slick and sophisticated products built by established companies and eccentric innovations made in a shed by a devoted individual. As is the nature of these things, most of the innovations will not make their way to the mass market (presuming they were designed for it at all) but this does not detract from the creative value of the work. It is inspiring to see people make the brave effort of taking ideas from their imagination and putting them into the real world, so providing an audience for their efforts helps motivate and stimulate this innovation, by demonstrating that it has value and importance in our culture.
For the last few days, I have had the pleasure of indulging in a kind of digital sauntering, where I have explored and browsed through the wonderful collection of innovations on display. One original-looking instrument that immediately caught my eye is the Abacusynth, which as its name suggests, is built in the style of an abacus. The synth is intended to emphasise musical timbre with its creator stating:
“Timbral modulation is arguably just as ‘musical’ as melody or rhythm, but it’s not often emphasized for someone learning music, usually due to the complexity of synthesizer interfaces”.
One aspect of its interface design that is ingenious is that by spinning the blocks it creates a modulation effect, aligning the visual and kinetic aspects of the instrument in a playful and intuitive way.
Another visually appealing instrument is the Beat Scholar, which uses a novel pizza-slice-type interface to subdivide rhythms, provoking the visual imagination and making the likes of quintuplets and septuplets subdivisions much less intimidating. It is a much more visually appealing representation of rhythm than your average piano roll sequencer, where the interface for advanced rhythms often feels like an afterthought.
When it comes to slickness Roland’s AE-30 Aerophone Pro jumps out, with the company claiming it ‘the most fully-integrated and advanced MIDI wind controller ever created.’ It uses a saxophone key layout and mouthpiece and Bluetooth connection to free up players to move. It looks and sounds like a promising alternative to the keyboard and drum machine hegemony of electronic music, but will ultimately rely on the opinion of seasoned wind players as to whether it is adopted.
Finally, a music installation that stood out for its elegantly simple design is Sound Sculpture, which uses the collaboration and participation of a crowd to move glowing blocks around, which communicate their position to build a sequencer that creates a musical pattern. Watching people collaborate with strangers in this audio/visual artwork is particularly inspiring.
“This project utilizes 25 cubes in a space typically about the size of a half-basketball court. This spatial realization of composing, with blocks, allows multiple people to collaborate, co-compose as a community, and together create structures, rhythms, melodies, and harmonies.”
Whether you are in the depths of Argentina’s Patagonia or the buzzing metropolis of Lagos, you can join online to find out who the winners of this year’s MIDI Innovation Awards are, in a live-streamed 90 minutes show on Saturday, September 16th (10 am PDT / 1 pm EDT / 6 pm BST / 7 pm CET) that will be hosted by music Youtubers Tantacrul and Look Mum No Computer.
Ask me Anything about Visualizing Sound
Ask me Anything about Visualizing Sound