How can the close collaboration between sound artist and composer Moritz Nahold, and artist, writer, and performer Kenneth Constance Loe subtly challenge the concept of a straightforward algorithm, writing or composing by blurring and slipping the boundaries between input and output, embodiment and AI, text and sound through applying personal experiences and references? The project Ghost in the Cog: A Poetic Score invites users to activate a poem as a score by reading and playing it with no clear set of rules, and get lost in sound and webspace.
Moritz Nahold and Kenneth Constance Loe in conversation with Jazmina Figueroa and Denise Helene Sumi — Nov 22, 2023
Moritz Nahold and Kenneth Constance Loe during the presentation of their project »Not The Whole Truth« for Intermedia at Hothouse, Singapore, 2023.
Denise Sumi: The web residencies »Algorithmic Poetry.« called for projects that look at »how sound may poetically speak with algorithms.«1 How does the relation between a set of clear instructions or directives and poetic principles come across in Ghost in the Cog: A Poetic Score?
Kenneth Constance Loe: As a writer and poet, I am more interested in literary conceptions about the algorithm (Roald Dahl’s The Great Automatic Grammatizator) and its potential for wordplay (@portmanteau_bot from early Twitter days) than in the algorithmic per se. For me, there is also this question of what happens when you extend or slow down the rhythm.
Moritz Nahold: In this project we tried to see ourselves as algorithms. An algorithm always has this set of rules, and it executes something. If I get text from Kenneth, I get inspired to react to it. It’s acting, reacting, and so forth. Algorithmic composition itself is such a big topic in music; Iannis Xenakis or Karl Heinz Stockhausen are among its many protagonists. There are many ways you could approach music algorithmically in a traditional sense with mathematical models like stochastic or fractals, for example, but I’m not really interested in this kind of music composition technique. Seeing ourselves as algorithms seemed more personal and more suitable for the project.
Kenneth: Envisioning ourselves as algorithms is a way of bouncing off different temporalities. Sort of this back-and-forth between something that is hyper quick, and something that’s a slow process.
Jazmina Figueroa: The concept of a feedback loop is central to your approach. How do you envision this process aiding in producing, intertwining, and refining sound and text material?
Kenneth: This bouncing back and forth of the feedback loop is sort of the only parameter that we’ve set for ourselves. It’s not a content-driven parameter. It gives a very nice rounding because this parameter doesn’t block stuff from entering, but maybe it doesn’t leave stuff to escape.
Moritz: It’s always interesting what you end up with. The constant back-and-forth might be the start of something completely different. It’s like this with every task. It is like doing research. At some point, it gets very defined and you get very good at it. It’s the same with my artistic practice, how I compose, or how I perform music. It might happen that the whole structure crumbles at some point because you took a wrong turn or whatever, but it’s a positive thing.
Denise: There is this beautiful coexistence of both your heterogeneous individual practices, working with text and sound. The focus is on the process. This way of performing, acting, and thinking with each other goes beyond singular or binary perspectives. Also, the website incorporates this significant improvisational and performative element. Have you discussed how to bring the performative dimension to the webspace?
Moritz: The original idea of the score was not as performative or interactive as it ended up being. The focus on having a website that also functions as a performative element changed drastically after the positive reaction during our live Zoom performance at the first internal crit meeting with the other web residents.
Kenneth: I agree. Everyone seemed enamored by its performative potential. For example, kirby pointed us to Tiger Dingsun’s Reading Machines, an amazing publishing platform that really shaped how we imagined our website. At this point, we hadn’t coded anything yet. Obviously, there are many things you could do on a performative level. Create an AR environment, a gamescape, but I envisioned the website to be flat. Like the way you would read a poem in a book. When it comes to the activation, we talked about the way you read poems and their evocativeness or how words bring out certain associations, which in turn informed the decision of having textual elements would either trigger a particular soundtrack, grow bigger, extend, and/or potentially modulate sound. Conceptually, I am indebted to the collaboration between graphic designer Will Holder who founded F.R.DAVID and writer Mason Leaver-Yap , which explored the relationship between typography and the voice.
»I think the functionality of our score becomes like an instrument where there’s no start or end, there’s no linearity. The text becomes the four strings of a violin, or the keys of a piano, and you’re free to just play whatever.«
Jazmina: Kenneth, can you elaborate on the creation of the script/score for this work, specifically the literary connections and apparatuses incorporated, and how they are situated on screen?
Kenneth: Concrete Poetry has been a source of inspiration, from Guillaume Apollinaire’s »Calligrammes« to CAConrad’s »Shards.« I also think a lot about Italo Calvino’s Invisible Cities. I like the idea of hyperlinks as a form of encountering things, passages, or texts, or whatever. But it still draws you back to a singular place. The initial inspiration I used was a text that I had written with sound in mind: a bio of a sound artist where I had experimented with descriptions analogous to that of a rollercoaster ride. Like the phrase »chain lift on high frequency oscillating« was my attempt to capture the quality of his work. I used that as the start of the score and then folded in fragments from other texts and poems that I’ve written before while searching for onomatopoeia across the web, somehow landing on this »chuf-chuf-chuffing« sound that helicopters make. These led me to fixating on phonic descriptions in books from the 70s through 90s, these soundbites serving as mini time capsules, so to speak.
Denise: Mo, can you talk about the link between the textual layer and the score as the instrument?
Moritz: It’s interesting for me because I’m classically trained. I went to music school, and I know how to read sheet music. It’s very clear what you do when you encounter certain notes and stuff, but with text it’s different. But in the end, it’s just another way of how you could envision a score with a different set of rules. It’s more of a direct meaning, where certain text phrases would be associated with certain sound qualities and feelings.
Kenneth: It’s so funny that we never talked about this between the two of us – I also can read sheet music but mostly only notes in the treble clef from my violin-playing days. My initial impulse for the website was to reference the aesthetics of the classical music score – plain white background, a simple serif font, symbols like the forte (f) and caesura(//), etc. I think the functionality of our score becomes like an instrument where there’s no start or end, there’s no linearity. The text becomes the four strings of a violin, or the keys of a piano, and you’re free to just play whatever.
Jazmina: Mo, in Sonic Fiction, Holger Schulze emphasizes the role of the audience in co-creating the fictional worlds he proposes. How do you envision readers/players participating in the co-creation of soundscapes and poetry within the poetic score?
Moritz: First and foremost, I envision them going on the website and clicking through the links and experiencing something they haven’t experienced before in a way. How could they participate? If we’re able to implement this idea of recording their score, we would end up with about 100 different scores. Users can type promptly their own words and see what kind of sound would come out and layer them over while they read the text base. The text is quite visual and all phrases create a lot of imagery in one’s head.
Jazmina: How do you both feel you incorporate storytelling or narrative aspects in the generation of sound and poetry in this project?
Kenneth: The idea of storytelling and a narrative is a bit difficult to answer because I imagined the text growing much longer and at some point, there was a realization that, you know, this text alone was already enough. There’s so much packed into it and I haven’t necessarily considered the narrative of it. The storytelling comes through these references to books, movies, my own memories, computer games that I’d played a lot as a child, such as Roller Coaster Tycoon and Red Alert 2. Honestly, a lot of the content also comes from more personal experiences.
Apollo ASM
Denise: What role did the practice of hyperlinks, bookmarks, personal archives, and browsing play for you?
Kenneth: I often wonder about the omission of footnotes in poems, and that’s something that I’ve sort of tried to incorporate in some poems of mine. In this score, it’s quite apparent that the books where these occurrences of phrases that are sound-related are also mentioned. It’s important because it really extends on the idea of sound and its relation to text. There’s a historicity to it, I would say, through this citation and quoting. Some of the code is also borrowed or adapted from existing user-created snippets from online communities like CodePen and JSFiddle.
Denise: When it comes to your visual approach to the screen, referencing the internet wormhole no longer comes as a hyper-visual overstimulation,2 but rather as an almost plain Google Docs, okay, not plain, but with a simple hyperlink aesthetic and some mouse over animations. Can you both say something about this aesthetic decision?
Moritz: The Google Doc thing in the beginning was just a very easy way to visualize what we wanted to do. Everybody knows Google Docs, right? You are immediately in a framework of familiarity. You don’t have to explain how this whole thing works. And if you have something visually very overstimulated, there’s always a need for explanation in a way for people to get them in a mood or understand what you’re trying to do. If you have a lot of sounds going on and a lot of hyperlinks, there’s already quite a lot of stimulation, you can break it down visually and have it quite simple in a way.
Kenneth: I would like to reference this scene from Contact, which also appears on our website, where Jodie Foster’s character falls through the wormhole and is transported into the future or a parallel universe or whatever. She’s going through these landscapes and otherworldly visions, meeting this other hyperintelligent species. It feels like a dream … and then she comes back … she realizes that she was away for one second. But then after that, it is revealed that 18 hours of static had actually been recorded in her recording device, but no visuals had been captured. For me that scene really encapsulates your question. It’s something that is not initially or not overtly stimulating or hyper-aesthetic but still potentially unfurling a world that could be 18 hours long, yeah?
Denise: Did you implement current machine learning tools that generate text to sound? How do you feel about these tools in times of advanced digital visual or textual reproduction?
Moritz: Probably half of it was AI generated and half was made as a reaction to Kenneth’s text input. I was working with Sounds.Studio (https://sounds.studio/), a browser-based DAW (Digital Audio Workstation). Besides having the option to automatically split full songs into separate tracks (useful for remixes), this tool has a more elaborate text to sound generator than I encountered so far. But it’s still mostly focused on the traditional Western music theory. If you put in something very abstract, the AI doesn’t really know what to do with it and it tries to follow a given set of rules. So, I fed it with different poetry chunks, which then were further processed. So yeah, it’s really a mixture of let’s say conventionally produced sounds and AI generated ones. The website itself currently plays only samples, so there is no real time AI generation happening. But the funny thing is that most of the code for the website I contributed, is made with ChatGPT. It is helpful for sure, but you also must tweak and poke your prompts quite a lot to get what you want.
Kenneth: I tried a bit of this text-to-sound AI that and it was a bit frustrating because what it was generating was very literal. And there was this one prompt about the »chuf chuf chuffing of the helicopter« and it sounded stunningly similar to something that Mo had already produced. I’ve been thinking about this meta layer of code, too. It’s not visible, right? I mean, of course, if you went into like the browser and you said view, inspect elements or view source code or something, you could see the code. But it’s not something that we’ve talked about or discussed or explored. It’s interesting to imagine this code that’s so integral to the score, for lack of a better metaphor, as the wood to the violin, the backbone, and it happens to be text-based, too. And that adds another layer to what we’re doing.
Denise: One last question. What is a »Ghost in the Cog«?
Kenneth: It’s a combination of two references, the anime Ghost in the Shell and the idiom »a cog in the machine.« But I’m laughing because I’ve never watched Ghost in the Shell. But this is how it was like, for me: us just being another cog in the machine, right? And ultimately, the blurring between exteriority and interiority, embodiment and AI, tools and products, text and sound.
Presentation of the project »Not The Whole Truth« for Intermedia at Hothouse, Singapore, 2023.
Moritz Nahold is a Vienna-based sound artist and composer.
Kenneth Constance Loe (he/they) is an artist, writer, and performer from Singapore, and currently based in Vienna, Austria.
Jazmina Figueroa initiated the nineteenth call for Web Residencies and the collaboration between Digital Solitude and Liquid Architecture. Figueroa is a writer and performer.
Denise Sumi is coordinator of the program Digital Solitude and doctoral researcher at the Peter-Weibel-Research Institute for Digital Cultures in Vienna.
All images courtesy the artists.
See call for Web Residencies No. 19 »Algorithmic Poetry.«: https://www.akademie-solitude.de/de/web-residencies/call-for-web-residencies-no-19-algorithmic-poetry/ (accessed on November 6, 2023).
See for example Still Life (Betamale) by Jon Rafman and Oneohtrix Point Never (2013) with its references to the desktop-performance aesthetic (window within a window within a window), the Vapor Wave aesthetics.
© 2024 Akademie Schloss Solitude and the author
Beteiligte Person(en)
