Developing a critical technical audiovisual performance practice engaging with 'the web' as both medium and subject matter.

Project Overview

Clock is the first project I made as an exploration into the idea of ‘website as medium’. Specifically, I created it to fill a 12-15 minute performance slot at the 2024 Electric Spring Festival. It is a part automated, part live, conceptual audiovisual piece that explores ideas relating to time, work, and a tension that arises when digital systems influence human notions of productivity.

Project

Development

There were a number of concepts that influenced the development of this concept:

Data Sonification

Since completing my installation Entity (see Context: My Background), I have been interested in data sonification as a method of giving shape and meaning to sound works. Data sonification is any kind of process where sound is used to represent a dataset, much in the same charts and graphs are used to represent data in the field of data visualisation. It is often a scientific practice, though there are a number of interesting artistic applications that have influenced my practice (Mardakheh and Wilson, 2022).

At the time, I had also incidentally picked up the habit of keeping spreadsheets of personal information; things like spending and income, hours worked each day, units of alcohol consumed, weight, and instances of keeping up ‘good habits’ I wanted to instill in myself. I wasn’t keeping these spreadsheets for any particular reason beyond the satisfaction of watching the datasets grow (and maybe a belief on some level that they helped me be ‘good’). However, this habit did strike me as being somewhat reflective of a certain mindset. One you might see more often in programmers, and (I hate to admit) one you might see more often in ‘productivity’ influencers. A fixation on improvement, goal setting, efficiency, and on tracking oneself. I thought it could be a good starting point to make a work that reflected on that particular aspect of being an ‘internet user’: the calendars, the spreadsheets, the numbers, the reminders, and the way that our devices passively capture all this information about us just by virtue of us using them.

Personal Reflections

Reflecting particularly on a Weekly Hours spreadsheet ↗I was keeping, and how it showed my inability to be even and consistent in meeting my goal week to week (I was sometimes over, and sometimes under) I felt it highlighted the difference between how humans and computers work. Computers count time evenly and mechanically work at the fastest possible pace, always, executing a given task. Humans experience time subjectively, some days feel longer than others for example, and our ability to apply effort consistently is affected by a myriad of factors; physical health, mental state, the presence of other distractions, the tendency to procrastinate when further away from a deadline etc.

I was struck by how I felt a genuine frustration at not being able to keep my weekly hours in the light green zone of ‘weeks met’ and it brought to mind Agre’s paper Surveillance and Capture: Two Modes of Privacy (Agre, 1994). He describes the way in which a given activity, when analysed in the pursuit of computerising some or all of that activity (for example, say, the automation of the collection of customer feedback data in a shop), the activity itself ends up needing to change to adapt to the new, computerised, activity. I wondered if, on some level, through using a computer so much, I had started to expect myself to be able to behave like a computer.

I made a series of sketches to start working through the idea conceptually:

I kept coming back to an idea of grids/squares/blocks sliding in and out of uniformity, and chose that as a starting point in which to begin designing both my user interface and my performance concept.

Musical Influences

At this time, alongside beginning to negotiate my overall approach to the practice, I was also early in the process of formulating a coherent approach to sound and musical style. Knowing that my aim was to situate this practice within and across sound art, computer music, and multimedia/new media performance, there are a handful of artists operating in those worlds I identified as being particularly exemplary of a style I would like to explore. They are Yara Mekawei ↗, Ryoji Ikeda ↗,Alva Noto ↗.

There is a sonic aesthetic across all their works that strikes me as being particularly digital. Small, discrete, units of abstract sound, arranged in a way that implies machine or mechanical intervention. Works that particularly influence this piece include: Unitxt by Noto (Nicolai, 2008), and 658 by Mekawei (Mekawei, 2019).

How it works

Taking my allotted 12 minute time slot, I broke that duration down into 52 blocks of roughly 13 seconds, each to represent a week in the last year period in which I had been collecting data. Within each 13 second block I would perform a sequence of audio that mapped, in a loose representational way, to data that I had gathered about that week. This would be alongside an automation within the performance tool that activated a new part of the UI at the start of each block, effectively resetting whatever sequence I was performing and prompting me to start another one. Through this process I would be layering the computer’s even sense of timing with my variable, expressive, human one with the aim of creating some sense shifting caused by the tension between the two.

I utilised different datasets in different ways. The core one being the aforementioned ‘Weekly Hours’. With this I built a grid of 365 buttons, each to represent a day in the year. The function of each being to switch a continuous sound on and off, each sound was white noise filtered at a frequency value mapped to the amount of time I spent working that day.

The automation activated every block of 7 buttons in order, while also turning off anything else actively making sound, to introduce a kind of baseline sound for that block. I would then use other bits of data I had for that week, as well as information I could glean from my digital ephemera (messages, emails, photos) to compose something that reflected what could have been assumed to be the ‘vibe’ of that week? How would I have been feeling to have produced these kinds of numbers and digital artefacts? For example, one week may have shown that I got plenty of work done, hit a lot of goals, and took a lot of pictures with my friends, safe to assume that was a happy week. Another might have shown I took a few sick days, drank a lot of alcohol, and had an argument in my texts, on the whole a negative week.

Reflections

Overall, as a first pass at both a performance tool, and an av performance in the browser I was happy with how this turned out and it raised a few aesthetic and conceptual concerns to continue exploring.

Firstly, though it was included almost as an oversight, I was struck when watching the performance back at how much of an effective visual having the browser window and mouse cursor included in the projected image was. It serves to frame my practice as one of performing with the web, not seeking to hide the technical infrastructure required to run my tools, aligning with the notion of critical technical practice I’m trying to achieve.

Next, there is a roughness and glitchiness to both the sound and visuals that really reflect that sense of the ‘early web’ and net art adjacent aesthetic that I’m trying to achieve.

And lastly, given that I do have a stated interest in showing my work and processes, taken from live coding. This piece forced me to consider whether I was really doing that, and indeed consider the level and format in which I would want to show the audience what I am doing and thinking as I play. Reflecting on this, and my assumptions and experiences with live coding wherein I had had that initial misinterpretation of what is happening when someone is live coding (see Context: Live Coding) led me to formulate a certain position that defines how I situate my work as being in conversation with live coding.

You cannot really make an assumption about what the audience does or does not understand (Toplap.org, 2024). Someone who has a working understanding of TidalCycles has a different experience to someone who does not when watching somebody perform with it. But there is no objective better experience between those two. In both cases the performer is making the effort to show gesture, their manipulating code is analogous to a piano player pressing keys for example.