Composing Beyond the Screen
Originally published on Substack .
· Essay
In my last post, I wrote about the problem of how to get started with composing and how computer-assisted, algorithmic work can help move us past the blank page or screen. In this essay, I want to consider how focusing on tools and technique, whether software or music theory, can overshadow a fundamental skill in composing with digital technologies.
The issue of what a composer needs to know has more to do with human perception than it does with technical ability. So the skill that I want to explore has to do with the acquisition of knowledge that comes from perceiving sound itself through an embodied sensitivity to sound as a physical medium.
Listening to a musical composition means engaging with sound events that are constantly unfolding over time. But where composing is mediated through a screen, perhaps in the context of some music software, the act of looking and responding to visual information inevitably plays a central role in musical creation.
This primacy of the visual may be experienced when using digital audio workstations or even music notation software, where transformative operations on musical material rely on visual engagement. Often the developers will be led by good practice in UX design, which is intended to smooth the way a composer navigates the software typically with some input device like a computer mouse.
There is of course great compositional value in this kind of software, which offers algorithmic techniques, digital signal processing and more. But in order to access the procedures on offer, I would suggest that there is an assertion of the visual mode of perception (which may even lean upon visual literacy) even though looking is, intuitively at least, not the primary way of perceiving musical experience.
This leaning towards the visual in pursuing digital composition leads to a situation where a kind of ‘reaching’ becomes necessary, past the screen, towards the felt experience of music. A challenge here is encountering the visual domain in digital composition in a way that mitigates its dominance.
One ‘solution’ is to avoid the distraction of what is technically possible given the tools at our disposal. This means working in a way that relegates the constant sense of what the array of tools can do for us. The problem with this awareness is that the decisions we make in our creative work can be reduced to the ‘how’. Instead, we might shift awareness, ‘through the screen’ and the visual design of its toolkits, to focus on tracing the aurality of our work.
This aurality is a kind of reduction, a way of focusing our perceptual awareness on the trajectories and gestures of the sounds that we are making, on what these sounds might do and how we might act, next, so as to realize that unfolding.
What might transpire here is a shift in the performance of digital musicianship, from seeing towards hearing, from tools to the material of sound, in a way that structures our creativity as a dream-like mingling of listening and imagining. This follows on from a point I made in my last post on algorithmic composition: the significance of attuning to musical material in order to sense what that material wants to do, where it wants to ‘go’.
We have already observed that a musical composition unfolds over time. That is its nature. When I say wants to do or go, I mean how it might turn out to be very different from how it is right now. Or, how its future form accounts for our experience of it, in the present.
This ‘futural’ quality, which is inherent in musical creativity, is relevant to what we are discussing here. Imagine glancing at the screen, scrolling through all the available resources, all the plugins, code, instruments waiting to be used. There is a sense in becoming aware of these things, waiting in the wings as it were, that there are many possible futures for our sounds and that each of these futures may be accessed through the specific way in which each of these resources inscribes them, over time.
But the risk of looking too closely at these possible, future inscriptions is in losing all sense of what it is we are actually doing. It’s not so much that there are too many options as it is that there may not be sufficient, perceptual engagement with the material (the composition, as it stands) in order to know what to choose. So we ask ourselves, what is it we are making?
If the answer to this question is technical, then there is probably a sense in which we can say that we are composing ‘to’ the software. The means of production is leading the experience of the work, if you like. One way around this might be to answer the question ‘what are we making’ with a deep perceptual knowledge through an engagement with the musical material that is currently in play. Sometimes, this can be done simply by looking away from the screen and asking, what does this particular material or section want to do?
Perceptual knowledge of how a musical (sonic) gesture unfolds becomes a storehouse for future creativity, for knowing how to shape a musical idea. It is fundamental to musical expression since compositional creativity requires judgement, which means deciding whether the material is doing its work.
Sensitivity to this form of knowing lets us manoeuvre ourselves, as artists, into a position to speak, making unclouded use of available technologies in a way that refuses to be led by them. As human listeners, we are remarkably adept at detecting the scaffolding of technology. We have an uncanny facility for sensing this support mechanism, even if we can’t quite identify the source of our unease.
© N.G.Brown, 2025
📬 If you enjoyed this, subscribe to my Substack newsletter for more reflections on art and creativity: Sign up on Substack.