Investigating the cognitive foundations of
collaborative musical free improvisation

Experimental case studies
using a novel application of the
subsumption architecture

Adam Linson

The Open University (UK),
Centre for Research in Computing,
Faculty of Mathematics, Computing and Technology

Robin Laney (Open University, UK)
Chris Dobbyn (Open University, UK)
George Lewis (Columbia University, USA)

Geraint Wiggins (Queen Mary, University of London, UK)
Neil Smith (Open University, UK)

Submitted 31-Oct-2013
Examined 20-Dec-2013
Revised 24-Feb-2014


This thesis investigates the cognitive foundations of collaborative musical free improvisation. To explore the cognitive underpinnings of the collaborative process, a series of experimental case studies was undertaken in which expert improvisors performed with an artificial agent. The research connects ecological musicology and subsumption robotics, and builds upon insights from empirical psychology pertaining to the attribution of intentionality.

A distinguishing characteristic of free improvisation is that no over-arching framework of formal musical conventions defines it, and it cannot be positively identified by sound alone, which poses difficulties for traditional musicology. Current musicological research has begun to focus on the social dimension of music, including improvisation. Ecological psychology, which focuses on the relation of cognition to agent–environment dynamics using the notion of affordances, has been shown to be a promising approach to understanding musical improvisation.

This ecological approach to musicology makes it possible to address the subjective and social aspects of improvised music, as opposed to the common treatment of music as objective and neutral. The subjective dimension of musical listening has been highlighted in music cognition studies of cue abstraction, whereby listeners perceive emergent structures while listening to certain forms of music when no structures are identified in advance. These considerations informed the design of the artificial agent, Odessa, used for this study.

In contrast to traditional artificial intelligence (AI), which tends to view the world as objective and neutral, behaviour-based robotics historically developed around ideas similar to those of ecological psychology, focused on agent–environment dynamics and the ability to deal with potentially rapidly changing environments. Behaviour-based systems that are designed using the subsumption architecture are robust and flexible in virtue of their modular, decentralised design comprised of simple interactions between simple mechanisms. The competence of such agents is demonstrated on the basis of their interaction with the environment and ability to cope with unknown and dynamic conditions, which suggests the concept of improvisation.

This thesis documents a parsimonious subsumption design for an agent that performs musical free improvisation with human co-performers, as well as the experimental studies conducted with this agent. The empirical component examines the human experience of collaborating with the agent and, more generally, the cognitive psychology of collaborative improvisation. The design was ultimately successful, and yielded insights about cognition in collaborative improvisation, in particular, concerning the central relationship between perceived intentionality and affordances. As a novel application of the subsumption architecture, this research contributes to AI/robotics and to research on interactive improvisation systems. It also contributes to music psychology and cognition, as well as improvisation studies, through its empirical grounding of an ecological model of musical interaction.


A two-minute excerpt of Odessa (improvising robot, Disklavier piano)
with internationally renowned improvisors
Evan Parker (soprano sax) and John Russell (acoustic guitar),
recorded on August 21, 2013 at Goldsmiths, University of London.


Coming soon.


List of Figuresxi
1 Introduction1
 1.1 Musical free improvisation2
 1.2 Notation and formalisation4
 1.3 Situated activity 6
 1.4 Interaction 8
 1.5 The Physical and the Social 11
 1.6 Subsumption architecture 12
 1.7 Research questions 13
  1.7.1 Preliminary objections 15
 1.8 Overview and contributions 15
2 Literature Review18
 2.1 Contrast to composition systems 19
 2.2 Survey of systems relevant to the proposed research 20
  2.2.1 Cypher20
  2.2.2 Voyager21
  2.2.3 Swarm Music and Swarm Granulator22
  2.2.4 Free Improvisation Simulation 23
  2.2.5 VMMAS 24
 2.3 Cybernetics25
  2.3.1 CAIRA 27
  2.3.2 From cybernetics to Subsumption 29
 2.4 Other musical systems related to Subsumption31
  2.4.1 Reactive Accompanist31
  2.4.2 BeatBender33
 2.5 Psychology33
  2.5.1 Ecological psychology 34
  2.5.2 Intentional agency 35
 2.6 Methodological background 37
  2.6.1 Early AI and qualitative human judgement37
  2.6.2 Experimental context and participant perspective40 Analysing interactivity 43
  2.6.3 Methods applied to the study 45
 2.7 Summary and conclusions48
3 Building Odessa52
 3.1 Architectural principles: Using Subsumption for music53
  3.1.1 Real versus virtual input/output54
  3.1.2 Computational resources 54
  3.1.3 Unstructured input and models 55
 3.2 System design and implementation 56
  3.2.1 Input and output56 Note stream formation56 Note stream decomposition 58
  3.2.2 Interactivity 60 Interaction model 60 Interactive behaviour61 Layer interaction62
  3.2.3 Individual modules64 Modules in the Play layer 65 Modules in the Adapt layer66 Modules in the Diverge layer 67
  3.2.4 On the development of the initial prototype68
4 Evaluation70
 4.1 Methodology 70
  4.1.1 Experiment Description 70 Format and Procedure72 Participants 73 Apparatus74
 4.2 On the analysis 75
  4.2.1 Understanding the transcriptions76
  4.2.2 Anonymising the data 78 Anonymisation procedure79
  4.2.3 Detailed data analysis procedure79
5 Results82
 5.1 Introduction 83
  5.1.1 Initial analysis I 83 Central cross-sectional relationships in initial analysis87
  5.1.2 Initial analysis II89
  5.1.3 Summary95
 5.2 Assessment of system behaviour96
  5.2.1 Responsiveness97 The "zone" 99
  5.2.2 Lack of responsiveness 104
  5.2.3 Collaboration105 Leading and following 107 Partnership 108 "Strategy" or "approach" 110
 5.3 Design assessment 112
6 Follow-up study114
 6.1 Theoretical background for extending the design 114
 6.2 Design extension116
 6.3 Methodology of follow-up study 117
 6.4 Results of follow-up study 118
  6.4.1 Machine understanding121
  6.4.2 Playfulness122
  6.4.3 Impact of acoustic piano123
  6.4.4 Homeostasis and style 124
  6.4.5 Group interview (pre-trio)125
  6.4.6 Trio 126
 6.5 Discussion127
7 Conclusions132
 7.1 Revisiting the detailed research questions 133
 7.2 Insights from the study135
 7.3 Future work 137
 7.4 Beyond Odessa138
 7.5 Concluding remarks140
A On the implementation141
 A.1 Using ChucK for Odessa141
  A.1.1 Virtual machine and concurrency 141
  A.1.2 Rapid prototyping 142
  A.1.3 Native audio processing libraries 144
  A.1.4 Synthesis and MIDI 144
 A.2 Coding Subsumption in ChucK 145
  A.2.1 Modules and networks 145
  A.2.2 Synchronous versus asynchronous clock rates148
B Audio examples150

Download: Full PDF (with table-of-contents bookmarks & hyperlinks)


Coming soon.