Movements that cause sound are different from those that don’t

Research spotlight #1

Neszmélyi, B. & Horváth, J. (in press). Consequences matter: Self‐induced tones are used as feedback to optimize tone‐eliciting actions, Psychophysiology. doi: 10.1111/psyp.12845*

This paper addresses a conceptual and methodological issue in research which studies attenuated perception of sensory feedback of one’s own actions. Research has indicated that neural responses to sensations that are caused by one’s actions (e.g. the sound of a bell you press yourself) result in reduced neural activity when compared to sensations that are not self-generated (e.g. the sound of a bell you didn’t press). This idea makes intuitive sense: you are likely be more sensitive to things in your environment that you did not immediately cause to happen, since things you did cause can be anticipated. How this is studied is by recording brain EEG activity (electrical signals on the scalp) when someone makes a movement that causes a sound, subtracting the activity due to the movement itself (without sound), and comparing the result to just listening to the sound with no movement. The big assumption here, which this paper tackles, is that the muscle movements which cause sound and the same movements without auditory consequences are essentially controlled in the same way.

In the experiment reported, participants had to pinch a Force Sensing Resistor (FSR) with a pre-set amount of force. This constituted the movement of the task. After some training with a visual display to get the required amount of force right, participants then performed the task under two conditions. In one condition, their pinch was effectively sonified. When they applied the right level of force, a 1000 Hz sine tone sounded (audio-motor condition: AM). In another condition, they had to pinch the FSR without any sound feedback (motor only: M). A final condition involved listening to recordings of sounds created by their previous pinches during which they made no finger movements (audio-only: A). The AM condition always came first, while the order of the A and M conditions were varied between participants. Participants did 300 repetitions of each condition (to get good EEG data, lots of repetitions of a given event are necessary). Pinch force was recorded in the AM and M conditions, and EEG activity was recorded in all three conditions.

The first striking thing that was found in the results was that the pinch force applied differed dramatically between the condition in which pinches caused the tone (AM) versus the no-tone condition (M). Force applied was a lot lighter when the pinch produced a sound than when it didn’t. The second result of interest was that as a consequence of the differences in movements when there was an auditory consequence versus none could account for the attenuated EEG response to self-generated sounds, when applying the subtraction approach described above. While this second result is of interest to people working on the neuroscience of agency, for me the first finding is the most interesting.

The difference in force applied when expecting a tone to be caused by your action versus silent pinching suggests that auditory perceptual feedback from movement can materially alter the nature of coordination with the task at hand. When sounds feedback from movements is present, the movements change because the interaction has changed. This needs to be considered when comparing, for example, behaviour during movement sonification with non-sonified movement interactions (something I would routinely do in my own research on sonification in skill learning). It needs to be considered how the auditory feedback may not only change the way that movement is guided, but also the entire nature of the task as experienced by the agent involved (actor, learner, etc.). A shift from an experimenter’s-eye-view to a participant’s-eye-view is, as always, essential.

Another important idea which this study highlights was masterly laid out by John Dewey’s in his 1896 critique of the Reflex Arc Concept in Psychology (essentially the fixation on stimulus-response causality). In the AM condition, the tones may be the result of the pinches, but they also seem to modulate the coordination of subsequent pinch movements. Essentially, participants pinched more softly when they knew a sound would result. Thus, the sensory consequences of one action modified the control of the subsequent actions across the block of trials. Behaviour is here best characterised as an ongoing coordination between action and perception, not a set of isolated stimulus-response pairs.

A final idea that this study highlights which I want to mention is the potential folly of assuming that one can study coordination and the brain by adding and subtracting perceptual and motor elements of a task. In this study,  neural activity in the AM condition was not simply the sum of activity in A and M, as had previously been assumed. If the brain is a non-linear dynamical system (and it surely is), then the different ways that elements of a task inter-relate and affect the overall state of the system need to be very carefully considered.

Of course, there are limitations in the experiment (as there are in every individual scientific study). The sound used (1000 Hz tone) is typical of a lab-based experiment, but is frustratingly un-ecological. A more ‘pinch-relevant’ auditory consequence might have revealed further interesting motor and neural behaviours. The movement itself (pinch force) is rather limited and one-dimensional, so it is not clear how far this effect could generalise to more complex actions. Also, having the A condition after the AM condition for everyone is problematic as this might mean that neural responses to the tones in the listening condition are affected by the newly formed action-sound mapping from the previous 300 AM trials. Still, this paper did address a potentially pernicious assumption in the neural agency literature, and in so doing highlighted important concepts for thinking about auditory-motor coordination processes.


*If you cannot access this article, use the DOI in Sci-Hub to get it. Science should be open to everyone!