I am currently investigating a little-known form of synaesthesia, whereby visual motion or flashes evoke faint auditory sensations. I have a personal interest in this, because I experience this phenomenon myself. I ‘hear’ car indicator lights, flashing shop displays, animated adverts on web-browsers, lip-movements, and the footsteps of people as they walk. It is a clear auditory sensation, mostly in my mind’s ear, though sometimes I can confuse it with real sounds if the latter are very quiet. The sounds are like white noise (‘sshhh’), but often they have different harmonics, especially when there are sequences of flashes. For example alternating flashing beacons on either side of a zebra crossing might sound like ‘shiii’, ‘shuuu’. The demos below simulate (very crudely!) the kinds of sounds that might be experienced. It’s interesting but it can get pretty distracting, especially at the end of a tiring day. I have now met quite a few people who experience this too, so perhaps I’m not completely bonkers!

Below are some demos of movies that make sounds for me. For some of them, I have added sounds, generated by measuring the motion energy from the video and using that to modulate white noise. The result only resembles very crudely what I typically experience, but enough to give an idea.

The visual ear: Our research

Our first research paper on this (Fassnidge et al, 2017 in Consciousness and Cognition) reports a systematic investigation of this phenomenon, which I would like to call the ‘Visually-Evoked Auditory Response‘ (V-EAR, or visual ear).  We use a combination of subjective and objective tests (see demo above) to establish the prevalence and the perceptual reality of this phenomenon. See the demos below of the tasks we used. We find that:

1. (1)Visually-evoked auditory sensations are reported in a surprisingly high proportion of normal individuals (22%), and this predicts higher performance on the original visual measure
   

2. (2)The auditory response is perceptually real enough to interfere with detection of real sounds even if they are not always strong enough to reach awareness.
   

Our results provide converging validation of the perceptual reality of this visually-evoked auditory response in neurotypical individuals. Its greater prevalence relative to other forms of synaesthesia may be due to the natural correspondence between visual events and the sounds that predictably accompany them (e.g. footsteps, lip-movements, collisions, explosions etc).

In our continuing research we have been using electrical brain stimulation to test whether stimulation of temporal brain areas (including primary auditory areas) can modulate the ability to perform visual sequence comparisons (see demo above, based on Saenz & Koch, 2008). The latter task may index the ability to can ‘hear’ the flashes, which can improve comparison accuracy.  See our ECVP2015 poster presentation (summary below) and a slideshow (UKSA2016).