Lip reading is a technique used by many people, some of whom are deaf, while others have mastered the art of reading lips in order to fill in gaps from language differences or communication difficulties. To better understand what is happening inside the brains of people who are reading lips, a study was conducted that discovered that the brains of lip readers synchronize to the sound waves that would normally be heard, even when there is no audible sound present.
Published in the Journal of Neuroscience, the researchers measured 28 healthy, adult participants’ brain waves with magnetoencephalography while they silently viewed footage of a woman speaking. Some participants listened to a story. The brain scans revealed that the group who were reading lips reacted in the same way as those who were listening to the voice, and there was synchronization in the auditory cortices of the brain.
The sound waves produced by the woman speaking in the video were matched by the brain waves of those who only watched and lip-read. This was evidence that the visual cortex can synchronize to the movement of speech and then sends the message to the rest of the brain which can translate the lips’ movements into sound. This creates the sound waves and the real-time synchronization that was seen on the brain scans.
The first step in lip reading occurs when the sight of moving lips activates neuronal activity in the visual cortex in the same frequencies that the lip movements would produce. Next, the right angular gyrus of the brain detects the slower, more nuanced details of the lip movements and maps them onto matching auditory sound features. This process is completed when the information is sent to the auditory cortex where the speech is analyzed and constructed.
The authors of the study have found evidence into how lip-reading works, as it was debated if mental imagery of speech and non-speech sounds helped the lip-reader understand what was being said. It is actually the “synthesis of auditory stimulus” which helps the brain translate lip movement into sounds within the brain.
References
Mathieu Bourguignon, Martijn Baart, Efthymia C. Kapnoula, Nicola Molinaro. Lip-reading enables the brain to synthesize auditory features of unknown silent speech. The Journal of Neuroscience, 2019; 1101-19 DOI: 10.1523/JNEUROSCI.1101-19.2019
