Summary:
While Western people tend to look at lip movement when communicating, a
new study reports Japanese people are less influenced by looking at the
mouth when communicating.
Source: Kumamoto University.
Japanese people influenced less by lip movements when listening to another speaker: Evidence from neuroimaging study.
Which parts of a person’s face do you look at when you listen them
speak? Lip movements affect the perception of voice information from the
ears when listening to someone speak, but native Japanese speakers are
mostly unaffected by that part of the face. Recent research from Japan
has revealed a clear difference in the brain network activation between
two groups of people, native English speakers and native Japanese
speakers, during face-to-face vocal communication.
It is known that visual speech information, such as lip movement,
affects the perception of voice information from the ears when speaking
to someone face-to-face. For example, lip movement can help a person to
hear better under noisy conditions. On the contrary, dubbed movie
content, where the lip movement conflicts with a speaker’s voice, gives a
listener the illusion of hearing another sound. This illusion is called
the “McGurk effect.”
According to an analysis of previous behavioral studies, native
Japanese speakers are not influenced by visual lip movements as much as
native English speakers. To examine this phenomenon further, researchers
from Kumamoto University measured and analyzed gaze patterns, brain
waves, and reaction times for speech identification between two groups
of 20 native Japanese speakers and 20 native English speakers.
The difference was clear. When natural speech is paired with lip
movement, native English speakers focus their gaze on a speaker’s lips
before the emergence of any sound. The gaze of native Japanese speakers,
however, is not as fixed. Furthermore, native English speakers were
able to understand speech faster by combining the audio and visual cues,
whereas native Japanese speakers showed delayed speech understanding
when lip motion was in view.
“Native English speakers attempt to narrow down candidates for
incoming sounds by using information from the lips which start moving a
few hundreds of milliseconds before vocalizations begin. Native Japanese
speakers, on the other hand, place their emphasis only on hearing, and
visual information seems to require extra processing,” explained
Kumamoto University’s Professor Kaoru Sekiyama, who lead the research.
Kumamoto University researchers then teamed up with researchers from
Sapporo Medical University and Japan’s Advanced Telecommunications
Research Institute International (ATR) to measure and analyze brain
activation patterns using functional magnetic resonance imaging (fMRI).
Their goal was to elucidate differences in brain activity between the two languages.
The functional connectivity in the brain between the area that deals
with hearing and the area that deals with visual motion information, the
primary auditory and middle temporal areas respectively, was stronger
in native English speakers than in native Japanese speakers. This result
strongly suggests that auditory and visual information are associated
with each other at an early stage of information processing in an
English speaker’s brain, whereas the association is made at a later
stage in a Japanese speaker’s brain. The functional connectivity between
auditory and visual information, and the manner in which the two types
of information are processed together was shown to be clearly different
between the two different language speakers.
“It has been said that video materials produce better results when
studying a foreign language. However, it has also been reported that
video materials do not have a very positive effect for native Japanese
speakers,” said Professor Sekiyama. “It may be that there are unique
ways in which Japanese people process audio information, which are
related to what we have shown in our recent research, that are behind
this phenomenon.”
Abstract
Language/Culture Modulates Brain and Gaze Processes in Audiovisual Speech Perception
Several behavioural studies have shown that the interplay between
voice and face information in audiovisual speech perception is not
universal. Native English speakers (ESs) are influenced by visual mouth
movement to a greater degree than native Japanese speakers (JSs) when
listening to speech. However, the biological basis of these group
differences is unknown. Here, we demonstrate the time-varying processes
of group differences in terms of event-related brain potentials (ERP)
and eye gaze for audiovisual and audio-only speech perception. On a
behavioural level, while congruent mouth movement shortened the ESs’
response time for speech perception, the opposite effect was observed in
JSs. Eye-tracking data revealed a gaze bias to the mouth for the ESs
but not the JSs, especially before the audio onset. Additionally, the
ERP P2 amplitude indicated that ESs processed multisensory speech more
efficiently than auditory-only speech; however, the JSs exhibited the
opposite pattern. Taken together, the ESs’ early visual attention to the
mouth was likely to promote phonetic anticipation, which was not the
case for the JSs. These results clearly indicate the impact of language
and/or culture on multisensory speech processing, suggesting that
linguistic/cultural experiences lead to the development of unique neural
systems for audiovisual speech perception.
I am a professional ICT personnel, Chief System Analyst, blogger, Managing Director/Chief Executive Officer at Gatmond Internationals inc. and Country Director at Wake Up For Your Right Internationals USA (Nigeria Branch).
Subscribe to:
Post Comments (Atom)
AUTOMATIC RANKING OF TERTIARY INSTITUTIONS IN DEVELOPING NATIONS (A CASE STUDY OF NIGERIA UNIVERSITIES)
University ranking has become increasingly important in recent years among the general public at large because it creates a public platform...
-
SEO & Google AdWords Certified Freelance Consultant in Connecticut 860.633.6578 Professional SEO Services , Google AdWords S...
-
Healthy Energy drinks are generally consumed for various reasons. It could be before or after a strenuous workout regime , when you are cr...
No comments:
Post a Comment