Neocortical and Hippocampal Theta Oscillations Track Audiovisual Integration and Replay of Speech Memories

(Downloading may take up to 30 seconds. If the slide opens in your browser, select File -> Save As to save it.)

Click on image to view larger version.

Figure 1.
Figure 1.

Movie stimuli, experimental design, and behavioral performances. A, Left, Example of normalized lip movement (green line; pixel units) and auditory speech envelope (orange line; Hilbert units) time-series from the same movie. Center, Coupling between the lip and auditory envelope signals in the movies (MI). The lip movements and speech envelopes predominantly synchronized on 4–8 Hz theta rhythms in the movies (purple line, mean ± standard error mean). This theta coupling disappeared when lip and speech envelope signals came from different movies (green line, mean ± standard error mean). Right, Illustration of the absolute theta timing manipulation induced between lip movement and auditory envelope onsets in order to create the synchronous and asynchronous conditions. B, Example of one block of the procedure consisting in a sequence composed of encoding (speech movie presentations), distractor (rehearsal prevention), and retrieval (memory test on the movies perceived during the encoding). Participants saw a clear image of the face, which is blurred here for anonymity purpose. C, Behavioral performance during encoding and retrieval. The sensitivity to theta audiovisual synchrony during movie encoding was assessed with the d′, and the bias to respond “synchronous” independently from the actual synchrony in the movies was assessed with the c criterion. Memory accuracy was assessed using the hit rate (i.e., where participants selected the correct speech stimulus pronounced by the speaker depicted during the cueing; 1 = 100% correct). Participants with a sensitivity greater than the d′ median (0.461) during the encoding were labeled as discriminators (green dots), while participants with a sensitivity lower than the median were labeled as nondiscriminators (orange dots). D, Sensitivity to audiovisual synchrony depending on the dominant frequency aligning lip movements and speech envelope in the movies during encoding. Participants were only sensitive to audiovisual synchrony in movies for which the lip movements and speech envelope aligned on slower theta frequencies (i.e., 4 and 5 Hz). Nevertheless, the memory performances were not later influenced by the dominant theta frequency at which lip and sounds were desynchronized in the movies, even when participants were sensitive to the theta timing in the lower 4–5 Hz frequencies (memory performances split by dominant frequency are not plotted). The boxplots represent the mean ± standard deviation and the error bars indicate the 5th and 95th percentiles. The dots represent individual means. Significant contrasts are highlighted with asterisks (*p < 0.05; ***p < 0.001).

  1. The Journal of Neuroscience vol. 45 no. 21
  1. Free Access 
Responses to This Article
Be the first to respond to this article
View Similar Articles
Similar articles in this journal»
    Advertisement