Balancing the Senses: Electrophysiological Responses Reveal the Interplay between Somatosensory and Visual Processing During Body-Related Multisensory Conflict
- Alice Rossi Sebastiano1,
- Karol Poles1,
- Stefano Gualtiero1,
- Marcella Romeo1,2,
- Mattia Galigani1,
- Valentina Bruno1,
- Carlotta Fossataro1, and
- Francesca Garbarini1,3
- 1MANIBUS Lab, Psychology Department, University of Turin, Turin 10124, Italy
- 2IMT School for Advanced Studies Lucca, Lucca 55100, Italy
- 3Neuroscience Institute of Turin (NIT), Turin 10124, Italy
-
Author contributions: A.R.S., V.B., C.F., and F.G. designed research; A.R.S., K.P., S.G., M.R., and C.F. performed research; A.R.S. and M.G. analyzed data; A.R.S. and F.G. wrote the paper.
Abstract
In the study of bodily awareness, the predictive coding theory has revealed that our brain continuously modulates sensory experiences to integrate them into a unitary body representation. Indeed, during multisensory illusions (e.g., the rubber hand illusion, RHI), the synchronous stroking of the participant’s concealed hand and a fake visible one creates a visuotactile conflict, generating a prediction error. Within the predictive coding framework, through sensory processing modulation, prediction errors are solved, inducing participants to feel as if touches originated from the fake hand, thus ascribing the fake hand to their own body. Here, we aimed to address sensory processing modulation under multisensory conflict, by disentangling somatosensory and visual stimuli processing that are intrinsically associated during the illusion induction. To this aim, we designed two EEG experiments, in which somatosensory- (SEPs; Experiment 1; N = 18; F = 10) and visual-evoked potentials (VEPs; Experiment 2; N = 18; F = 9) were recorded in human males and females following the RHI. Our results show that, in both experiments, ERP amplitude is significantly modulated in the illusion as compared with both control and baseline conditions, with a modality-dependent diametrical pattern showing decreased SEP amplitude and increased VEP amplitude. Importantly, both somatosensory and visual modulations occur in long-latency time windows previously associated with tactile and visual awareness, thus explaining the illusion of perceiving touch at the sight location. In conclusion, we describe a diametrical modulation of somatosensory and visual processing as the neural mechanism that allows maintaining a stable body representation, by restoring visuotactile congruency under the occurrence of multisensory conflicts.
- body representation
- multisensory integration
- predictive coding theory
- rubber hand illusion
- somatosensory-evoked potentials
- visual-evoked potentials
Significance Statement
Given the inherent relationship between touch and body, the literature on body representation has mainly focused on the somatosensory system’s investigation, whereas less attention has been paid to the visual system. Here, we aim to investigate the modulation of both somatosensory and visual processing during a well-known multisensory illusion (i.e., the rubber hand illusion, RHI), in which visuotactile conflict is employed to challenge body representation. By recording electroencephalography, we show how the processing of somatosensory and visual stimuli is diametrically modulated during the RHI, with a decrease of the former and an increase of the latter. We conclude that this neural mechanism is triggered to restore visuotactile congruency, leading to the illusory feeling of perceiving touch at the sight location.
Introduction
According to the predictive coding framework, our brain works as an inference machine that interprets environmental events by comparing them to previously consolidated probabilistic models (Friston and Kiebel, 2009). Normally, bottom-up sensory inputs are congruent with top-down models, allowing us to experience consistency in our perceptions. However, when afferent inputs do not meet predictions, our brain may adjust the acquired information within the predictive model by exerting a top-down modulation on sensory processing [Tsakiris, 2017; Limanowski and Friston, 2020; see also Bayesian inference theories of multisensory integration (Chancel et al., 2022)].
Cognitive neuroscience has suggested that, as an effect of this modulatory mechanism, perception can be dramatically altered, even resulting in feeling tactile sensations beyond the physical body boundaries. A well-known example is the rubber hand illusion (RHI; Botvinick and Cohen, 1998), in which participants are deceived into thinking that a fake arm belongs to their own body. During the illusion, the participant’s concealed hand is synchronously stroked with a visible dummy hand, thus creating a multisensory conflict. Indeed, the afferent inputs coming from the somatosensory system (i.e., my hand is touched) and the visual one (i.e., the fake hand is touched) are conflicting under the probabilistic model that “my body is the only object in the world on which I feel touch when I see it being touched” (Limanowski and Blankenburg, 2013; Apps and Tsakiris, 2014). In other words, the dissociation between the felt touch on the own hand and the seen touch on the dummy violates the prediction of feeling and seeing touches at the same location, that is, the own body. Within the predictive coding theory (Friston, 2005), the illusory feeling has been interpreted as an attempt of solving this conflict, to restore a coherent representation of the body (Zeller et al., 2016). Accordingly, tactile processing might be realigned with the visual one through modulations of sensory processing, thus eliciting the sensation of feeling touch on the fake hand and, consequently, that the fake hand is part of the own body.
In the present work, through electroencephalography, we aimed to characterize somatosensory and visual processing modulations as the neural mechanism that allows maintaining a stable representation of the body under the occurrence of multisensory conflicts. Differently from previous studies, which largely employed the RHI visuotactile stimulation to measure the EEG-related changes (Zeller et al., 2015, 2016; Rao and Kayser, 2017), we designed two experiments in which we analyzed the amplitude of event-related potentials (ERPs) evoked by tactile (Experiment 1; somatosensory-evoked potentials, SEPs) and visual (Experiment 2; visual-evoked potentials, VEPs) stimuli delivered soon after the illusion induction. Thereby, we relied on the classical stroking of the real and fake hand to induce the RHI, while we employed more standard somatosensory and visual stimulations (i.e., electrical stimulation on the participant’s hand in Experiment 1 and an LED enlightening on the fake hand in Experiment 2), to reliably measure modulations of somatosensory and visual processing. This novel approach allows us to disentangle somatosensory and visual processing, by overcoming the intrinsic problem of EEG signals evoked by the same visuotactile stimulation employed during the RHI procedure, being such a stimulation multisensory in its nature.
Based on a preliminary study conducted in our lab, showing a decreased tactile and increased visual detection following the synchronous (illusion) as compared to the asynchronous (control) RHI (Rossi Sebastiano et al., 2021), we predicted to observe a coherent modulation of the neurophysiological processing of both tactile stimuli delivered on the participant’s hand and visual stimuli delivered over the rubber hand. Specifically, we expected the former to be decreased (i.e., smaller SEP amplitude) and the latter to be increased (i.e., greater VEP amplitude), favoring the realignment of touch onto vision. Moreover, we included baseline conditions in which no RHI stimulation was applied to describe the possibly observed modulations in terms of decrease/increase of sensory processing and to verify whether such modulation pertains to a specific RHI condition or both.
Materials and Methods
Participants
The sample size was estimated by means of an a priori power analysis based on our pilot study, in which the detection of somatosensory stimuli was assessed before and after synchronous or asynchronous RHI. Since we planned to perform a two-tailed t test between synchronous and asynchronous conditions as the main analysis, we computed the required sample size based on the contrast between the postsynchronous and postasynchronous RHI responses of our pilot data (dz = 0.87), employing the G*Power software. A sample of 16 participants was estimated considering α = 0.05 and power (1−β) = 0.9, coherently with previous electrophysiological studies on RHI-related effects on the somatosensory system (Zeller et al., 2015; Rao and Kayser, 2017; Sakamoto and Ifuku, 2021; with 13, 20, and 16 or 14 participants per experiment, respectively). Hence, to face possible dropouts given the multiple sessions planned for each experiment, two different samples of 18 human participants of either sex were recruited, one for each experiment [Experiment 1: 10 female and 8 men; age (mean ± SD), 25 ± 3.65; education level, 15.83 ± 2.64; Experiment 2: 9 female and 9 men; age, 26.79 ± 3.05; education level, 17.79 ± 0.42]. Note that, in Experiment 1, three participants dropped out before their third experimental session that was the baseline (see below). Thus, we included 18 participants in the main contrast of interest (i.e., synchronous vs asynchronous) and 15 participants in the contrast between each RHI condition and the baseline. In Experiment 2, one participant dropped out before her second session that was the synchronous session; thus, 17 participants were included in the analyses. All participants were right-handed according to the Edinburgh Handedness Inventory (Oldfield, 1971) and had normal tactile sensitivity and normal (or corrected-to-normal) visual acuity. They all gave informed consent to participate in the study, which was approved by the Ethical Committee of the University of Turin (Protocol No. 122571) and conducted in accordance with the Declaration of Helsinki.
Experimental procedures
In the present study, we employed EEG to collect ERPs to somatosensory (Experiment 1) and visual (Experiment 2) stimuli, following a period of RHI induction. The experimental procedure was identical for the two experiments, except for the sensory modality being tested (Fig. 1). In each experiment, participants underwent three experimental sessions, in which they performed either the synchronous, the asynchronous, or the baseline condition. The sessions’ order was randomized among subjects. Each session comprised two identical blocks, in which 50 trials were administered (in total, 100 trials). In between blocks, participants made a 5 min break to rest. Each trial consisted of a combined procedure comprising 12 s of synchronous or asynchronous RHI or no stimulation (baseline), immediately followed by the delivery of 10 subsequent somatosensory (Experiment 1) or visual (Experiment 2) stimuli at a frequency of 1.5 Hz (i.e., each train of 10 stimuli lasting 6.67 s; in total, 1,000 stimuli per condition). This procedure of alternating RHI miniblocks and stimulation periods has been previously demonstrated to be effective in maintaining the illusion throughout experimental tasks (della Gatta et al., 2016; Rossi Sebastiano et al., 2021, 2022a; Bruno et al., 2022), and we specifically adopted it to avoid the disappearance of the illusion. Hence, in the RHI conditions, stimuli delivery was alternated with 12 s of either synchronous or asynchronous RHI, and participants always gazed at a fixation cross immediately in front of the rubber hand (see below, Combined RHI and stimulation procedure). In the baseline, stimuli delivery was alternated with a 12 s rest period in which participants observed the fixation cross located in the empty RHI box. At the end of each RHI condition, embodiment and disembodiment questionnaires were collected (see below, Combined RHI and stimulation procedure).
Experimental procedure. The figure depicts the experimental procedures of Experiments 1 and 2. The darker hand represents the rubber hand; the lighter hands represent the participant’s hands. The combined RHI and stimulation procedure consisted of alternating 12 s of synchronous RHI, asynchronous RHI, or rest (according to the condition) with the delivery of a train of 10 stimuli (stimulus sensory modality, somatosensory in Experiment 1, visual in Experiment 2) at a frequency of 1.5 Hz (i.e., each train lasting 6.67 s). Participants underwent two blocks of 50 trials (in total, 100 trials, i.e., 1,000 stimuli per condition).
Combined RHI and stimulation procedure
The experimental materials used for the RHI procedure comprised a black wooden box (60 cm × 40 cm × 20 cm), a black piece of tissue, and two realistic rubber hands (male and female). The box was divided into two compartments: the right one was covered by a black panel, and the top of the left compartment was open. At the beginning of the session, participants sat comfortably at a desk in a dimly lit room and placed their right hand into the corresponding compartment of the box to hide it from their view. In the RHI sessions, the fake hand (male or female, according to the subject’s gender) was positioned inside the left open compartment to make the fake hand visible. The participant’s left hand was aligned with the left shoulder, and the fake hand was aligned with the right shoulder, whereas the participant’s right hand was externally positioned 20 cm to the right of the fake hand. We took care that the fake hand was placed in an anatomically congruent position from the participant’s egocentric perspective and that the black piece of tissue covered the shoulders and the proximal portions of the real and the dummy hand.
In the RHI conditions, 12 s of RHI procedure were performed before each train of 10 stimuli. Indeed, previous studies already showed that 12 s are sufficient to induce the illusion (Rossi Sebastiano et al., 2021; Bruno et al., 2022). In the synchronous RHI condition, in which the illusion generally occurs, the experimenter stroked the participant’s right hand synchronously with the rubber hand. Conversely, in the asynchronous condition, in which the illusion generally does not occur (i.e., control condition), the experimenter stroked the participant’s hand in antiphase with the rubber hand. Participants were asked to gaze at a fixation cross, which was placed immediately in front of the index finger of the rubber hand.
Note that, in the baseline condition, the rubber hand was removed from the left compartment of the box and participants did not receive any stroking. Thus, in this condition, participants just observed the fixation cross, which was in the same position as in the RHI conditions.
At the end of each RHI condition, participants rated (1) their subjective feeling of ownership over the fake hand by answering three items of the embodiment questionnaire (Botvinick and Cohen, 1998; Longo et al., 2008; Fossataro et al., 2018; Table 1) and (2) their subjective feeling of disownership over their real hand by answering three items of the disembodiment questionnaire (Botvinick and Cohen, 1998; Longo et al., 2008; Fossataro et al., 2018; Table 1). They were asked to rate their agreement with these items on a seven-point Likert scale, ranging from “−3” (“strongly disagree”) to “+3” (“strongly agree”).
Embodiment and disembodiment questionnaires (Longo et al., 2008; Fossataro et al., 2018)
Experimental stimuli
In Experiment 1, we employed the electrical stimulation of the median nerve (MN) to evoke the EEG responses, to reliably measure somatosensory processing following the illusion induction. By leveraging such a standard stimulation, we were able to interpret the effects in terms of latency, capitalizing on the well-grounded literature that exploited MN stimuli. Indeed, somatosensory processing is known to undergo several stages, ranging from lower-level encoding of the stimulus characteristics (e.g., frequency and intensity), reflected by short-latency MN-SEP components (<30 ms), to higher-level perceptual elaborations reflected by longer-latency MN-SEP components (Auksztulewicz et al., 2012; Muzyka and Estephan, 2019). In a specular way, in Experiment 2, we chose to employ a basic and punctual visual stimulation (i.e., a flashing LED) displayed in front of the fake hand, to characterize the visual processing modulations following the illusion induction (Odom et al., 2004, 2016; Creel, 2019). See below the details concerning each type of stimulus.
Somatosensory stimuli (Experiment 1)
Somatosensory stimuli consisted of transcutaneous electrical stimulation through constant current square-wave pulses (Digitimer D7AH) delivered over the right MN, using a pair of surface bipolar electrodes attached to a Velcro strap. The stimulus duration was 200 µs (Rossi Sebastiano et al., 2022b), and the intensity was set at 110% of the motor threshold, defined as the minimum stimulation intensity able to elicit a thumb twitch (Hu et al., 2011; Drewes and Haavik, 2019).
Visual stimuli (Experiment 2)
Visual stimulation consisted of brief flashes delivered through a red LED of 5 mm, and the stimulus duration was 5 ms (Creel, 2019). The LED was mounted immediately in front of the fixation cross, so that participants could see both the rubber hand and the LED during the RHI sessions.
Experimental design and statistical analysis
The present study shows two experiments (one investigating the somatosensory modality, Experiment 1, and the other investigating the visual modality, Experiment 2) with identical within-subject designs, consisting of three conditions: synchronous RHI, asynchronous RHI, and baseline. Please find below the detailed description of the performed statistical analyses.
Behavioral analyses
For both Experiments 1 and 2, mean ratings attributed to the embodiment and disembodiment questionnaires were separately compared between synchronous and asynchronous conditions by means of a Wilcoxon matched-pair t test (Shapiro–Wilk’s test on residuals, p < 0.05). Moreover, to investigate the relationship between embodiment and disembodiment phenomena in our samples, for each experiment, we calculated an embodiment index (i.e., mean embodiment ratings in synchronous minus mean embodiment ratings in asynchronous conditions) and a disembodiment index (i.e., mean disembodiment ratings in synchronous minus mean disembodiment ratings in asynchronous conditions) and entered these values in a Spearman’s correlation.
EEG recording and preprocessing
ERPs were recorded through 32 Ag-AgCl electrodes (international 10–20 system) referenced to the nose. Electrode impedances were kept below 5 kΩ. To track ocular movements and eye blinks, we recorded the electrooculogram by two surface electrodes, one placed over the right lower eyelid and the other placed lateral to the outer canthus of the right eye. Continuous EEG was recorded using a HandyEEG SystemPLUS Evolution amplifier (Micromed) at a 2,048 Hz sampling rate. Offline EEG preprocessing and analyses were performed with the Letswave6 toolbox (Nocions) for Matlab (MathWorks). The EEG signal was segmented into epochs lasting from 0.1 s before to 0.5 s after stimulus delivery (total epoch duration, 0.6 s). Epochs were bandpass filtered (1–100 Hz) using a Butterworth filter (Order 3) and then baseline corrected using as reference a signal period from 0.1 to 0.02 ms prior to the stimulus delivery. Artifacts due to eye blinks and eye movements were subtracted using a validated method based on an independent component analysis (Jung et al., 2000). Blinks were found to be the most frequent cause of rejection. Finally, epochs belonging to the same experimental condition (1,000 trials for each condition) were averaged, thus yielding three average waveforms, one for each condition (synchronous, asynchronous, and baseline).
ERP analyses
In the main analyses, to compare ERPs between the condition in which embodiment is elicited and the control one, we performed a matched-pair scalp-wise point-by-point t test between synchronous and asynchronous conditions. Furthermore, we ran additional analyses to verify whether the possibly observed modulations in the RHI conditions were in the direction of a decrease or an increase with reference to the baseline response to stimuli. Hence, we ran two matched-pairs point-by-point t tests for each experiment, comparing each RHI condition to the baseline.
Scalp-wise point-by-point t tests represent a statistical approach common in EEG studies (Harris et al., 2018; Novembre et al., 2018; Galigani et al., 2021; Fossataro et al., 2023), aimed to determine the temporal profile of an effect, highlighting significantly different ERP time windows between two experimental conditions without any a priori assumption. This between-condition comparison is performed for each time point composing the two waveforms, separately for each single channel. This approach allows us to capture, at the same time, significant amplitude differences and latency shifts between conditions, since both are displayed as discrepant waveform distributions across time. With the sole purpose of correcting these t tests for multiple comparisons among all points composing the two waveforms, cluster-based permutation testing is employed for each point-by-point analysis (1,000 random permutations; α level, 0.05; percentile of mean cluster sum set as threshold, 95; Maris and Oostenveld, 2007). Permutation testing is based on temporal adjacency and yields the identification of significantly different time point clusters between conditions for each channel. More specifically, we employed permutation testing as implemented in the Letswave6 toolbox (Nocions) for Matlab (MathWorks), based on Maris and Oostenveld (2007). The toolbox compares the EEG signal of the two conditions by means of a t value. Then, all samples whose t value is greater than the threshold are clustered in connected sets on the basis of temporal adjacency (i.e., time windows), and, after computing the sum of the t values within each time window, cluster-level statistics are calculated by selecting the largest of the time windows statistics, that is, this method selects the portion of the curves in which the difference between conditions is significant for the greatest number of adjacent points. Then, the statistical test is performed by calculating a p value under the permutation distribution and comparing it with the critical α level. The permutation distribution is obtained by randomly permuting the subject-specific average waveforms in the two conditions within every subject for 1,000 times, and, after the test statistic is calculated for each random partition, a histogram of random partitions’ test statistics is created. From the test statistic that was actually observed and the histogram obtained from random partitions’ test statistics, the proportion of random partitions that results in a larger test statistic than the observed one is calculated. This proportion is the p value under the permutation distribution. If the p value is smaller than the critical α level, then the datasets in the two experimental conditions are considered significantly different. Moreover, to determine the spatial location of the effect, a minimum number of two adjacent channels are conventionally considered as a significant cluster of electrodes (Pyasik et al., 2021; Ronga et al., 2021). In the result section, we therefore list each significant time window and channel cluster identified by the cluster-based permutation testing, and, for illustrative purposes, we report the results from the electrode in which the maximal difference between conditions was found for each time window.
Correlational analyses
We also performed correlational analyses to investigate the relationship between RHI susceptibility and the degree of ERP modulations between conditions. We extracted the mean amplitude from the electrodes in which maximal differences were observed for each experiment [Experiment 1 (somatosensory stimulation), C3 and C4 (see main analysis’ results and Fig. 2B); Experiment 2 (visual stimulation), C3 (see main analysis’ results and Fig. 4B)], in the time windows in which a difference between conditions was observed. In both experiments, for each channel, we calculated a modulation index, computed as the mean amplitude in the synchronous minus the mean amplitude in the asynchronous condition. This index, in Experiment 1, gives rise to an RHI-dependent somatosensory modulation that we referred to as amplitude decrease, while in Experiment 2, it gives rise to an RHI-dependent visual modulation that we refer to as amplitude increase. Based on the subjective ratings that participants attributed to the items of the embodiment and the disembodiment questionnaires, we calculated an RHI susceptibility index, computed as the mean ratings in the synchronous minus the mean ratings in the asynchronous condition. We chose to compute a single index that took into account both embodiment and disembodiment items since the two measures were highly correlated (see Results), as frequently found in RHI studies (della Gatta et al., 2016; Romano et al., 2021). Note that, since the RHI susceptibility index was entered in multiple correlational analyses (i.e., four comparisons in Experiment 1, C3 and C4 channel data over N140 and P300 components; one comparison in Experiment 2, C3 channel data over the P2–N3 complex), we set the statistical threshold for significance at α = 0.0125 for Experiment 1 and α = 0.05 for Experiment 2.
Results of Experiment 1. A, RHI questionnaire results: in the top left-hand corner, embodiment questionnaire results are plotted as mean embodiment ratings; in the top right-hand corner, disembodiment questionnaire results are plotted as mean disembodiment ratings; in the bottom part, the results of the correlation between the disembodiment index and the embodiment index are plotted. The dots represent each participant’s value, and the bars represent the standard error of the mean (SEM); ****p < 0.0005; ***p < 0.005. B–D, Results of SEPs analyses: the colored butterfly plots on top of each panel represent the grand-average waveforms of synchronous (in pink), asynchronous (in blue), and baseline (in yellow) conditions. Each wave represents a single electrode, the darker waves represent the mean amplitude across electrodes, and scalp maps with colored frames represent the amplitude of the evoked signal across the scalp in the synchronous (framed in pink), asynchronous (framed in blue), and baseline (framed in yellow) conditions. For each comparison, significant time intervals surviving permutation testing are highlighted by gray-shadowed boxes. The bottom waveforms represent the t value of each significant comparison on the electrodes showing maximal significance. The dashed horizontal lines represent the critical t values for statistical significance, and the darker black and gray lines correspond to the time windows in which the comparison survived permutation testing. Scalp maps framed in black represent the distribution of the t value across the scalp, and the black circles highlight the plotted electrodes. Note that, since asynchronous and baseline conditions were used as reference datasets respectively in the synchronous versus asynchronous and synchronous versus baseline comparisons, positive t values in the N140 interval and negative t values in the P300 interval represent smaller amplitude in the synchronous condition.
Results
Experiment 1
Behavioral analyses
The Wilcoxon tests revealed a significant difference between conditions for both embodiment (t(17) = 0; Z = 3.72; p = 0.0002) and disembodiment (t(17) = 14.5; Z = 2.93; p = 0.003) questionnaires, with higher values in the synchronous as compared to the asynchronous condition. Hence, we replicated the classical RHI effect on subjective reports. Indeed, participants experienced greater embodiment and disembodiment in the synchronous condition, in which the visuotactile conflict is elicited, as compared with the asynchronous one. Moreover, we found a significant positive correlation between the embodiment and the disembodiment indices (r(17) = 0.72; p = 0.0007). This result, in line with previous RHI studies (della Gatta et al., 2016; Romano et al., 2021), confirms that embodiment and disembodiment feelings are two related subjective experiences, both reflecting the participant’s susceptibility to the illusion (Table 2 and Fig. 2A).
RHI results of Experiments 1 and 2
SEP analyses
In this result section, we report the time windows in which the difference between conditions survived the cluster-based permutation testing that corrects for multiple comparisons. Note that the reported statistics pertain to the electrodes in which maximal statistical difference between conditions was found after correcting for multiple comparisons. The t test between RHI conditions revealed a significant difference over frontal, central, temporal, and parietal channels, with smaller amplitude in the synchronous as compared to the asynchronous condition in two time intervals: 0.14–0.18 s (C4, t(17) = 3.38; p = 0.003), coinciding with the latency of the N140 wave, and 0.24–0.32 s (C3, t(17) = −3.73; p = 0.002), coinciding with the latency of the P300 wave. These results suggest that, during the visuotactile conflict, whose settlement triggers the fake hand’s embodiment (i.e., in the synchronous RHI), somatosensory processing is decreased in N140 and P300 long-latency components. Moreover, the scalp distribution of both the evoked signal amplitude and the effect was maximal over central (C3, C4) electrodes in line with the somatosensory nature of the delivered stimuli (Fig. 2B).
Furthermore, the t test between the baseline and each RHI condition revealed significant differences only in the baseline versus synchronous comparison (and not in the baseline vs asynchronous comparison; Fig. 2C), in frontal and central channels. More precisely, we found a smaller amplitude in the synchronous as compared to the baseline condition within the N140 (C4, t(14) = −3.04; p = 0.009) and the P300 (F7, t(14) = 3.51; p = 0.003) time windows. Again, the scalp distribution of both the evoked signal amplitude and the effect was maximal over central (C4) and frontal (F7) electrodes in line with the somatosensory nature of the delivered stimuli. These results further corroborate the previous ones, showing that in the synchronous condition, in which embodiment is elicited, a decrease in somatosensory processing is observed (Fig. 2D).
Correlational analyses
The correlational analyses between the amplitude decrease index (mean amplitude in synchronous minus mean amplitude in asynchronous conditions) and the RHI susceptibility index (mean rating attributed to RHI questionnaires in synchronous minus mean rating attributed to RHI questionnaires in asynchronous conditions) highlighted a positive correlation in the N140 component over both central electrodes (C3, r = 0.59; p = 0.011; C4, r = 0.53; p = 0.022). Note that only the correlation with C3 survived multiple-comparisons corrections (α = 0.0125). These results suggest that the greater the RHI susceptibility, the stronger the decrease of the N140 amplitude (Fig. 3). No significant correlation was found with the P300 component (C3, r = −0.16; p = 0.54; C4, r = −0.28; p = 0.26).
Correlational results of Experiment 1. The figure shows the correlations between the RHI susceptibility index (computed as mean rating in synchronous minus mean rating in asynchronous conditions) and the amplitude decrease index (computed as mean amplitude in the N140 time window in synchronous minus mean amplitude in the N140 time window in asynchronous conditions in C3 and C4 electrodes). Note that, since the N140 has a negative polarity, a greater value in the synchronous condition, leading to positive values of the amplitude decrease index, indicated a decrease effect (i.e., smaller amplitude in synchronous as than that in asynchronous condition).
Experiment 2
Behavioral analyses
As in Experiment 1, the Wilcoxon tests revealed a significant difference between conditions for both embodiment (t(16) = 1; Z = 3.57; p = 0.0004) and disembodiment (t(16) = 22.5; Z = 2.56; p = 0.01) questionnaires, with higher values in the synchronous as compared to the asynchronous condition. Thus, also in Experiment 2, we replicated the classical RHI effect on subjective reports. Moreover, we found a significant positive correlation between the embodiment and the disembodiment indices (r(16) = 0.58; p = 0.016). This result replicates that of Experiment 1 and further confirms that embodiment and disembodiment feelings are two related subjective experiences, both reflecting the participant’s susceptibility to the illusion (Table 2 and Fig. 4A).
Results of Experiment 2. A, RHI questionnaire results: in the top left-hand corner, embodiment questionnaire results are plotted as mean embodiment ratings; in the top right-hand corner, disembodiment questionnaire results are plotted as mean disembodiment ratings; in the bottom part, the results of the correlation between the disembodiment index and the embodiment index are plotted. The dots represent each participant’s value, and the bars represent the standard error of the mean (SEM); ****p < 0.0005, *p < 0.05. B–D, Results of VEPs analyses: the colored butterfly plots on top of each panel represent the grand-average waveforms of synchronous (in pink), asynchronous (in blue), and baseline (in yellow) conditions. Each wave represents a single electrode, the darker waves represent the mean amplitude across electrodes, and scalp maps with colored frames represent the amplitude of the evoked signal across the scalp in the synchronous (framed in pink), asynchronous (framed in blue), and baseline (framed in yellow) conditions. For each comparison, significant time intervals surviving permutation testing are highlighted by gray-shadowed boxes. The bottom waveforms represent the t value of each significant comparison on the electrodes showing maximal significance. The dashed horizontal lines represent the critical t values for statistical significance, and the darker black and gray lines correspond to the time windows in which the comparison survived permutation testing. Scalp maps framed in black represent the distribution of the t value across the scalp, and the black circles highlight the plotted electrodes. Note that, in the synchronous versus asynchronous comparison, since the asynchronous condition was used as a reference dataset, positive t values represent greater amplitude in the P2–N3 complex in the synchronous condition, while, in the synchronous versus baseline comparison, since the baseline condition was used as a reference dataset, negative t values for the N2 and positive t values for the P2–N3 complex represent greater amplitude in the synchronous condition.
VEP analyses
In this result section, we report the time windows in which the difference between conditions survived the cluster-based permutation testing that corrects for multiple comparisons. Note that the reported statistics pertain to the electrodes in which maximal statistical difference between conditions was found after correcting for multiple comparisons. The t test between RHI conditions revealed a significant difference over frontal, central, temporal, parietal, and occipital channels, with greater amplitude in the synchronous as compared to the asynchronous condition in a time window between 0.2 and 0.3 s (C3, t(16) = 4.27; p = 0.0006), coinciding with the latency of the P2–N3 complex. These results suggest that, during the visuotactile conflict whose settlement triggers the fake hand’s embodiment (i.e., in the synchronous RHI), visual processing is increased in P2–N3 long-latency components. Moreover, although the scalp distribution of the evoked signal amplitude was maximal over posterior occipital electrodes in line with the visual nature of the stimuli, the scalp distribution of the effect was maximal over centroparietal electrodes (peaking at C3), suggesting the involvement of somatosensory cortices in the observed modulation (Fig. 4B).
Furthermore, the t test between the baseline and each RHI condition revealed significant differences only in the baseline versus synchronous comparison (and not in the asynchronous vs baseline comparison, Fig. 4C). We found greater amplitude in the synchronous as compared to the baseline condition over frontal, central, parietal, and occipital electrodes. More precisely, these differences were observed both in a 0.15–0.17 s time window (O2, t(16) = −4.85; p = 0.0002), coinciding with the latency of the N2 component, and in 0.18−0.22 s (FC6, t(16) = 3.44; p = 0.003) and 0.24–0.28 s (t(16) = 3.15; p = 0.006) time windows, corresponding to the latency of the P2–N3 complex. The scalp distribution of the effect was maximal in frontocentral (FC6) and occipital (O2) electrodes. These results further corroborate the previous ones, showing that in the synchronous condition, in which embodiment is elicited, an increase in visual processing is observed (Fig. 4D).
Correlational analyses
The correlational analyses between the amplitude increase index (mean amplitude in synchronous minus mean amplitude in asynchronous conditions) and RHI susceptibility index (mean rating attributed to RHI questionnaires in synchronous minus mean rating attributed to RHI questionnaires in asynchronous conditions) revealed no significant correlation over the central electrode (C3, r = 0.15; p = 0.57). This negative result suggests that, at least in our data, there is no significant linear relationship between the observed increase in visual processing and the RHI susceptibility.
Discussion
In the present study, we described a sensory processing modulation, as the neural mechanism that allows maintaining a stable representation of the body, under multisensory conflict. To this aim, we capitalized on a well-known body-related multisensory illusion, the RHI (Botvinick and Cohen, 1998), to measure the modulations of SEPs’ (Experiment 1) and VEPs’ (Experiment 2) amplitude during the multisensory conflict (synchronous RHI) as compared to a control condition in which no conflict is elicited (asynchronous RHI) and a baseline in which no RHI stimulation is applied. Throughout the task, each trial alternated 12 s of RHI procedure (replaced by 12 s of rest in the baseline) with a tactile/visual stimulation period lasting 6.67 s, allowing the illusion to be maintained. Our results compellingly show that, during synchronous as compared to asynchronous conditions, somatosensory-evoked activity is decreased (i.e., smaller SEP amplitude following tactile stimulation of the participant’s hand), whereas visual-evoked activity is increased (i.e., enhanced VEP amplitude following visual stimulation on the rubber hand). Importantly, such effects are illusion-dependent, since significant differences emerged only when the synchronous (and not the asynchronous) condition was compared to the baseline. Taken together, these findings provide the first electrophysiological evidence of a diametrical modulation of somatosensory and visual processing as the neural mechanism underpinning the settlement of the visuotactile conflict during the RHI.
Previous electrophysiological studies described a modulation of somatosensory processing, mainly analyzing the signal evoked by the same tactile stimulation employed during the RHI procedure (Peled et al., 2003; Zeller et al., 2015; Rao and Kayser, 2017). These works provide mixed results: while some found that the illusion modulated the amplitude of the evoked potentials by downregulating it (Zeller et al., 2015; Rao and Kayser, 2017), some others found it enhanced (Peled et al., 2003). However, in these studies, the analyzed tactile stimulus was always coupled with a visual one. Hence, here, to disentangle somatosensory and visual processing, we employed a different methodological approach, by analyzing the signal evoked by tactile and visual stimuli delivered soon after repeated time windows of the illusion induction. Specifically, while we relied on the classical stroking as a visuotactile procedure to induce the RHI, we measured the electrophysiological responses to punctual only-somatosensory (Experiment 1) and only-visual (Experiment 2) stimulations to characterize their processing soon after the multisensory conflict. Thereby, we were able to dissociate somatosensory and visual systems’ modulations, showing that they have an opposite direction.
In Experiment 1, we found decreased amplitude of two late-latency MN-SEP components, that is, the N140 and the P300, in the synchronous condition. Both the scalp distribution of the evoked signal and the topography of the observed effects were maximal over frontal and central electrodes, consistently with the somatosensory stimulation. These findings are compatible with previous studies highlighting a downregulation of both somatosensory and the adjacent motor system, by employing either transcranial magnetic stimulation (TMS; della Gatta et al., 2016; Kilteni et al., 2016; Isayama et al., 2019), transcranial direct current stimulation (Hornburger et al., 2019), TMS-EEG (Casula et al., 2022), or EEG. More specifically, previous EEG studies measured an embodiment-dependent amplitude decrease in late time windows that exceed the short-latency components (Aspell et al., 2012; Zeller et al., 2015; Rao and Kayser, 2017). For instance, Rao and coworkers highlighted an attenuation of the ERPs elicited by RHI stimulation in a long-latency range (0.33 s) over frontocentral channels (Rao and Kayser, 2017). Aspell and coworkers found a smaller amplitude of tibial nerve SEP between 0.11 and 0.2 s during the full-body illusion over frontal electrodes (Aspell et al., 2012). This late latency suggests that the illusion-dependent modulation does not involve early stages of somatosensory processing localized in primary areas but rather pertains to more high-level perceptual and attentional aspects, reflected at later processing stages. Our correlational results further suggest a crucial role of late-latency modulations, since the N140 amplitude decrease was linearly related to RHI susceptibility. Outside the context of body representation literature, modulations occurring at such a latency have been linked to tactile perceptual processes (Schubert et al., 2006; Auksztulewicz et al., 2012; Dembski et al., 2021). Capitalizing on tactile detection paradigms, the N140 was found to be the first component to reliably differ between detected and undetected stimuli, demonstrating that tactile awareness modulations affect the N140 amplitude. Our findings, pinpointing somatosensory modulations in a late-latency range previously linked to tactile awareness, are in line with behavioral evidence showing that the RHI induces a reduction of tactile perception (Folegatti et al., 2009; Zopf et al., 2011; Rossi Sebastiano et al., 2021).
Although previous studies focused on somatosensory and motor domains, less attention has been paid to visual processing. Van der Hoort and colleagues addressed whether embodiment affects visual awareness in a binocular rivalry paradigm (van der Hoort et al., 2017). They found that the amount of time in which participants detect a hand image alternating with a mask increased only when the participants’ body ownership over that hand was manipulated. This result suggests that visual processing might be promoted during bodily illusions. Coherently, we recently showed that visual detection of stimuli occurring around a fake hand is increased during the RHI (Rossi Sebastiano et al., 2021). Here, we provide the electrophysiological counterpart of this visual effect. Specifically, ERP modulation was localized in long-latency time windows, with a significant increase in VEP amplitude in N2 (0.15–0.17 s) and P2–N3 (0.18–0.3 s) components, in the synchronous condition. Interestingly, previous studies linked visual perception processes to the recurrent activity between early visual areas and higher-order cortices, occurring after 0.16–0.2 s from stimulus onset (Fahrenfort et al., 2008; Taylor et al., 2010; Dembski et al., 2021). For instance, Fahrenfort and colleagues recorded EEG, while participants were engaged in the detection of masked visual targets. They found that posterior activity peaking at 0.16 s and subsequent alternating patterns of frontoparietal and occipital activity correlated highly with target perception (Fahrenfort et al., 2008). Our results indicate embodiment-dependent enhancement of VEP amplitude in late components encompassing similar latencies, in line with the previously described enhanced visual perception around an embodied rubber hand (Rossi Sebastiano et al., 2021).
Interestingly, in Experiment 2, while the scalp distribution of the evoked signal was maximal over posterior occipital electrodes consistently with the delivery of visual stimuli, the difference between RHI conditions was maximal over central electrodes, suggesting the involvement of somatosensory processing when a visual stimulus appears near an embodied fake hand. This finding is interesting if we consider the nature of the RHI phenomenon. When the illusion is triggered, participants report to feel touches on the fake hand, that is, to feel touch at the “visual touch” location. Based on this reasoning, we might speculate that, through the modulation of sensory processing described above, touch is remapped on vision, resulting in the recruitment of somatosensory regions at the simple sight of a visual stimulus near the embodied hand. Accordingly, among the number of studies that investigated brain activity modulations during the RHI multisensory stimulation employing fMRI (Ehrsson et al., 2004; Ehrsson, 2005; Brozzoli et al., 2012; Bekrater-Bodmann et al., 2014), Limanowski and Blankenburg observed lower touch-related activity in the primary somatosensory cortex and a significant activity increase in high-order visual and somatosensory areas (Limanowski and Blankenburg, 2016), as well as an increase of the functional connectivity between sensorimotor and visual areas (Limanowski and Blankenburg, 2015). Coherently, Zeller and colleagues, by performing dynamic causal modeling on the EEG activity evoked by the RHI stimulation, found not only a downregulation of somatosensory activity but also an increased connectivity between visual and premotor areas (Zeller et al., 2016).
To conclude, despite the evidence that we provide, some study limitations must be acknowledged. First, correlational results were significant only in Experiment 1. This could be ascribed either to the absence of a linear relationship between RHI susceptibility and visual enhancement or to the relatively small sample size, being our study designed to test between-condition differences. Future studies could test larger samples, to specifically investigate the relationship between RHI susceptibility and the magnitude of such a diametrical sensory modulation. Second, our study capitalized on an experimental context of visual dominance over tactile input (i.e., the RHI). However, this represents a one-sided test of a theory that could predict also reverse results. In future studies, by employing experimental contexts of somatic dominance over visual input, diminished VEPs and enhanced SEPs should be expected. Furthermore, our model could be generalized to other sensory modalities, as in the ventriloquism effect (Alais and Burr, 2004), in which visual input drives the illusion of perceiving sounds at the sight location. This should result in increased VEPs and decreased auditory-evoked potentials.
In sum, our approach, able to disentangle between somatosensory and visual processing, allowed us to describe somatosensory and visual diametrical modulations as the neural mechanism that restores visuotactile congruency, inducing the illusion of perceiving touch at the sight location (Fig. 5). Differently from previous accounts that focused mainly on somatosensation, we show not only a decrease of tactile stimuli processing but also an increase of visual stimuli processing, probing a crucial role of the visual system in maintaining a stable body representation during multisensory conflict. Moreover, since our effects were observed in late-latency stages of sensory processing, they might regard higher-level elaborations linked to conscious somatosensory and visual perception, possibly resulting from the modulation of high-order areas through the mediation of attentional factors (Tsakiris, 2017; Limanowski and Friston, 2020).
Diametrical modulation of somatosensory and visual processing. In this study, the RHI was exploited as a model of multisensory conflict settlement through visual dominance over tactile input. Since our brain predicts that touch and visual touch should converge on the own body, when touches are applied to both the real (hidden) hand and the fake (visible) one in synchrony, a multisensory conflict is detected, thus generating a prediction error. To explain it away, a concomitant modulation of somatosensory and visual processing occurs: while the former is reduced (as supported by reduced SEP amplitude), the latter is enhanced (as supported by increased VEP amplitude). The effect of such a modulatory mechanism is that tactile sensations are remapped on visual input resulting in visual dominance over tactile input, so that tactile sensations are felt on the fake hand and, thus, the fake hand is felt as part of the body.
Footnotes
- Received July 24, 2023.
- Revision received December 27, 2023.
- Accepted January 3, 2024.
-
We thank all of the participants involved in the study. This work was funded by the European Union (ERC-STG, MyFirstBody, 101078497) to F.G. The views and opinions expressed are however those of the authors only and do not necessarily reflect those of the European Union or the European Research Council Executive Agency. Neither the European Union nor the granting authority can be held responsible for them. Moreover, this work was supported by the BIAL Foundation Grant (311/2020) to C.F.
-
The authors declare no competing financial interest.
- Correspondence should be addressed to Francesca Garbarini at francesca.garbarini{at}unito.it.
- Copyright © 2024 the authors










