BACKGROUND & AIMS: :Previous event-related potential (ERP) studies have identified the superior parietal lobule (SPL) as actively multisensory. This study compares effective, or contextually active, connections to this region under unisensory and multisensory conditions. Effective connectivity, the influence of one brain region over another, during unisensory visual, unisensory auditory and multisensory audiovisual stimulation was investigated. ERPs were recorded from subdural electrodes placed over the parietal lobe of three patients while they conducted a rapid reaction-time task. A generative model of interacting neuronal ensembles for ERPs was inverted in a scheme allowing investigation of the connections from and to the SPL, a multisensory processing area. Important features of the ensemble model include inhibitory and excitatory feedback connections to pyramidal cells and extrinsic input to the stellate cell pool, with extrinsic forward and backward connections delineated by laminar connection differences between ensembles. The framework embeds the SPL in a plausible connection of distinct neuronal ensembles mirroring the integrated brain regions involved in the response task. Bayesian model comparison was used to test competing feed-forward and feed-backward models of how the electrophysiological data were generated. Comparisons were performed between multisensory and unisensory data. Findings from three patients show differences in summed unisensory and multisensory ERPs that can be accounted for by a mediation of both forward and backward connections to the SPL. In particular, a negative gain in all forward and backward connections to the SPL from other regions was observed during the period of multisensory integration, while a positive gain was observed for forward projections that arise from the SPL.

背景与目标： : 先前的事件相关电位 (ERP) 研究已确定顶叶上小叶 (SPL) 为主动多感官。这项研究比较了在单感觉和多感觉条件下与该区域的有效或上下文活动的联系。研究了在单感觉视觉，单感觉听觉和多感觉视听刺激期间，一个大脑区域对另一个大脑区域的影响。在进行快速反应时间任务时，从放置在三名患者顶叶上方的硬膜下电极记录了erp。在一种方案中，反转了erp相互作用的神经元集合的生成模型，该方案允许研究多感觉处理区域SPL之间的连接。集合模型的重要特征包括与锥体细胞的抑制性和兴奋性反馈连接以及对星状细胞池的外部输入，外部前向和后向连接由集合之间的层流连接差异描绘。该框架将SPL嵌入到不同神经元集合的合理连接中，从而反映了响应任务中涉及的整合大脑区域。贝叶斯模型比较用于测试如何生成电生理数据的竞争性前馈和后馈模型。在多感觉和单感觉数据之间进行了比较。来自三名患者的发现表明，单感和多感觉erp的总差异可以通过与SPL的前向和后向连接的中介来解释。特别是，在多感觉整合期间，从其他区域到SPL的所有向前和向后连接中均观察到负增益，而对于SPL产生的向前投影则观察到正增益。

BACKGROUND & AIMS: :Rett syndrome (RS) is a neurological disorder resulting from an X-linked dominant mutation. It is characterized by a variety of physical and perceptual disabilities, resulting in a need for continuous intervention programs to be administered on a regular basis throughout life. Many of these individuals with RS show fear of movement and, therefore, find it hard to accept external facilitation (so common in physical therapy intervention). In a search for novel intervention techniques that might improve their ability to cope with difficulties in daily situations, while also reducing their difficulty in handling motion inflicted by an external physical facilitator, we examined the use of the Snoezelen room. The Snoezelen, also known as the controlled multisensory environment, can provide a soothing atmosphere that appeals to the individual with RS, while at the same time it can improve physical, sensorial, and functional abilities. This article suggests various intervention goals that are appropriate for individuals with RS at different stages of the disorder. Since the management of young children with RS in the multisensory environment has been discussed at length in the past, this article will mainly describe intervention with adults with RS, focusing on three case stories. The present article reviews the available scientific materials on the topic of Snoezelen, incorporating clinical knowledge in the field of RS and suggesting this approach as an appropriate intervention method for this population.

背景与目标： : Rett综合征 (RS) 是一种由X连锁显性突变引起的神经系统疾病。它的特点是各种各样的身体和知觉残疾，导致需要在一生中定期执行持续的干预计划。这些患有RS的人中的许多人表现出对运动的恐惧，因此很难接受外部便利 (在物理治疗干预中很常见)。为了寻找新颖的干预技术，这些技术可以提高他们在日常情况下应对困难的能力，同时还可以减少他们处理外部物理促进者造成的运动的难度，我们研究了Snoezelen房间的使用。Snoezelen，也称为受控的多感官环境，可以提供舒缓的氛围，吸引个人RS，同时可以改善身体，感官和功能能力。本文提出了各种干预目标，这些目标适用于处于疾病不同阶段的RS患者。由于过去已经详细讨论了在多感官环境中对患有RS的幼儿的管理，因此本文将主要描述对患有RS的成年人的干预，重点是三个案例故事。本文回顾了有关Snoezelen主题的现有科学资料，结合了RS领域的临床知识，并建议将这种方法作为该人群的适当干预方法。

BACKGROUND & AIMS: :The capacity to integrate information from different senses is central for coherent perception across the lifespan from infancy onwards. Later in life, multisensory processes are related to cognitive functions, such as speech or social communication. During learning, multisensory processes can in fact enhance subsequent recognition memory for unisensory objects. These benefits can even be predicted; adults' recognition memory performance is shaped by earlier responses in the same task to multisensory - but not unisensory - information. Everyday environments where learning occurs, such as classrooms, are inherently multisensory in nature. Multisensory processes may therefore scaffold healthy cognitive development. Here, we provide the first evidence of a predictive relationship between multisensory benefits in simple detection and higher-level cognition that is present already in schoolchildren. Multiple regression analyses indicated that the extent to which a child (N = 68; aged 4.5-15years) exhibited multisensory benefits on a simple detection task not only predicted benefits on a continuous recognition task involving naturalistic objects (p = 0.009), even when controlling for age, but also the same relative multisensory benefit also predicted working memory scores (p = 0.023) and fluid intelligence scores (p = 0.033) as measured using age-standardised test batteries. By contrast, gains in unisensory detection did not show significant prediction of any of the above global cognition measures. Our findings show that low-level multisensory processes predict higher-order memory and cognition already during childhood, even if still subject to ongoing maturation. These results call for revision of traditional models of cognitive development (and likely also education) to account for the role of multisensory processing, while also opening exciting opportunities to facilitate early learning through multisensory programs. More generally, these data suggest that a simple detection task could provide direct insights into the integrity of global cognition in schoolchildren and could be further developed as a readily-implemented and cost-effective screening tool for neurodevelopmental disorders, particularly in cases when standard neuropsychological tests are infeasible or unavailable.

背景与目标： : 整合来自不同感官的信息的能力对于从婴儿期开始的整个生命周期的一致感知至关重要。在以后的生活中，多感官过程与认知功能有关，例如言语或社交交流。在学习过程中，多感觉过程实际上可以增强对单感觉对象的后续识别记忆。这些好处甚至可以预测; 成年人的识别记忆性能是由同一任务中对多感觉信息 (而不是单感觉信息) 的早期反应所决定的。发生学习的日常环境，例如教室，本质上是多感官的。因此，多感官过程可能会支撑健康的认知发展。在这里，我们提供了第一个证据，证明简单检测中的多感官益处与学童中已经存在的高级认知之间存在预测关系。多元回归分析表明，儿童 (n =   68; 4.5-15岁) 在简单的检测任务上表现出多感官益处的程度，不仅在涉及自然物体的连续识别任务上预测益处 (p =   0.009)，即使在控制年龄时，但是，同样的相对多感官益处也预测了使用年龄标准化测试电池测量的工作记忆得分 (p   =   0.023) 和流体智力得分 (p   =   0.033)。相比之下，单感检测的收益并未显示出对上述任何全球认知指标的显着预测。我们的发现表明，即使仍在持续成熟，低水平的多感觉过程也可以预测童年时期的高阶记忆和认知。这些结果要求修订传统的认知发展模型 (可能还包括教育)，以说明多感官处理的作用，同时也为通过多感官程序促进早期学习提供了令人兴奋的机会。更一般地说，这些数据表明，简单的检测任务可以直接了解学童的全球认知的完整性，并且可以进一步开发为易于实施且具有成本效益的神经发育障碍筛查工具，尤其是在标准神经心理学测试不可行或不可用的情况下。

BACKGROUND & AIMS: :The caudal medial auditory area (CM) has anatomical and physiological features consistent with its role as a first-stage (or "belt") auditory association cortex. It is also a site of multisensory convergence, with robust somatosensory and auditory responses. In this study, we investigated the cerebral cortical sources of somatosensory and auditory inputs to CM by injecting retrograde tracers in macaque monkeys. A companion paper describes the thalamic connections of CM (Hackett et al., J. Comp. Neurol. [this issue]). The likely cortical sources of somatosensory input to CM were the adjacent retroinsular cortex (area Ri) and granular insula (Ig). In addition, CM had reliable connections with areas Tpt and TPO, which are sites of multisensory integration. CM also had topographic connections with other auditory areas. As expected, connections with adjacent caudal auditory areas were stronger than connections with rostral areas. Surprisingly, the connections with the core were concentrated along its medial side, suggesting that there may be a medial-lateral division of function within the core. Additional injections into caudal lateral auditory area (CL) and Tpt showed similar connections with Ri, Ig, and TPO. In contrast to CM injections, these lateral injections had inputs from parietal area 7a and had a preferential connection with the lateral (gyral) part of Tpt. Taken together, the findings indicate that CM may receive somatosensory input from nearby areas along the fundus of the lateral sulcus. The differential connections of CM compared with adjacent areas provide additional evidence for the functional specialization of the individual auditory belt areas.

背景与目标： : 尾内侧听觉区 (二1212) 具有与其作为第一阶段 (或 “带”) 听觉关联皮层的作用一致的解剖和生理特征。它也是多感官融合的场所，具有强大的体感和听觉反应。在这项研究中，我们通过在猕猴中注射逆行示踪剂来研究二1212的体感和听觉输入的大脑皮层来源。一篇伴随论文描述二1212的丘脑连接 (Hackett等人，J. Comp. Neurol. [本期])。二1212输入体感输入的可能的皮质来源是相邻的岛后皮层 (Ri区) 和颗粒状岛叶 (Ig)。此外二1212与多感觉整合位点Tpt和TPO区域具有可靠的联系。二1212还与其他听觉区域具有地形联系。正如预期的那样，与相邻的尾部听觉区域的连接要强于与鼻端区域的连接。令人惊讶的是，与核心的连接沿其内侧集中，这表明核心内可能存在功能的内侧-外侧划分。向尾侧听觉区域 (CL) 和Tpt的其他注射显示与Ri，Ig和TPO的连接相似。与二1212注射相反，这些侧向注射具有来自顶叶区域7a的输入，并且与Tpt的外侧 (回旋) 部分具有优先联系。总之，研究结果表明二1212可能会从沿侧沟眼底的附近区域接受体感输入。二1212与相邻区域相比的差分连接为各个听觉带区域的功能专业化提供了额外的证据。

BACKGROUND & AIMS: :Children who are over-responsive to sensation have defensive and "fight or flight" reactions to ordinary levels of sensory stimulation in the environment. Based on clinical observations, sensory over-responsivity is hypothesized to reflect atypical neural integration of sensory input. To examine a possible underlying neural mechanism of the disorder, integration of simultaneous multisensory auditory and somatosensory stimulation was studied in twenty children with sensory over-responsivity (SOR) using event-related potentials (ERPs). Three types of sensory stimuli were presented and ERPs were recorded from thirty-two scalp electrodes while participants watched a silent cartoon: bilateral auditory clicks, right somatosensory median nerve electrical pulses, or both simultaneously. The paradigm was passive; no behavioral responses were required. To examine integration, responses to simultaneous multisensory auditory-somatosensory stimulation were compared to the sum of unisensory auditory plus unisensory somatosensory responses in four time-windows: (60-80 ms, 80-110 ms, 110-150 ms, and 180-220 ms). Specific midline and lateral electrode sites were examined over scalp regions where auditory-somatosensory integration was expected based on previous studies. Midline electrode sites (Fz, Cz, and Pz) showed significant integration during two time-windows: 60-80 ms and 180-220 ms. Significant integration was also found at contralateral electrode site (C3) for the time-window between 180 and 220 ms. At ipsilateral electrode sites (C4 and CP6), no significant integration was found during any of the time-windows (i.e. the multisensory ERP was not significantly different from the summed unisensory ERP). These results demonstrate that MSI can be reliably measured in children with SOR and provide evidence that multisensory auditory-somatosensory input is integrated during both early and later stages of sensory information processing, mainly over fronto-central scalp regions.

背景与目标： : 对感觉反应过度的儿童对环境中普通水平的感觉刺激有防御性和 “战斗或逃跑” 反应。根据临床观察，假设感觉过度反应反映了感觉输入的非典型神经整合。为了检查该疾病的潜在神经机制，使用事件相关电位 (erp) 研究了20名具有感觉过度反应 (SOR) 的儿童同时进行的多感觉听觉和体感刺激的整合。提出了三种类型的感觉刺激，并从32个头皮电极记录了erp，同时参与者观看了无声的动画片: 双侧听觉咔嗒声，右体感正中神经电脉冲或同时进行。范式是被动的; 不需要行为反应。为了检查整合，在四个时间窗口 (60-80 ms，80-110 ms，110-150 ms和180-220 ms) 中，将对同时进行多感觉听觉-体感刺激的反应与单感觉听觉和单感觉体感反应的总和进行了比较。根据先前的研究，在预期听觉-体感整合的头皮区域上检查了特定的中线和侧面电极部位。中线电极位点 (Fz，Cz和Pz) 在两个时间窗口期间显示出显着的集成: 60-80 ms和180-220 ms。在180和220 ms之间的时间窗，在对侧电极部位 (C3) 也发现了显着的整合。在同侧电极部位 (C4和CP6)，在任何时间窗口中均未发现显着的整合 (即，多感官ERP与汇总的单感官ERP没有显着差异)。这些结果表明，在患有SOR的儿童中可以可靠地测量MSI，并提供了证据，表明在感觉信息处理的早期和后期 (主要是在前中央头皮区域) 整合了多感觉听觉-体感输入。

BACKGROUND & AIMS: :Recent studies, conducted almost exclusively in primates, have shown that several cortical areas usually associated with modality-specific sensory processing are subject to influences from other senses. Here we demonstrate using single-unit recordings and estimates of mutual information that visual stimuli can influence the activity of units in the auditory cortex of anesthetized ferrets. In many cases, these units were also acoustically responsive and frequently transmitted more information in their spike discharge patterns in response to paired visual-auditory stimulation than when either modality was presented by itself. For each stimulus, this information was conveyed by a combination of spike count and spike timing. Even in primary auditory areas (primary auditory cortex [A1] and anterior auditory field [AAF]), approximately 15% of recorded units were found to have nonauditory input. This proportion increased in the higher level fields that lie ventral to A1/AAF and was highest in the anterior ventral field, where nearly 50% of the units were found to be responsive to visual stimuli only and a further quarter to both visual and auditory stimuli. Within each field, the pure-tone response properties of neurons sensitive to visual stimuli did not differ in any systematic way from those of visually unresponsive neurons. Neural tracer injections revealed direct inputs from visual cortex into auditory cortex, indicating a potential source of origin for the visual responses. Primary visual cortex projects sparsely to A1, whereas higher visual areas innervate auditory areas in a field-specific manner. These data indicate that multisensory convergence and integration are features common to all auditory cortical areas but are especially prevalent in higher areas.

背景与目标： : 最近的研究几乎只在灵长类动物中进行，表明通常与形态特定的感觉加工相关的几个皮质区域会受到其他感觉的影响。在这里，我们使用单单元记录和互信息估计来证明视觉刺激可以影响麻醉雪貂听觉皮层中单位的活动。在许多情况下，这些单元也具有声学响应能力，并且响应成对的视觉听觉刺激，通常以其尖峰放电模式传输更多信息，而不是单独呈现任何一种模式。对于每种刺激，此信息都是通过尖峰计数和尖峰定时的组合来传达的。即使在初级听觉区域 (初级听觉皮层 [A1] 和前听觉场 [AAF])，也发现大约15% 的记录单位具有非听觉输入。这一比例在位于A1/AAF腹侧的较高水平场中增加，在前腹侧场中最高，其中发现近50% 的单元仅对视觉刺激有反应，而对视觉和听觉刺激又有四分之一。在每个领域中，对视觉刺激敏感的神经元的纯音响应特性与视觉无反应的神经元没有任何系统的差异。神经示踪剂注射显示视觉皮层直接输入听觉皮层，表明视觉反应的潜在来源。初级视觉皮层稀疏地投射到A1，而较高的视觉区域则以特定于领域的方式支配听觉区域。这些数据表明，多感觉会聚和整合是所有听觉皮层区域的共同特征，但在较高的区域尤为普遍。

BACKGROUND & AIMS: :Responses of multisensory neurons to combinations of sensory cues are generally enhanced or depressed relative to single cues presented alone, but the rules that govern these interactions have remained unclear. We examined integration of visual and vestibular self-motion cues in macaque area MSTd in response to unimodal as well as congruent and conflicting bimodal stimuli in order to evaluate hypothetical combination rules employed by multisensory neurons. Bimodal responses were well fit by weighted linear sums of unimodal responses, with weights typically less than one (subadditive). Surprisingly, our results indicate that weights change with the relative reliabilities of the two cues: visual weights decrease and vestibular weights increase when visual stimuli are degraded. Moreover, both modulation depth and neuronal discrimination thresholds improve for matched bimodal compared to unimodal stimuli, which might allow for increased neural sensitivity during multisensory stimulation. These findings establish important new constraints for neural models of cue integration.

背景与目标： : 相对于单独呈现的单个线索，多感觉神经元对感觉线索组合的反应通常会增强或降低，但是控制这些相互作用的规则仍然不清楚。我们检查了猕猴区域MSTd中视觉和前庭自我运动提示的整合，以响应单峰，全等和冲突的双峰刺激，以评估多感觉神经元采用的假设组合规则。双峰响应通过单峰响应的加权线性和很好地拟合，权重通常小于1 (次加性)。令人惊讶的是，我们的结果表明，权重随两个线索的相对可靠性而变化: 当视觉刺激降低时，视觉权重降低而前庭权重增加。此外，与单峰刺激相比，匹配的双峰刺激的调制深度和神经元辨别阈值均得到改善，这可能允许在多感觉刺激期间增加神经敏感性。这些发现为线索整合的神经模型建立了重要的新约束。

BACKGROUND & AIMS: :Merging information derived from different sensory channels allows the brain to amplify minimal signals to reduce their ambiguity, thereby improving the ability of orienting to, detecting, and identifying environmental events. Although multisensory interactions have been mostly ascribed to the activity of higher-order heteromodal areas, multisensory convergence may arise even in primary sensory-specific areas located very early along the cortical processing stream. In three experiments, we investigated early multisensory interactions in lower-level visual areas, by using a novel approach, based on the coupling of behavioral stimulation with two noninvasive brain stimulation techniques, namely, TMS and transcranial direct current stimulation (tDCS). First, we showed that redundant multisensory stimuli can increase visual cortical excitability, as measured by means of phosphene induction by occipital TMS; such physiological enhancement is followed by a behavioral facilitation through the amplification of signal intensity in sensory-specific visual areas. The more sensory inputs are combined (i.e., trimodal vs. bimodal stimuli), the greater are the benefits on phosphene perception. Second, neuroelectrical activity changes induced by tDCS in the temporal and in the parietal cortices, but not in the occipital cortex, can further boost the multisensory enhancement of visual cortical excitability, by increasing the auditory and tactile inputs from temporal and parietal regions, respectively, to lower-level visual areas.

背景与目标： : 合并来自不同感觉通道的信息可以使大脑放大最小的信号以减少其模糊性，从而提高定向，检测和识别环境事件的能力。尽管多感觉相互作用主要归因于高阶异态区域的活动，但即使在皮质加工流早期的主要感觉特定区域，也可能出现多感觉会聚。在三个实验中，我们使用一种新颖的方法，基于行为刺激与两种非侵入性脑刺激技术 (TMS和经颅直流刺激 (tDCS)) 的耦合，研究了较低水平视觉区域中的早期多感觉相互作用。首先，我们证明了冗余的多感觉刺激可以增加视觉皮层兴奋性，这是通过枕骨TMS的phosphene诱导来测量的; 这种生理增强之后是通过在感觉特定的视觉区域中放大信号强度来促进行为。结合的感觉输入越多 (即三峰刺激与双峰刺激)，对phosphene感知的好处就越大。其次，tDCS在颞叶和顶叶皮层 (而在枕叶皮层中没有) 引起的神经电活动变化可以通过增加来自颞叶和顶叶区域的听觉和触觉输入来进一步增强视觉皮层兴奋性，分别到较低水平的视觉区域。

BACKGROUND & AIMS: :The aim of the present report was to evaluate the effectiveness and impact of multisensory and cognitive stimulation on improving cognition in elderly persons living in long-term-care institutions (institutionalized [I]) or in communities with their families (noninstitutionalized [NI]). We compared neuropsychological performance using language and Mini-Mental State Examination (MMSE) test scores before and after 24 and 48 stimulation sessions. The two groups were matched by age and years of schooling. Small groups of ten or fewer volunteers underwent the stimulation program, twice a week, over 6 months (48 sessions in total). Sessions were based on language and memory exercises, as well as visual, olfactory, auditory, and ludic stimulation, including music, singing, and dance. Both groups were assessed at the beginning (before stimulation), in the middle (after 24 sessions), and at the end (after 48 sessions) of the stimulation program. Although the NI group showed higher performance in all tasks in all time windows compared with I subjects, both groups improved their performance after stimulation. In addition, the improvement was significantly higher in the I group than the NI group. Language tests seem to be more efficient than the MMSE to detect early changes in cognitive status. The results suggest the impoverished environment of long-term-care institutions may contribute to lower cognitive scores before stimulation and the higher improvement rate of this group after stimulation. In conclusion, language tests should be routinely adopted in the neuropsychological assessment of elderly subjects, and long-term-care institutions need to include regular sensorimotor, social, and cognitive stimulation as a public health policy for elderly persons.

背景与目标： : 本报告的目的是评估多感官和认知刺激对改善长期护理机构 (制度化 [I]) 或家庭社区 (非制度化 [NI]) 老年人认知的有效性和影响。我们在24和48次刺激之前和之后使用语言和迷你精神状态检查 (MMSE) 测试分数比较了神经心理表现。两组按年龄和受教育年限进行匹配。由10名或更少的志愿者组成的小组接受了刺激计划，每周两次，超过6个月 (总共48个疗程)。会议基于语言和记忆练习，以及视觉，嗅觉，听觉和音乐刺激，包括音乐，唱歌和舞蹈。两组均在刺激计划的开始 (刺激前)，中间 (24个疗程后) 和结束 (48个疗程后) 进行评估。尽管与I受试者相比，NI组在所有时间窗口的所有任务中均显示出更高的性能，但两组在刺激后均提高了性能。此外，I组的改善明显高于NI组。语言测试似乎比MMSE更有效地检测认知状态的早期变化。结果表明，长期护理机构的贫困环境可能会导致刺激前较低的认知评分和刺激后较高的改善率。总之，语言测试应常规用于老年人的神经心理学评估，长期护理机构需要将定期的感觉运动，社交和认知刺激作为老年人的公共卫生政策。

BACKGROUND & AIMS: :The need to engage in multiple tasks simultaneously is often encountered in everyday experience, but coordinating between two or more tasks can lead to impaired performance. Typical investigations of multitasking impairments have focused on the performance of two tasks presented in close temporal proximity on discrete trials; however, such paradigms do not match well with the continuous performance situations more typically encountered outside the laboratory. As a result, the stages of information processing that are affected during multisensory continuous dual tasks and how these changes in processing relate to behavior remain unclear. To address these issues, participants were presented simultaneous rapid visual and auditory stimulus sequences under three conditions: attend visual only, attend auditory only, and dual attention (attend both visual and auditory). Performance, measured in terms of response time and perceptual sensitivity (d'), revealed dual-task impairments only in the auditory task. Neural activity, measured by the ERP technique, revealed that both early stage sensory processing and later cognitive processing of the auditory task were affected by dual-task performance, but similar stages of processing of the visual task were not. Critically, individual differences in neural activity at both early and late stages of information processing accurately rank-ordered individuals based on the observed difference in behavioral performance between the single and dual attention conditions. These results reveal relationships between behavioral performance and the neural correlates of both early and late stage information processing that provide key insights into the complex interplay between the brain and behavior when multiple tasks are performed continuously.

背景与目标： : 在日常经验中经常会遇到同时从事多个任务的需求，但是在两个或多个任务之间进行协调可能会导致绩效受损。多任务损伤的典型研究集中在离散试验中在时间上非常接近的两个任务的性能上; 但是，这种范例与实验室外通常遇到的连续性能情况不太匹配。结果，在多感官连续双重任务期间受影响的信息处理阶段以及这些处理变化与行为之间的关系尚不清楚。为了解决这些问题，在三种情况下同时向参与者展示了快速的视觉和听觉刺激序列: 仅参加视觉，仅参加听觉和双重注意 (同时参加视觉和听觉)。以响应时间和感知灵敏度 (d') 衡量的性能仅在听觉任务中显示了双重任务障碍。通过ERP技术测量的神经活动表明，听觉任务的早期感觉处理和后来的认知处理都受到双重任务性能的影响，但视觉任务的相似处理阶段却没有。至关重要的是，在信息处理的早期和后期，神经活动的个体差异会根据观察到的单个和双重注意条件之间的行为表现差异，准确地对个体进行排序。这些结果揭示了行为表现与早期和晚期信息处理的神经相关性之间的关系，这些关系提供了对连续执行多个任务时大脑与行为之间复杂相互作用的关键见解。

BACKGROUND & AIMS: :We experience the world through multiple senses simultaneously. To better understand mechanisms of multisensory processing we ask whether inputs from two senses (auditory and visual) can interact and drive plasticity in neural-circuits of the primary visual cortex (V1). Using genetically-encoded voltage and calcium indicators, we find coincident audio-visual experience modifies both the supra and subthreshold response properties of neurons in L2/3 of mouse V1. Specifically, we find that after audio-visual pairing, a subset of multimodal neurons develops enhanced auditory responses to the paired auditory stimulus. This cross-modal plasticity persists over days and is reflected in the strengthening of small functional networks of L2/3 neurons. We find V1 processes coincident auditory and visual events by strengthening functional associations between feature specific assemblies of multimodal neurons during bouts of sensory driven co-activity, leaving a trace of multisensory experience in the cortical network.

背景与目标： : 我们通过多种感官同时体验世界。为了更好地理解多感官处理的机制，我们询问来自两种感官 (听觉和视觉) 的输入是否可以相互作用并驱动初级视觉皮层 (V1) 的神经回路中的可塑性。使用遗传编码的电压和钙指标，我们发现重合的视听体验会改变小鼠v1 L2/3中神经元的超阈值和亚阈值响应特性。具体来说，我们发现在视听配对后，多模态神经元的子集对配对的听觉刺激产生增强的听觉反应。这种交叉模态可塑性持续数天，并反映在L2/3神经元小功能网络的增强上。我们发现V1通过在感觉驱动的共同活动过程中加强多模式神经元的特定功能组件之间的功能关联来处理同时发生的听觉和视觉事件，从而在皮质网络中留下了许多多感觉经验。

BACKGROUND & AIMS: :Multisensory integration of stimuli occurring in the area surrounding our bodies gives rise to the functional representation of peripersonal space (PPS). PPS extent is flexible according to the affective context and the target of an action, but little is known about how social context modulates it. We used an audiotactile interaction task to investigate PPS of individuals during social interaction. Participants had to detect as fast as possible a tactile stimulus while task-irrelevant looming sounds were presented, while they were paired as collaborative dyads and as competitive dyads. We also measured PPS in participants seated near an inactive individual. PPS boundaries were modulated only when participants collaborated with a partner, in the form of an extension on the right hemispace and independently of the location of the partner. This suggests that space processing is modified during collaborative tasks. During collective actions, a supra-individual representation of the space of action could be at stake in order to adapt our individual motor control to an interaction as a group with the external world. Reassessing multisensory integration in the light of its potential social sensitivity might reveal that low-level mechanisms are modified by the need to interact with others.

背景与目标： : 在我们身体周围区域发生的刺激的多感觉整合产生了个人周围空间 (PPS) 的功能表示。PPS范围根据情感环境和行为目标是灵活的，但对社会环境如何调节它知之甚少。我们使用了听觉互动任务来调查社交互动过程中个人的PPS。参与者必须尽可能快地检测到触觉刺激，同时呈现与任务无关的隐约可见的声音，同时将它们配对为协作二元组和竞争二元组。我们还测量了坐在不活跃个体附近的参与者的PPS。仅当参与者与合作伙伴合作时，才以右侧半球空间上的扩展形式且与合作伙伴的位置无关，才对PPS边界进行调整。这表明在协作任务期间修改了空间处理。在集体行动期间，可能会危及行动空间的超个人表示，以使我们的个人运动控制适应作为一个整体与外部世界的互动。根据其潜在的社会敏感性重新评估多感官整合可能会发现，与他人互动的需求会改变低级机制。

BACKGROUND & AIMS: :A question that has emerged over recent years is whether audiovisual (AV) speech perception is a special case of multi-sensory perception. Electrophysiological (ERP) studies have found that auditory neural activity (N1 component of the ERP) induced by speech is suppressed and speeded up when a speech sound is accompanied by concordant lip movements. In Experiment 1, we show that this AV interaction is not speech-specific. Ecologically valid nonspeech AV events (actions performed by an actor such as handclapping) were associated with a similar speeding-up and suppression of auditory N1 amplitude as AV speech (syllables). Experiment 2 demonstrated that these AV interactions were not influenced by whether A and V were congruent or incongruent. In Experiment 3 we show that the AV interaction on N1 was absent when there was no anticipatory visual motion, indicating that the AV interaction only occurred when visual anticipatory motion preceded the sound. These results demonstrate that the visually induced speeding-up and suppression of auditory N1 amplitude reflect multisensory integrative mechanisms of AV events that crucially depend on whether vision predicts when the sound occurs.

背景与目标： : 近年来出现的一个问题是，视听 (AV) 语音感知是否是多感官感知的特例。电生理 (ERP) 研究发现，当语音伴随着一致的嘴唇运动时，语音引起的听觉神经活动 (ERP的N1成分) 会受到抑制并加快。在实验1中，我们表明这种AV交互不是特定于语音的。生态上有效的非语音AV事件 (演员执行的动作，例如拍手) 与AV语音 (音节) 类似的听觉N1振幅的加速和抑制有关。实验2表明，这些AV相互作用不受A和V是否一致的影响。在实验3中，我们表明，当没有预期的视觉运动时，N1上的AV相互作用不存在，这表明AV相互作用仅在视觉预期运动先于声音时才发生。这些结果表明，视觉引起的听觉N1振幅的加速和抑制反映了AV事件的多感觉统合机制，这在很大程度上取决于视觉是否能预测声音何时发生。

BACKGROUND & AIMS: :Left parietal cortex has been associated with the human-specific ability of sophisticated tool use. Yet, it is unclear how tool information is represented across senses. Here, we compared auditory and visual tool-specific activations within healthy human subjects to probe the relation of tool-specific networks, uni- and multisensory response properties, and functional and structural connectivity using functional and diffusion-weighted MRI. In each subject, we identified an auditory tool network with regions in left anterior inferior parietal cortex (aud-aIPL), bilateral posterior lateral sulcus, and left inferior precentral sulcus, and a visual tool network with regions in left aIPL (vis-aIPL) and bilateral inferior temporal gyrus. Aud-aIPL was largely separate and anterior/inferior from vis-aIPL, with varying degrees of overlap across subjects. Both regions displayed a strong preference for tools versus other stimuli presented within the same modality. Despite their modality preference, aud-aIPL and vis-aIPL and a region in left inferior precentral sulcus displayed multisensory response properties, as revealed in multivariate analyses. Thus, two largely separate tool networks are engaged by the visual and auditory modalities with nodes in parietal and prefrontal cortex potentially integrating information across senses. The diversification of tool processing in human parietal cortex underpins its critical role in complex object processing.

背景与目标： : 左顶叶皮层与复杂工具使用的人类特异性能力有关。然而，目前尚不清楚工具信息如何跨感官表示。在这里，我们比较了健康人类受试者中听觉和视觉工具特定的激活，以使用功能和扩散加权MRI探索工具特定网络，单感觉和多感觉反应特性以及功能和结构连通性之间的关系。在每个受试者中，我们确定了一个听觉工具网络，该听觉工具网络具有左前下顶叶皮层 (aud-aIPL)，双侧后外侧沟和左下下中央前沟的区域，以及一个视觉工具网络，该视觉工具网络具有左aIPL (vis-aIPL) 的区域) 和双侧颞下回。Aud-aIPL在很大程度上与vis-aIPL分开，并且位于前/下方，在受试者之间有不同程度的重叠。与同一方式中呈现的其他刺激相比，这两个区域都表现出对工具的强烈偏好。尽管有模态偏好，但aud-aIPL和vis-aIPL以及左下心前沟中的一个区域仍显示出多感觉反应特性，如多变量分析所示。因此，视觉和听觉方式参与了两个很大程度上独立的工具网络，顶叶和前额叶皮层中的节点可能会跨感官整合信息。人类顶叶皮层工具处理的多样化是其在复杂物体处理中的关键作用。

BACKGROUND & AIMS: :To investigate how visual and vestibular cues are integrated for the perception of gravity during passive self-motion, we measured the ability to maintain a handheld object vertical relative to gravity without visual feedback during sinusoidal roll-tilt stimulation. Visual input, either concordant or discordant with actual dynamic roll-tilt, was delivered by a head-mounted display showing the laboratory. The four visual conditions were darkness, visual-vestibular concordance, stationary visual scene, and a visual scene 180 degrees phase-shifted relative to actual tilt. Tilt-indication performance using a solid, cylindrical joystick was better in the presence of concordant visual input relative to the other visual conditions. In addition, we compared performance when indicating the vertical by the joystick or a full glass of water. Subjects indicated the direction of gravity significantly better when holding the full glass of water than the joystick. Matching the inertial characteristics, including fluid properties, of the handheld object to the glass of water did not improve performance. There was no effect of visual input on tilt performance when using the glass of water to indicate gravitational vertical. The gain of object tilt motion did not change with roll-tilt amplitude and frequency (+/-7.5 degrees at 0.25 Hz, +/-10 degrees at 0.16 Hz, and +/-20 degrees at 0.08 Hz), however, the phase of object tilt relative to subject tilt showed significant phase-leads at the highest frequency tested (0.25 Hz). Comparison of the object and visual effects observed suggest that the task-dependent behavior change may be due to an attentional shift and/or shift in strategy.

背景与目标： : 为了研究在被动自我运动过程中如何将视觉和前庭提示整合在一起以感知重力，我们测量了在正弦侧倾刺激过程中保持手持物体相对于重力垂直而没有视觉反馈的能力。通过显示实验室的头戴式显示器提供了与实际动态滚动倾斜一致或不一致的视觉输入。这四个视觉条件是黑暗，视觉-前庭一致，静止的视觉场景以及相对于实际倾斜180度相移的视觉场景。与其他视觉条件相比，在存在一致的视觉输入的情况下，使用实心圆柱操纵杆的倾斜指示性能更好。此外，我们比较了通过操纵杆或一整杯水指示垂直时的性能。当拿着满杯水时，受试者指示的重力方向比操纵杆好得多。将手持物体的惯性特性 (包括流体特性) 与水杯相匹配并不能改善性能。当使用水杯指示重力垂直时，视觉输入对倾斜性能没有影响。物体倾斜运动的增益不随侧倾振幅和频率变化 (在0.25Hz时 +/-7.5度，在0.16Hz时 +/-10度，在0.08Hz时 +/-20度)，但是，相对于对象倾斜的相位在所测试的最高频率 (0.25Hz) 处显示出显著的相位超前。观察到的对象和视觉效果的比较表明，与任务相关的行为变化可能是由于注意力的转移和/或策略的转移。