Processing of Binaural Information in Human Auditory Cortex 427 perception. The experiments reviewed here show that EEG and MEG responses to DP consist of a sequence of auditory cortical responses that provide important markers of a number of functionally distinct stages of auditory scene analysis in the human brain.: (1) The M100 ERF seems to reflect the operation of right-hemispheric mechanisms for analysis of spatial information pitted against left hemisphere mechanisms for analysis of timing information; (2) The ORN ERP and ERF reflect the operation of fairly automatic and generalized brain mechanisms for auditory scene segregation. The ORN mechanisms can broadly draw on information about scene analysis from a variety of acoustic cues, including inharmonicity, ITDs, and ILDs. As such, the ORN appears to represent a stage of auditory processing that draws on information extracted from disparate cues into a common code that can be used to solve the broad perceptual problems of auditory scene analysis. (3) The P400 ERP is an electrophysiological signpost of a later, more controlled stage of processing, involving identification and generation of a behavioural response. This stage is highly dependent on the task and context in which stimuli are presented. (4) The N2 ERP recorded at lateral sites over the temporal lobes is highly sensitive to the spatial attributes of dichotic pitch, suggesting that this component reflects a location-specific phase of neural processing. The N2 has not been observed in MEG responses, likely because the generators have a radial orientation that the MEG is relatively less sensitive to than EEG. Future work can leverage these electrophysiological markers to gain clearer insights into clinical conditions in which one or more of these important central processing stages may have gone awry. For example, psychophysical studies have reported that DP detection is significantly impaired in individuals with developmental dyslexia compared to normal readers (e.g. Dougherty et al., 1998). A current study in our laboratory is measuring concurrent EEG-MEG responses to DP in dyslexic and normal reading children (Johnson et al., submitted), to determine if auditory processing deficits in reading impaired children can be localized to one or more of the processing stages delineated in studies of healthy adults. 8. Acknowledgements The MEG work described in this chapter was supported by Australian Research Council Linkage Infrastructure Equipment and Facilities Grant LEO668421. The author gratefully acknowledges the collaboration of Professor Stephen Crain, the Kanazawa Institute of Technology and Yokogawa Electric Corporation in establishing the KIT-Macquarie MEG laboratory. 9. References Alain, C. (2007). Breaking the wave: effects of attention and learning on concurrent sound perception. Hearing Research, 229, 1-2., (July 2007) 225-236, 0378-5955 (Print). Alain, C., & Izenberg, A. (2003). Effects of attentional load on auditory scene analysis. Journal of Cognitive Neuroscience, 15, 7, 1063-1073. 0898-929X (Print) 1530-8898 (Electronic) Alain, C., Schuler, B. M., & McDonald, K. L. (2002). Neural activity associated with distinguishing concurrent auditory objects. Journal of the Acoustical Society of America, 111, 990-995, 0001-4966 (Print) 1520-8524 (Electronic). Bell, A. J., & Sejnowski, T. J. (1995). An information-maximization approach to blind separation and blind deconvolution. Neural Computation, 7, 6, 1129-1159, 0899-7667 (Print). Advances in Sound Localization 428 Bilsen, F. A. (1976). Pronounced binaural pitch phenomenon. Journal of the Acoustical Society of America, 59, 2, 467-468, 0001-4966 (Print) Blauert, J. (1997). Spatial hearing: The psychophysics of human sound localization, MIT Press, 0- 262-02413-6 , Cambridge, MA. Bregman, A. (1990). Auditory scene analysis: The perceptual organization of sound, MIT Press, 0- 262-52195-4 , Cambridge, MA. Cramer, E., & Huggins, W. (1958). Creation of pitch through binaural interaction. Journal of the Acoustical Society of America, 30, 413-417, 0001-4966 (Print) 1520-8524 (Electronic). Dougherty, R. F., Cynader, M. S., Bjornson, B. H., Edgell, D., & Giaschi, D. E. (1998). Dichotic pitch: a new stimulus distinguishes normal and dyslexic auditory function. Neuroreport, 9(13), 3001-3005, 0959-4965 (Print) 1473-558X (Electronic). Drennan, W. R., Gatehouse, S., & Lever, C. (2003). Perceptual segregation of competing speech sounds: the role of spatial location. Journal of the Acoustical Society of America, 114, 2178-2189, 0001-4966 (Print). Dyson, B. J. & Alain, C. (2004). Representation of concurrent acoustic objects in primary auditory cortex. Journal of the Acoustical Society of America, 115, 280-288, 0001- 4966 (Print). Erikson, M., & McKinley, R. (1997). The intelligibility of multiple talkers separated spatially in noise. In R. Gilkey & T. Anderson (Eds.), Binaural and Spatial Hearing in Real and Virtual Environments (pp. 701-724). New Jersey, 13: 978-080581654, Lawrence Erlbaum. Goldberg, J. M. & Brown, P. B. (1969). Response of binaural neurons of dog superior olivary complex to dichotic tonal stimuli: some physiological mechanisms of sound localization. Journal of Neurophysiology, 32, 613-636, 0022-3077 (Print) 1522-1598 (Electronic). Griffiths, T. D., & Warren, J. D. (2002). The planum temporale as a computational hub. Trends in Neurosciences, 25, 348-353, 0166-2236. Hafter, E. R., & Carrier, S. C. (1972). Binaural interaction in low-frequency stimuli: the inability to trade time and intensity completely. Journal of the Acoustical Society of America, 51, 6, 1852-1862, 0001-4966 (Print). Hafter, E. R. & Jeffress, L. A. (1968). Two-image lateralization of tones and clicks. Journal of the Acoustical Society of America, 44, 2, 563-569, 0001-4966 (Print). Harris, G. (1960). Binaural interactions of impulsive stimuli and pure tones. Journal of the Acoustical Society of America, 32, 685-692, 0001-4966 (Print). Hautus, M. J., & Johnson, B. W. (2005). Object-related brain potentials associated with the perceptual segregation of a dichotically embedded pitch. Journal of the Acoustical Society of America, 117, 275-280, 0001-4966 (Print). Johnson, B. W., Hautus, M., & Clapp, W. C. (2003). Neural activity associated with binaural processes for the perceptual segregation of pitch. Clinical Neurophysiology, 114, 2245- 2250, 1388-2457 (Print) 1872-8952 (Electronic). Johnson, B. W., & Hautus, M. J. (2010). Processing of binaural spatial information in human auditory cortex: neuromagnetic responses to interaural timing and level differences. Neuropsychologia, 48, 2610-2619, 0028-3932 (Print) 1873-3514 (Electronic). Johnson, B. W., Hautus, M. J., Duff, D. J., & Clapp, W. C. (2007). Sequential processing of interaural timing differences for sound source segregation and spatial localization: Processing of Binaural Information in Human Auditory Cortex 429 evidence from event-related cortical potentials. Psychophysiology, 44, 541-551, 0048- 5772 (Print) 1540-5958 (Electronic). Johnson, B.W., McArthur, G., Hautus, M., Reid, M., Brock, J., Castles, A., Crain, S. (submitted). Development of lateralized auditory brain function and binaural processing in children with normal reading ability and in children with dyslexia. Julesz, B. (1971). Foundations of Cyclopean Perception, University of Chicago Press, 0-262- 10113-0, Chicago. Kutas, M., Van Petten, C. & Kluender, R. (2006). Psycholinguistics Electrified II: 1994-2005. In: Handbood of Psycholinguistics, M. Traxler & M. Gernsbacher (Eds.) (2nd ed.), (659- 724), Elsevier, 0-12-369374-8, New York. Liegois-Chauvel, C., Musolino, A., & Chauvel, P. (1991). Localization of the primary auditory areas in man. Brain, 114, 139-153, 0006-8950 (Print) 1460-2156 (Electronic). McGee, T., Kraus, N., Littman, T., & Nicol, T. (1992). Contribution of the medial geniculate body subdivision to the middle latency response. Hearing Research, 61, 147-152, 0378-5955 (Print). Palomäki, K. J., Tiitinen, H., Mäkinen, V., May, P. J., & Alku, P. (2005). Spatial processing in human auditory cortex: the effects of 3D, ITD, and ILD stimulation techniques. Cognitive Brain Research, 24, 364-379, 0926-6410 (Print). Pantev, C., Bertrand, O., Eulitz, C., Verkindt, C., Hampson, S., Schuierer, G., et al. (1995). Specific tonotopic organizations of different areas of the human auditory cortex revealed by simultaneous magnetic and electric recordings. Electroencephalography and Clinical Neurophysiology, 94, 26-40, 0013-4694 (Print) 0424-8155 (Electronic). Phillips, D., & Brugge, J. (1985). Progress in the neurobiology of sound direction. Annual Review of Psychology, 36, 245-274, 0066-4308 (Print) 1545-2085 (Electronic). Phillips, D. P. (1993). Representation of acoustic events in the primary auditory cortex. Journal of Experimental Psychology: Human Perception and Performance, 19, 203-216, 0096-1523 (Print) 1939-1277 (Electronic). Picton, T. W., Alain, C., Woods, D. L., John, M. S., Scherg, M., Valdes-Sosa, P., et al. (1999). Intracerebral sources of human auditory-evoked potentials. Audiology & Neurotology, 4, 64-79, 1420-3030 (Print). Pratt, H., Polyakov, A., & Kontorovich, L. (1997). Evidence for separate processing in the human brainstem of interaural intensity and temporal disparities for sound lateralization. Hearing Research, 108, 1-8, 0378-5955 (Print) 1878-5891 (Electronic). Rauschecker, J. P., & Tian, B. (2000). Mechanisms and streams for processing of "what" and "where" in auditory cortex. Proceedings of the National Academy of Sciences of the United States of America, 97, 11800-11806, 0027-8424 (Print). Rayleigh, L. J. (1907). On our perception of sound direction. Philosophical Magazine (Series 6), 13, 74, 214-232, 1941-5982 (Print). Scherg, M., Vajsar, J., & Picton, T. W. (1986). A source analysis of the late human auditory evoked potentials. Journal of Cognitive Neuroscience, 1, 326-355, 0898-929X (Print) 1530-8898 (Electronic). Scherg, M., & Von Cramon, D. (1986). Evoked dipole source potentials of the human auditory cortex. Electroencephalography and clinical Neurophysiology, 65, 344-360. 0013-4694 (Print). Schnupp, J., & Carr, C. (2009). On hearing with more than one ear: lessons from evolution. Nature Neuroscience, 12(6), 692-697. 0022-3077. Advances in Sound Localization 430 Schroger, E. (1996). Interaural time and level differences: Integrated or separated processing? Hearing Research, 96, 191-198, 0378-5955 (Print) 1878-5891 (Electronic). Smith, P. H., Joris, P. X., & Yin, T. C. (1993). Projections of physiologically characterized spherical bushy cell axons from the cochlear nucleus of the cat: evidence for delay lines to the medial superior olive. Journal of Comparative Neurology, 331(2), 245-260, 0021-9967 (Print). Spierer, L., Bellmann-Thiran, A., Maeder, P., Murray, M. M., & Clarke, S. (2009). Hemispheric competence for auditory spatial representation. Brain, 132(Pt 7), 1953- 1966. 1460-2156 (Electronic). Tardif, E., Murray, M. M., Meylan, R., Spierer, L., & Clarke, S. (2006). The spatio-temporal brain dynamics of processing and integrating sound localization cues in humans. Brain Research, 1092, 161-176. 0006-8993 (Print). Thiran, A. B., & Clarke, S. (2003). Preserved use of spatial cues for sound segregation in a case of spatial deafness. Neuropsychologia, 41, 1254-1261. 0028-3932. Ungan, P., Yagcioglu, S., & Goksoy, C. (2001). Differences between the N1 waves of the responses to interaural time and intensity disparities: scalp topography and dipole sources. Clinical Neurophysiology, 112, 485-498, 1388-2457 (Print) 1872-8952 (Electronic). Ungan, P., Yagcioglu, S., & Ozmen, B. (1997). Interaural delay-dependent changes in the binaural difference potential in cat auditory brainstem response: implications about the origin of the binaural interaction component. Hearing Research, 106, 66-82, 0378- 5955 (Print) 1878-5891 (Electronic). Wagner, H. (2004). A comparison of neural computations underlying stereo vision and sound localization. Journal of Physiology Paris, 98, 135-145. 0928-4257 (Print). Werner-Reiss, U., & Groh, J. M. (2008). A rate code for sound azimuth in monkey auditory cortex: Implications for human neuroimaging studies. Journal of Neuroscience, 28, 3747-3758. 0270-6474. Wright, B. A., & Fitzgerald, M. B. (2001). Different patterns of human discrimination learning for two interaural cues to sound-source location. Proceedings of the National Academy of Sciences USA, 98, 12307-12312, 0027-8424 (Print). Yamada, K., Kaga, K., Uno, A., & Shindo, M. (1996). Sound lateralization in patients with lesions including the auditory cortex: comparison of interaural time difference (ITD) discrimination and interaural intensity difference (IID) discrimination. Hearing Research, 101, 173-180, 0378-5955 (Print). Yin, T., & Kuwada, S. (1984). Neuronal mechanisms of binaural interaction. In G. Edelman (Ed.), Dynamic Aspects of Neocortical Function. New York, 0471805599, Wiley. Yin, T. C., & Chan, J. C. (1990). Interaural time sensitivity in medial superior olive of cat. Journal of Neurophysiology, 64, 465-488. 1522-1598 (Electronic), 0022-3077 (Print). Yost, W. A. (1991). Thresholds for segregating a narrow-band from a broadband noise based on interaural phase and level differences. Journal of the Acoustical Society of America, 89, 838-844. 0001-4966 (Print). 23 The Impact of Stochastic and Deterministic Sounds on Visual, Tactile and Proprioceptive Modalities J.E. Lugo, R. Doti and J. Faubert Visual Psychophysics and Perception Laboratory, School of Optometry, University of Montreal, C.P. 6128 succ. Centre Ville, Montréal, Quebéc, H3C3J7 Canada 1. Introduction Stimulus localization and particularly directional hearing can be considered as methods for investigating neural activity and they have proven to be useful tools for research in physiology and psychology. Human directional hearing techniques have been reflected upon way back by Von Békésy in Austrian forests [1]. For example, he observed that some of the roads took a perfectly straight course through deep, dark woods. He could not imagine how such straight roads had been cut through the forest when the usual optical methods used by road surveyors would seem to be useless in this case. Further some of these roads were very old and probably built before the introduction of the theodolite. Many of these roads were laid out by an acoustic method. How did they do it? A man stationed at the starting point noted the direction of the sound produced by someone at the other end blowing a horn. The first man then walked toward the sound source, marking the threes on the way. It turned out that this method produced a straight line from start to finish [1]. From this observation Békésy was motivated to perform a series of studies on stimuli localization not limited to hearing but also to vibration sensations on the skin, electrical pulses on the tongue and odors through the nose as well. Strikingly, his results showed an underlying ubiquitous mechanism present in the different stimuli localization modalities. For instance, the effect on localization of the time delay between two stimuli on the skin, the tongue, the two nostrils in the nose and the two ears, presented the same dynamics [2-4]. These results were quite exciting because it showed that, in humans, the senses work similarly for stimuli localization although the basic underlying neural pathways are not the same. It was this kind of general principle on stimuli localization that motivated us in the search for more general principles related to how senses interact to generate multisensory perceptions but with a special emphasis on auditory stimulation. This is known as multisensory integration and its study is very important because it is the foundation of how humans bind all the information coming from the senses to generate a coherent percept. We began by studying something that we called cross-modal stochastic resonance. This consists Advances in Sound Localization 432 in the concurrence of a threshold, a subthreshold stimulus present in one sense and noise at different amplitudes entering through another sense. What we found was that the same auditory noise can enhance the sensitivity of tactile, visual and propioceptive system responses to weak signals. Specifically, we showed that the effective auditory noise significantly increased tactile sensations of the finger, decreased luminance and contrast visual thresholds and significantly changed EMG recordings of the leg muscles during posture maintenance [5]. We also found that in all the cases the interactions follow the same sort of physical dynamics. Moreover, we unveil that the same result is obtained if we use auditory deterministic sounds instead of auditory noise [6] to enhance tactile sensations. We further demonstrated that we could use tactile noise and enhance visual detection [7] or use visual deterministic signals to enhance tactile detection [6]. These surprising results guided us to propose that these multisensory integration interactions can be explained under the same general principle that we call the Fulcrum principle. In this chapter we present material emerging from our own research experience concerning human perception in general with emphasis in auditory interactions. We introduce in an accessible way a non-linear mathematical model supporting our hypothesis, and we provide experimental results and conclusions. We also propose that the Fulcrum principle may have numerous implications in a number of neurobiological alterations such as autism, aging and age-related neurodegenerative disorders and ADHD. We conclude by presenting to the readers with what we consider could be the next hurdles in this area, and the main points that we think should be emphasized in future work. 2. Multisensory Integration: MI A general description: MI is a non linear process that binds information from all the participating sensory stimuli. The original approach shows that MI results from the brain’s capacity for integrating information originating from more than a single sensory stimulus. Here we would like to present the two stimuli conditions allowing us to introduce the mathematical model. The first aspect involves the concept of Signal Coherence, and the second important aspect is the Sense Threshold for those signals [8]. Coherence is intended to be the propriety that gives the signal a continuous and repetitive harmonic shape. A signal involves the concept of evolution in the time domain, harmonic shape implies the same amplitude at regular time intervals, and very importantly, the same amount of energy transferred per unit of time [9]. If we have more than one stimulus applied to a big surface interface, we can split this concept in two: Temporal Coherence (frequency) and Spatial Coherence (front- wave) Temporal Coherence: when we consider the coexistence of more than one stimulus signal, the coherence associated with this compound stimulus is the correlation (proportional correspondence) between the evolutions in the time domain for both signals (together). When the signals are periodic this represents the same frequency spectrum content and results in the same bandwidth (BW) [10]. In the case of a pure tone, we would have only one frequency component in the signal spectrum. Spatial Coherence: if for a fixed point in space along the signals pass the superposition of these simultaneous signals presents Temporal Coherence, we say that signals have spatial coherence. The front –wave of this compound signal preserves the shape along its pass (when traveling along an ideal non dispersive mean). The Impact of Stochastic and Deterministic Sounds on Visual, Tactile and Proprioceptive Modalities 433 Examples of periodic signals So, depending on the intensity and characteristic of the stimulus signal we can have different situations. For instance, for a given perceptual threshold level we can have: supra- threshold (perceived signal), or sub-threshold (not perceived) stimuli. Depending on the stability and consistency of the signal stimuli we can have deterministic signals (coherent or not) or stochastic signals. Deterministic signals always present a limited bandwidth or a repetitive pattern. They can be described and recreated without error along the time domain. We know the evolution of the instantaneous energy transferred trough these signals. Periodic signals means a fixed frequency spectrum content or a fixed bandwidth (BW).For a pure tone, we have a narrow frequency s p ectru m Advances in Sound Localization 434 Stochastic signals represent a random pattern and a very large bandwidth. We can establish the limits of their characteristics (amplitude or BW), but we do not know in advance their evolution along the time domain. We know the mean energy transferred trough these signals. A good example of a Stochastic Signal is White Noise [11]. Example of a deterministic signal with noise Because of the random instantaneous frequency content compared with a pure tone, we call it NOISE. As its Frequency Spectrum extends from zero Hz to infinite, we call it WHITE (in analogy with the visible spectrum and the eyes perception of the white light). 3. The Inverse-effectiveness law So far we defined the MI as the complex way in which our brain binds the different sensory stimuli that contributes to create a phantom image of the real world outside its perceptual limits. This image is the only reality we have. Researchers tried to define the human sensory stimulus span from threshold to ceiling. They tested humans applying deterministic stimuli signals to the different senses. This generated normalized thresholds for auditory, tactile, visual, etc. Here we find the first cue in reference to MI: it was determined that if two weak (close to threshold level) stimuli are applied together, the presence of the additional stimulus facilitates perception. And this happens for an elastic temporal coincidence. But, this perceptual improvement is not possible if one of the stimuli is clearly supra-threshold. This Deterministic signals always present a limited bandwidth or a repetitive pattern Stochastic signals present a random pattern and a very large bandwidth The Impact of Stochastic and Deterministic Sounds on Visual, Tactile and Proprioceptive Modalities 435 is known as: Inverse-Effectiveness Law [12]. This means that perceptual enhancement takes place trough the MI mechanism when we apply: weak, supra-threshold, deterministic and coincident signals to the subject. However, there is an MI phenomenon that cannot be described by the inverse-effectiveness rule: cross-modal SR. 4. Stochastic resonance Stochastic resonance (SR) [13] is a nonlinear phenomenon whereby the addition of noise can improve the detection of weak stimuli. An optimal amount of added noise results in the maximum enhancement, whereas further increases in noise intensity only degrade detection or information content. The phenomenon does not occur in linear systems, where the addition of noise to either the system or the stimulus only degrades the measures of signal quality. The SR phenomenon was thought to exist only in stochastic, nonlinear, dynamical systems but it also exists in another form referred to as ‘threshold SR’ or ‘non-dynamical SR’. This form of stochastic resonance results from the concurrence of a threshold, a subthreshold stimulus, and noise. These ingredients are omnipresent in nature as well as in a variety of man-made systems, which accounts for the observation of SR in many fields and conditions. The SR signature is that the signal-to-noise ratio, which is proportional to the system’s sensitivity, is an inverted U-like function of different noise levels. That is, the signal-to-noise ratio first is enhanced by the noise up to a maximum and then lessened. The SR phenomenon has been shown to occur in different macro [14], micro[15] and nano physical systems [16]. From the cyclic recurrence of ice ages, bistable ring lasers, electronic circuits, superconducting quantum interference devices (SQUIDs) and neurophysiological systems [17] such as receptors in animals. Several studies have suggested that the higher central nervous system might utilize the noise to enhance sensory information [13]. SR studies in humans can be divided in unimodal SR (signal and noise enter the same sense) [18,19], central SR (signal and noise enters in similar local receptors and later mix in the cortex) [20] and behavioral SR (similar to central SR but its effect is observed in one sense and then enacted in the behavior of the subjects) [21]. Before the SR principle was proposed, Harper [22] discovered what we currently would call crossmodal stochastic resonance while studying the effect of auditory white noise on sensitivity to visual flicker. Recently a similar result [23] has been found where auditory noise produces SR when subthreshold luminance stimuli are present. However what has not been explored is the extension of these interactions in humans. New results show that the noise induces large scale phase synchronization of human-brain activity associated with behavioral SR [24]. It is shown that both detection of weak visual signals to the right eye and phase synchronization of electroencephalogram (EEG) signals from widely separated areas of the human brain are increased by addition of weak visual noise to the left eye. These results imply that noise- induced large-scale neural synchronization may play a significant role in information transmission in the brain. Interestingly SR can be seen as a synchronization-like phenomenon between two energy states of a physical system for example [25]. Furthermore, the synchronization-like phenomenon plays a key role in the enhancement of the signal-to- noise ratio in SR. Therefore, we can hypothesized that if the noise induced large scale phase synchronization in different areas of the cortex and peripheral systems with dynamics similar to SR, the crossmodal SR would be a ubiquitous phenomenon in humans because it involves different cortical areas and peripheral systems. Consequently under the same auditory noise conditions, the crossmodal SR should be present among tactile, visual and proprioceptive sensory systems, for instance. Advances in Sound Localization 436 5. Facilitating and excitatory stimulus In order to outline a synoptic scheme that represents the basis of some experiments that we have performed, we introduce another two concepts. First, Excitatory Stimulus: signal applied to the sense that we want to study. Second, Facilitating Stimulus: signal applied simultaneously to the same subject, intended to trigger the MI mechanism in a way that facilitates the perception of the Excitatory Stimulus. When both, facilitating and excitatory signals act as stimuli of the same sense (auditory, tactile, visual stimulus, etc) we have Uni- modal Interactions (U.M). When each one of these signals act in different senses (for instance excitatory: tactile; and facilitating: auditory) we have Cross-modal Interactions (C.M). Either of the precedent cases are part of the general Multi-modal Interactions model. 6. Crossmodal interactions paradigms and the sensory threshold enhancement On the basis of what was presented so far, it is possible to combine those elements to create the experiments that allow us to explore human perception and outline a plausible model. All of them allow a positive response from the subject under test, by the action of the facilitating stimulus, when the excitatory stimulus is Sub threshold. This means an improvement of the human perception. Examples of multimodal interactions that have been tested so far are: 1. Excitatory: Tactile – Deterministic- threshold E:T-D- T Facilitating: Auditory or Visual -Deterministic – threshold F: AoV-D-T 2. Excitatory: Tactile – Deterministic- Sub threshold E:T-D-ST Facilitating: Auditory - Stochastic - Supra threshold F: A-S- SST 3. Excitatory: Visual – Deterministic- Sub threshold E:V-D-ST Facilitating: Auditory - Stochastic - Supra threshold F: A-S- SST 4. Excitatory: Propioception – Deterministic- Sub threshold E:P-D-ST Facilitating: Auditory - Stochastic - Supra threshold F: A-S- SST 5. Excitatory: Visual – Deterministic- Sub threshold E:V-D- ST Facilitating: Tactile - Stochastic - Supra threshold F: T-S- SST 6. Excitatory: Tactile – Deterministic- Sub threshold E:T-D-ST Facilitating: Auditory - Deterministic - Supra threshold F: A-D- SST 7. Excitatory: Tactile – Deterministic- Sub threshold E:T-D-ST Facilitating: Visual - Deterministic - Supra threshold F: V-D- SST We observe that 1 is a cross modal example of the Inverse Effectiveness Law (IEL). These kinds of examples have been studied massively and they are well documented on the literature [12]. 2 to 5 belong to the Multi modal Stochastic Resonance (MmSR) and 6 and 7 belong to the Multi modal Deterministic Resonance (MmDR). In what follows we will explain more in detail these multimodal interactions. Excitatory: Tactile – Deterministic- Sub threshold E:T-D-ST Facilitating: Auditory - Stochastic - Supra threshold F: A-S- SST In the first series of experiments we studied the effects of auditory noise on tactile sensations in three subjects. Tactile vibrations were delivered to the middle finger of the right hand of the subjects at a frequency of 100Hz and were asked to report the tactile sensation. If they felt the signal they had to click on a yes button or on a no button otherwise (yes-no paradigm). Each subject was tested twice for every auditory noise and baseline condition. In [...]... arranged in four rows and 50 metal pins placed in a container In every trial subjects were asked to pick these 50 pins with their right hand, one by one, from the container and insert them into 50 holes on the peg-board Every hole is inserted by one pin If one pin is dropped during the transfer, they were instructed to pick the next pin from the container to insert the hole that they just failed The dependent... graph in the second column shows the integral SNR (left y-axis) for three subjects 448 Advances in Sound Localization Finally, in the last experiment, we tested tactile-auditory interaction using deterministic auditory signals with different amplitudes and measured EMG activation in 1 subject (S4) A different amplitude of the auditory signal was tested at each session The six amplitudes were 0, 8, 12, ... is indeed a major trend in solid mechanics research A great effort is also been directed in scaling laws able to predict the behaviour of materials in components of different size, trying to model the sample-size effects of damage, structural failure and other properties existing in real material systems (Krajcinovic, 1996; Krajcinovic & Rinaldi, 2005, Carpinteri & Lacidogna, 2008) 6 Approaches in. .. threshold is the point where the noise hinders the signal detection and the sensitivity worsens to levels above threshold (the crossing point in the inverse u-shape curve) If we use this level as our reference instead of the SPL absolute scale (we will call this level the noise ceiling level that defines a ceiling decibel dBc) then we found that crossmodal SR threshold minima occur approximately in the same... Standards Institute (Standard 3.1-1991) for permissible ambient noise levels (in one-third-octave bands) for testing in free-field conditions with headphones During the experimental trials, all subjects were seated and 446 Advances in Sound Localization were asked to listen to the sound in the headphones and report when they first felt a tactile sensation Once the subjects reported a change in tactile... internal noise for luminance and contrast-modulated stimuli detection Journal of Vision, 6(4): 322334 [28] Allard R, Faubert J (2007) Double dissociation between first- and second-order processing Vision Research, 47(9): 1129 -1141 [29] Priplata A A, Niemi J B, Harry J D, Lipsitz L A, Collins J J (2003) Vibrating insoles and balance control in elderly people The Lancet 362: 1123 - 1124 458 Advances in. .. stimuli At the beginning SR was thought as a local peripheral effect But evidence has confirmed the ubiquitous influence of the facilitating stimulus by triggering a mechanism that involves the cortex acting upon the peripheral sensory activity And this mechanism was shown for both, stochastic and deterministic supra threshold facilitating signals as part of a general principle for the stimuli interactions... displayed 444 Advances in Sound Localization Fig 5 Interactions between tactile noise and first order visual signals Normalized visual threshold changes with the noise level in seven subjects for luminance modulated (first order) stimuli In all the graphs the no-noise condition is taken as baseline; the black dots indicate the probability to replicate the same result (right y-axis) and the broken line represents... best describes the fundamental principle at work in these The Impact of Stochastic and Deterministic Sounds on Visual, Tactile and Proprioceptive Modalities 449 multisensory interactions The principle can be summarized as follows: a subthreshold excitatory signal (entering in one sense) that is synchronous with a facilitation signal (entering in a different sense) can be increased (up to a resonant-like... Since we are using auditory noise, one might argue that 70 dBSPL (clearly audible) could be judged annoying by some people (although previous crosmodal SR claims have shown that this is the effective range [23]) Indeed sound annoyance is a complex thing and no single level can be pointed to as a threshold for it, there are reports of high levels of annoyance for very soft sounds indeed (e.g 35dBA sound . W. C. (2007). Sequential processing of interaural timing differences for sound source segregation and spatial localization: Processing of Binaural Information in Human Auditory Cortex 429. lateralization in patients with lesions including the auditory cortex: comparison of interaural time difference (ITD) discrimination and interaural intensity difference (IID) discrimination. Hearing. Fulcrum principle. In this chapter we present material emerging from our own research experience concerning human perception in general with emphasis in auditory interactions. We introduce in an