Although our primitive prediction seemed to be correct, we did not think our investigation was systematic (or beautiful) enough because the effect of the temporal gap was checked only in conditions where the gap was in the shorter glide. Our next step was to shift the temporal gap from the shorter to the longer glide, and now the gap was in the middle of the glide that had been continuous. Thus, we obtained the stimulus patterns in File_4.wav (stimulus patterns (b) and (d) in Figure 1). Surprisingly, the change of the physical position of the gap did not cause a substantial change of the percept as indicated in the lower panel of Figure 1. Although the temporal gap existed physically in the longer frequency glide, it was perceived as if it had been in the shorter frequency glide. We named this perceptual phenomenon "the gap transfer illusion."
2. Auditory subevents
We needed a certain amount of time to publish a journal paper to report the gap transfer illusion (Nakajima et al. [2]). One of the reasons was that we had to adopt phenomenological procedures, which often do not meet the standards of modern scientific journals. We analyzed phenomenological data so that the continuity of each perceptual component corresponding to a frequency glide could be dealt with as if it had been a psychophysical judgment.
In order to explain the illusion, we assumed that the onsets and the offsets (terminations) of glides are detected as if they were independent perceptual elements, and that the detected onsets and offsets are integrated according to the proximity principle in time and frequency. If an onset and an offset are connected with each other in this order, they should make the percept of an auditory event. We call the percepts of onsets and offsets "auditory subevents" because these units and "fillings" and "silences" comprise auditory streams and auditory events. An auditory stream can be defined as a linear string of auditory events and silences. Auditory subevents cannot be connected in arbitrary order to make a coherent auditory stream, and there should be a kind of grammar providing the perceptual system with rules to connect perceptual elements.
Our tentative hypothesis is that, if an onset and an offset are close to each other in a subjective space corresponding to time and frequency, they are likely to be connected. Stimulus patterns (b) and (c) in Figure 1 are almost the same if we look at only the 500 ms temporal ranges around the temporal gaps. However, the ascending component is perceived as continuous in pattern (b), whereas the descending component is perceived as continuous in pattern (c). The difference should be determined outside these temporal ranges, and the gap transfer illusion, which takes place in pattern (b) should not be caused by a peripheral process. In pattern (b), the onset of the descending glide and the offset immediately before the temporal gap of the ascending glide are close to each other, and they are likely to be connected, perceptually building up the first short tone in the typical percept indicated in the left half of the lower panel of Figure 1. The onset immediately after the temporal gap and the offset of the descending glide are also likely to be connected to each other building up the second short tone. Because the offset and the onset delimiting the gap have already been interpreted perceptually, the auditory system need not interpret them any more, thus leaving the ascending component continuous. This is the essence of "the event construction model" [2].
3. New stimulus patterns
Turning now to future research, there are a couple of points to keep in mind. The gap transfer illusion takes place in a stable manner when the stimulus patterns are presented through headphones, but sometimes fails in a reverberant room, especially when the listener tries to hear the continuity of the shorter glide. File_5.wav contains stimulus patterns (a) and (b) in Figure 1 with an artificial reverberation, and File_6.wav stimulus patterns (c) and (d) with the same reverberation. The reverberation is indicated in File_7.wav as an impulse response. The illusion, which should take place in patterns (b) and (d), does not seem to occur very easily. My tentative explanation is that the reverberation makes the timbres of the ascending and the descending components different, and that it is now easier for the auditory system to follow them as separate streams. It is also possible that the reverberation simply makes the details of the patterns unclear. We would need further investigation of auditory continuity to tackle the problem (Ciocca & Bregman [6]; Warren [7]; Darwin [8]).
I have also made demonstrations to present ascending and descending frequency glides to different ears. File_8.wav presents stimulus patterns (a) and (b) in this manner, and File_9.wav stimulus patterns (c) and (d). The listener's attitude seems to affect the occurrence of the illusion in patterns (b) and (d). If one tries to hear the ascending and the descending component separately at the two ears, the illusion is not likely to take place. If one tries to grasp the whole pattern holistically neglecting presentation details, the illusion can occur. The crossing pattern as made by Halpern [4] (see also [3] and [5]) can also be presented in the same manner. The pattern in File_1.wav is converted in such a way in File_10.wav. If one pays attention to the signal presented to one of the ears, a crossing percept is likely to take place, while with holistic listening a bouncing percept is likely to take place. This kind of dichotic listening paradigm may give new insights into auditory organization if employed together with the classic auditory illusions Deutsch [9] discovered.
Now my colleagues and I are attempting to connect the event construction model, which can relate several different auditory phenomena systematically [10], [11], with speech perception on the assumption that speech perception is not so special [11]. On the other hand, our recent analyses of Japanese and English speech on an acoustic basis revealed a specialty of speech signals [12]. Some perceptual experiments to fill the gap are urgently needed.
Acknowledgments
This study was supported by Japan Society for the Promotion of Science Grant 14101001 and the Japanese government Center of Excellence program Design of Artificial Environments on the Basis of Human Sensibility at Kyushu University. Jonathan Goodacre and Kazuo Ueda gave me valuable comments.
[1] Y. Nakajima and T. Sasaki, gPerceptual transfer of onsets and offsets between glide tone components [in Japanese, abstract in English],h Tech. Rep. No. SP92-146, pp. 73-80, (Institute of Electronics, Information and Communication Engineers, Tokyo, 1993).
[2] Y. Nakajima, T. Sasaki, K. Kanafuka, A. Miyamoto, G. Remijn and G. ten Hoopen, g Illusory recouplings of onsets and terminations of glide tone components,h Percept. Psychophys., 62, 1413-1425 (2000).
[3] A. S. Bregman, Auditory Scene Analysis: The Perceptual Organization of Sound (MIT Press, Cambridge, Mass., 1990).
[4] L. Halpern, gThe effect of harmonic ratio relationship on auditory stream segregation,h Unpublished research report. McGill University, Psychology Department (1977).
[5] A. S. Bregman and P. A. Ahad, Demonstrations of Auditory Scene Analysis: The Perceptual Organizations of Sound (CD)(MIT Press, Cambridge, MA, 1995).
[6] V. Ciocca and A. S. Bregman, gPerceived continuity of gliding and steady-state tones through interrupting noise,h Percept. Psychophys., 42, 476-484 (1987).
[7] R. M. Warren, Auditory Perception: A New Analysis and Synthesis (Cambridge University Press: New York, 1999).
[8] C. J. Darwin, gSimultaneous grouping and auditory continuity,h Percept. Psychophys., 67, 1384-1390 (2005).
[9] D. Deutsch, Musical Illusions and Paradoxes (CD)(Philomel Records, La Jolla, CA, 2003).
[10] G. B. Remijn and Y. Nakajima, gThe Perceptual integration of auditory stimulus edges: An illusory short tone in stimulus patterns consisting of two partly overlapping glides,h J. Exp. Psychol.: Hum. Percept. Perform., 31, 183-192 (2005).
[11] Y. Nakajima, T. Sasaki, G. B. Remijn and K. Ueda, gPerceptual organization of onsets and offsets of sounds,h J. Physiol. Anthropol. Appl. Hum. Sci., 23, 345-349 (2004).
[12] Y. Nakajima and K. Ueda, gAuditory events in language and music,h to be presented in the 4th Joint Meeting, Acoustical Society of America/Acoustical Society of Japan, Honolulu (2006).
Figure caption
Figure 1
The gap transfer illusion. The illusion takes place in stimulus patterns (b) and (d) [1], [2].
