Shared Temporal Accuracy of Action Execution and Sensory Perception

Shared Temporal Accuracy of Action Execution and Sensory Perception Tomomitsu Herai (herai@brn.dis.titech.ac.jp) Tokyo Institute of Technology, 4259 Nagatsuta-cho, Midori-ku, Yokohama, 226-8502, Japan Ken Mogi (kenmogi@csl.sony.co.jp) Sony Computer Science Laboratories, Takanawa Muse Bldg., 3-14-13, Higashigotanda, Shinagawa-ku, Tokyo, 141-0022, Japan Abstract Integration of an action and its sensory feedback is important in interacting with an uncertain environment and construct a consistent model of the world. In this process, multisensory data need to be processed, in which audition and vision play important roles. Subjective simultaneity of audiovisual stimuli is affected by various factors. To investigate the relation between our subjective simultaneity of audiovisual stimuli and our action, we conducted an experiment in which subjects’ action affected the temporal patterns of resulting stimuli. The modes of contingency between action and stimuli were made variable. We found significant correlations between the accuracies of actions and the window of subjective simultaneity among subjects, although their task performances were widely varied. In addition, the correlation patterns were found to depend on the contingency between the key pressing and stimuli. These results suggest that the subjective simultaneity of audiovisual stimuli correlates with the accuracy of execution of action, indicating a common mechanism engaging the perception of subjective simultaneity in sensorimotor integration and action execution. Keywords: sensorimotor contingency; integration; subjective simultaneity; timing audiovisual Introduction We interact with our environments through voluntary movements and its sensory feedback. We recognize ourselves and the external world through sensorimotor interactions. However, it is rarely the case that we have a complete knowledge about the sensorimotor contingency in a given context. Understanding the mechanism of integrating motor and sensory information in the presence of uncertainty provides us with important information as to how we construct a consistent model of the world. The sensorimotor contingency affects both our perception and action. Recent studies have shown that if sensory stimuli are the results of our own action (i.e. pressing a key which generates beep), our perceived timing of those sensory feedback are closer to the timing of action than actually (Haggard, Clark & Kalogeras, 2002; Tsakiris & Haggard, 2003). In addition, this temporal shift was affected by the event probability of sensory stimuli (Engbert & Wohlschlager, 2007; Moore & Haggard, 2008). A higher event probability has been found to lead to a larger temporal shift. Furthermore, the adaptation of this temporal shift recalibrates the perceived timing of the action and the visual stimulus, leading to a illusionary temporal reversal in which the subject perceive the flash before the action (Stetson et al., 2006). These studies suggest that our cognitive processes related to agency and expectation affect the perceived timing of sensory events. Temporal shifts in perception are ubiquitous in our daily life. We have to always treat multisensory information, where the knowledge about the external world is mainly from vision and audition. Since the light travels much faster than the sound, the delay of the timings at which each sensory organ receives its appropriate stimuli becomes progressively larger depending on the distance from the event. Despite this temporal disturbance, we tend to perceive the light and sound signals from a single source simultaneously (Kopinska & Harris, 2004; Stone et al., 2001; Sugita & Suzuki, 2003). The temporal window of subjective simultaneity has shown to be affected by various factors. Studies have shown that adjacent adaptation affected the width of simultaneous temporal window and subjective simultaneity (Fujisaki et al., 2004; Navarra et al., 2005; Vroomen et al., 2004). Spatial position (Zampini, Shore & Spence, 2003; Zampini et al., 2005) and attention changed the subjective simultaneity of audiovisual stimuli (Zampini, Shore & Spence, 2005). The subjects in these studies were typically presented with the stimuli passively. The relation between our action and the audiovisual integration remain still unclear. As noted above, intentional action changes the perceived timing of its sensory feedback. Based on this point, we hypothesized that the subjective judgment of the simultaneity of audiovisual stimuli would be affected by the processes of sensorimotor integration. To examine this hypothesis, we conducted an experiment using two keys which generated either a beep sound or flash. The timings of key pressings by the subjects were reflected in the following generations of flash and beep. In general, variances exist in the accuracy of motor performance and the sensitivity of perception. We were interested if there was a common mechanism involving action execution and simultaneity perception as reflected in the performances of subjects.