Spatial learning and navigation using a virtual verbal display

Citation: 
Giudice, N.A., Bakdash, J.Z., Legge, G.E., & Roy, R. (2010). Spatial learning and navigation using a virtual verbal display. ACM Transactions on Applied Perception, 7(1), 3:1-3:22 (Article 3).
Abstract: 
We report on three experiments that investigate the efficacy of a new type of interface, called a virtual verbal display (VVD) for non-visual learning and navigation of computer-based virtual environments (VEs). Although verbal information has been studied for route-guidance, little is known about the use of context-sensitive, speech-based displays, e.g. the VVD, for supporting free exploration and wayfinding behavior. During training, participants used the VVD (Experiments I and II) or a visual display (Experiment III) to search the VEs and find four hidden target locations. At test, all participants performed a route-finding task in the corresponding real environment, navigating with vision (Experiments I and III) or from verbal descriptions (Experiment II). Training performance between virtual display modes was comparable, but wayfinding in the real environment was worse after VVD learning than visual learning, regardless of the testing modality. Our results support the efficacy of the VVD for searching computer-based environments but indicate a difference in the cognitive maps built up between verbal and visual learning, perhaps due to lack of physical movement in the VVD.
AttachmentSize
Giudice-etal2010, Learning with a VVD (ACM-TAP).pdf598.51 KB

Comments

This paper is part of a

This paper is part of a series of papers I have done on the use of dynamically-updated verbal displays to learn and navigate large-scale indoor layouts. It extends earlier work showing the efficacy of these displays for learning real buildings (Giudice et al., 2007) into learning computer-based (virtual) environments. Results demonstrate that people can effectively use virtual verbal displays (VVDs) to freely explore unfamiliar simulated layouts and show that hearing updated verbal descriptions about the space is equivalent to seeing the same environmental information for supporting efficient search behavior of the virtual environments. Exp. I results also provide a good analysis of search strategy and speak to the optimal amount of environmental information that should be provided by the verbal display (a minimalist verbal view depth describing local geometric information is sufficient). However, compared to visual learning (Exp III), problems in transferring the knowledge learned with the VVD to navigation of the corresponding real building were exposed, suggesting the development of impoverished cognitive maps. Comparisons of Exp. I (which used verbal information during learning and vision during the transfer test) and Exp. II (which used verbal information for both learning and test) are important here, as the similar transfer performance between exps indicates the spatial representation built up from verbal learning (1) can be equally accessed by verbal and visual read-out operations at test and (2) that deficits observed after verbal learning arise from the process of forming an accurate representation from training with the VVD, rather than problems in transferring VE learning to real-world navigation. these results argue against sensory-specific representations and suggest that the problem lies with the information content of the VVD--verified in Giudice and Tietz (2008), which showed that the transfer deficit is eliminated when spatial information is added to the verbal signal of the VVD.