Our Approach.

Research in the VEMI Lab investigates how the encoding of spatial information from different inputs can serve as a common framework for learning about, representing, and acting in the world and how these spatial behaviors can be supported by the development of multimodal interface technologies.

Experiments in our lab are conducted using both real-world layouts and virtual environments (VEs). We employ a number of methodological approaches in our research, often combining techniques from Psychophysics, Experimental Psychology, and Cognitive Neuroscience, with principles from Human-Computer Interaction and Human Factors Engineering. Through collaborations with colleagues at UC Santa Barbara and the University of Edinboro, we have also contributed to projects using neuroimaging techniques(fMRI) to address the neural substrates of multimodal spatial processing.

For more on our research and specific experiments, check out our Research Interests,  Publications, and Current Projects pages. To learn more about virtual reality and the equipment in the lab, have a read of our VR and Lab Resources pages.

Our Philosophy.

We adopt a research philosophy that good science should strive to be both theoretically motivated and functionally relevant. As such, our studies combine basic questions about how humans learn, represent, and act in space with applied efforts toward the development of navigational technologies to support these endeavors.

Underlying Theory.

Although all of our senses encode spatial information--with hearing, touch, and vision representing the three primary spatial modalities--the vast majority of research on spatial cognition and interface design only considers the role of vision. An important premise underlying research in the lab is that many of the same tasks and interface technologies traditionally performed with vision can also be supported by other senses. Indeed, much of what most people consider “visual” we argue is “spatial”. Take a look around you. How much of what you see would you consider purely visual? Color is certainly in this camp, but we believe there is far more commonality in the information perceived about the world than is sensory-specific. For instance, geometric properties such as 3-D structure, relations between surfaces, lines, and edges, as well as distance and direction cues are not related to any one modality. Indeed, this information can be similarly specified through any number of spatial inputs. For example, feeling or seeing the edge of your desk provides the same information content and spatial computation of “edgeness” for both touch and vision. Spatial information is one example of a common thread which is specified, often redundantly, by multiple inputs in our nervous system. Temporal coding and emotional valence represent other examples.

Although vision may be our best conduit of spatial information, it does not have a monopoly on space. The guiding tenet of our research is that when information is matched between inputs during encoding, all spatial modalities (vision, touch,  audition, and spatial language) can support a similar level of spatial behavior. This assertion is predicated on the view that separate inputs develop into a common representation in working memory (called the spatial image) which is independent of the source modality and functions equivalently in supporting action (called the Functional Equivalence hypothesis). These theories are strongly influenced by the pioneering work in this area by Jack Loomis (UCSB), Roberta Klatzky (CMU), and their colleagues.

Applications.

The practical fall-out of the notion of a common spatial representation is that assuming availability of appropriate information, equivalent performance could be obtained when learning from spatial displays based on different sensor inputs. This opens the door for development of non-visual interfaces (haptic, 3-D spatialized audio, and dynamically-updated verbal displays) to support spatial behavior which is traditionally considered to be solely mediated by visual displays. Our primary focus in this domain is research into the information requirements and interface design of multimodal spatial displays for use in real-time indoor navigation systems. We are also interested in the efficacy of using multimodal virtual reality to support pre-journey spatial learning and cognitive map development. If effective, people could learn a place in advance of going there, thereby benefiting orientation and wayfinding behavior once in the physical space.

More about our basic and applied research programs and the bidirectional flow of information exchange that motivates both can be found on our Research Interests page.