Publication

We present a novel AAC application, HoloAAC, based on mixed reality that helps people with expressive language difficulties communicate in grocery shopping scenarios via a mixed reality device. A user, who has difficulty in speaking, can easily convey their intention by pressing a few buttons. Our application uses computer vision techniques to automatically detect grocery items, helping the user quickly locate the items of interest. In addition, our application uses natural language processing techniques to categorize the sentences to help the user quickly find the desired sentence. We evaluate our mixed reality-based application on AAC users and compare its efficacy with traditional AAC applications. HoloAAC contributed to the early exploration of context-aware AR-based AAC applications and provided insights for future research.

Video Code

We synthesized virtual reality fire evacuation training drills in a shared virtual space to explore people's collaboration behavior. We formulate the authoring process of the fire evacuation training drill in a total cost function, which we later solve with a Markov Chain Monte Carlo (MCMC) optimization-based method. The users' assigned task in the synthesized training drill is to help virtual agents evacuate the building as quickly as possible using predefined interaction mechanisms. The users can join the training drill from different physical locations and collaborate and communicate in a shared virtual space to finish the task. We conducted a user study to collect both in-game measurements and subjective ratings to evaluate whether the synthesized training drills would affect how the participants collaborated.

Video

We discuss the existing facilities at the Design Computing and Extended Reality (DCXR) Lab at George Mason University, which comprise mostly commercial-off-the-shelf computing and extended reality devices, for conducting research on virtual reality-based training. We also share thoughts on extending the facilities for conducting more sophisticated virtual reality (VR) training research in the future, which features more advanced functionalities such as remote VR training, adaptive training, and co-training in VR. In particular, we discuss a remote VR training platform to be established between George Mason University and Purdue University.