Paper - HoloCook: A Real-Time Remote Mixed Reality Cooking Tutoring System

HCII 2024

           

Abstract

Advancements in extended reality (XR) technology have spurred research into XR-based training and collaboration. On the other hand, mixed reality (MR) fuses the real and the virtual world in real time and provides interaction, which brings the possibility of completing real-world tasks collaboratively through MR headsets. We present HoloCook, a novel real-time remote cooking tutoring system utilizing HoloLens 2. HoloCook is a lightweight system that doesn’t require any additional devices. HoloCook can not only synchronize the coach’s action with the trainee in real time but also provide the trainee with animations and 3D annotations to aid in tutoring process. HoloCook supports tutoring two recipes: pancakes and cocktails. Our user evaluation with one coach and four trainees establishes HoloCook as a feasible and usable remote cooking tutoring system in mixed-reality environments.

Acknowledgments

We are grateful to the participants for their feedback on HoloCook. We thank Huizhen Zhou for his generously providing the kitchen. This project was supported by NSF grants (award numbers: 2235049 and 2128867) and the GMU IDIA P3 Faculty Fellowship.

BibTeX

@inproceedings{yu2025holocook,
  title={HoloCook: A Real-Time Remote Mixed Reality Cooking Tutoring System},
  author={Yu, Liuchuan and Han, Bo and Chen, Songqing and Yu, Lap-Fai},
  booktitle={International Conference on Human-Computer Interaction},
  pages={244--264},
  year={2025},
  organization={Springer}
}

Last update @ Dec 31, 2024