BU Research Blog

Paper titled ” Context-aware Mixed Reality: A Learning-based Framework for Semantic-level Interaction” has been accepted for publication in the leading journal Computer Graphics Forum.

Dr Long Chen, the first author, was a matched-funded PhD student graduated in April 2019. He was under the supervision of Professor Wen Tang, Professor Jian Jun Zhang at BU, Dr Tao Ruan Wan at the University of Bradford and Professor Nigel John at the University of Chester as the matched-funder.

Mixed Reality is a powerful interactive technology for new types of user experience. This paper presents a semantic-based interactive
MR framework that is beyond the current geometry-based approaches, offering a step-change in generating high-level
context-aware interactions. The key insight described in this paper is that semantic understanding in Mixed Reality not only greatly enhances user experience through context-aware object behaviours, but also paves the way for solving complex interaction design challenges. The proposed computational framework generates semantic properties of the real-world environment for Mixed Reality, through a dense 3D scene reconstruction and deep image understanding scheme. A simple MR game has been developed to evaluate the proposed concept and the efficacy of the framework.

The team is invited to give an oral presentation at the premier conference Eurographics or Pacific Graphics, depending on presentation slot arrangement.