BU Research Blog

Our proposal titled “Multimodal Immersive NEuro-sensing (MINE) for natural neuro-behavioural measurement” is one of the three successful research concepts to be supported by BU’s Strategic Investment Area scheme. It aims at developing a pioneering multimodal and immersive system for the measurement of human behaviour and neural activities in realistic and controlled environments.

This project has several aims. Firstly, conducting human behaviour and neuroscience research traditionally requires laboratory-based settings and well-controlled environments. However, this reduces the ecological validity of research studies because the laboratory environment and the experimental stimuli are not realistic, and natural social interactions are often difficult, if not impossible.

To embrace high ecological validity, one solution is to go out of the laboratory and conduct studies in real natural environments. However, these natural environments are very uncontrollable due to all the random events happening constantly. This dilemma can be bridged by using VR technologies. VR can exert high experimental control while keeping a high ecological validity. The virtual environments can be of high realism and fidelity, for example, for social interactions (thus immersive) or scenes with affective content.

Secondly, the MINE team proposes to combine virtual reality (VR) and various neural and physiological measures in an easier to use and mobile integrated platform (see the figure below). Electroencephalography (EEG), the recording of brain activities from scalp surface to reveal neural underpinnings of human behaviour (thus neuro-sensing). Mobile EEG allows users to carry all equipment components in a backpack and walk around in a natural manner, maximally allowing natural social interactions. For instance, in the figure, a human user interacts with a virtual character while EEG is being recorded. Other measures that will be combined with VR and/or EEG setups are eye tracking, facial muscular activity, head and body motion, heart rate and skin conductance recordings (thus multimodal). This integrated platform allows researchers to have a complete picture of human behaviour and physiological responses and it opens up many possibilities to expand experimental design possibilities in natural settings. Overall, the MINE concept allows researchers to study human-environment and human-human interactions during neural measurement in controlled and realistic virtual environments.

The MINE concept greatly promotes interdisciplinary research across all faculties and with external industry partners in research fields such as mental health, way finding, healthy ageing, consumer behaviour, neuromarketing, social and affective neuroscience. The leading MINE research team (Dr Xun He, Prof Fred Charles, Dr Ellen Seiss, Dr Emili Balaguer-Ballester) are based in Faculty of Science and Technology, and also members of Interdisciplinary Neuroscience Research Centre (INRC). There are nine other collaborating team members from Faculty of Health & Social Sciences, BU Clinical Research Unit, Faculty of Media & Communication, Institute of Medical Imaging & Visualisation, and Ageing & Dementia Research Centre.

We very much look forward to the journey ahead of us!

If you would like to collaborate with the project team, please contact Dr Xun He directly. For any other queries in relation to the SIAs, please contact the RDS team on sia@bournemouth.ac.uk.