BU Research Blog

This week is UK Postdoc Appreciation Week and we are celebrating and showcasing the achievements of our postdoctoral researchers and their important contribution to research at BU. 

Today’s post is by Dr Julie Kirkby and Professor Marcin Budka about the work of Postdoctoral Researcher in Machine Learning for the Modelling of Eye Movements Thomas Mercier… 

Enabling Insights into Reading Behaviour and related Pathologies through Eye-Tracking Technology and Machine Learning

In the past decades, eye-tracking technology has emerged as an invaluable tool for uncovering the cognitive processes involved in reading by offering unique insights into individuals’ reading patterns. This technique involves measuring an individual’s gaze position on a computer screen over time with high accuracy to reveal critical information about where and how long their eyes are fixating while navigating text. This provides essential clues about mental processes at play during the act of reading.

Eye-Tracking Technology in Reading Research:

The data collected from eye-tracking technology has proven valuable not only for studying general reading behaviour but also specific disorders such as dyslexia. By examining an individual’s eye movements during a reading task, researchers can better understand the cognitive mechanisms engaged in comprehending written material and potentially improve interventions for those who struggle with reading due to neurological differences or other factors. Additionally, this technology has been used to gain insight into accessibility-related issues of visual stimuli such as web pages.

Challenges in Eye-Tracking Data Analysis: Line Assignment and Measurement Noise

While technological advancements have enabled the recording of gaze points during reading with high accuracy, raw eye-tracking data still requires post-processing to identify which gaze positions are part of fixations (periods of relative positional stability) and which are part of saccades (rapid ballistic eye-movements). Furthermore, for most data analysis in reading research these fixations need to be assigned to an area of interest in the reading stimulus, such as a character or word, depending on the experiment design.

This line assignment can become significantly more difficult when dealing with multi-line passages of text, usually requiring laborious manual correction. The assignment process is made non-trivial by noise present in the tracking data due to factors such as loss of calibration during an experiment, subtle head movements or pupil dilation. Such measurement noise may manifest as dynamically changing vertical drift of recorded gaze positions, causing them to appear closer to lines above or below the actual line being read.

Attempts have been made to create algorithms that automate the line assignment process to enable researchers to carry out larger studies involving multi-line reading experiments that more closely resemble reading as it would happen outside the lab. However, these techniques often lack sufficient accuracy and reliability, leading to manual correction remaining the gold standard for addressing noise in eye-tracking fixation data.

Julie Kirkby (Department of Psychology) and Marcin Budka (Department of Computing and Informatics) are working with post-doctoral researcher Thomas Mercier, to tackle this noise correction/lines assignment problem using modern machine learning algorithms. This works by utilising deep neural networks that work directly on sequences of fixations and assign each of them to their most appropriate line of text.

Thanks to the rich and diverse datasets from previous studies carried out at BU, Thomas was able to train such a model to outperform all previously published methods of automatic line assignment. This new model is highly consistent across all datasets, unlike previous models, which makes our model a robust, default choice that will automate this task and enable researchers across psychology and the cognitive sciences to carry out and analyse eye-tracking studies with larger amounts of text without being limited by the bottleneck of manual line assignments or the need to test multiple models.

As a result, Thomas’s work represents an important step forward in advancing our understanding of cognitive processes through improved methodologies for analysing large volumes of text-based eye-tracking data (paper currently under review in the IEEE Transactions on Pattern Analysis and Machine Intelligence, a high impact journal).

Thomas will make the current code public and follow up with a publication focusing on usability of the program for non-machine learning researchers. Thomas is currently extending this model, to include diagnosing specific disorders such as dyslexia and schizophrenia.

If you’d like to write a blog post to share your appreciation for our postdoctoral researchers, please contact research@bournemouth.ac.uk. You can also get involved on social media during Postdoc Appreciation Week by using #LovePostdocs and #NPAW2023 on Twitter and Instagram and tagging us @BU_Research or @UK_NPAW.