Pseudo-online framework for BCI evaluation: A MOABB perspective
Igor Carrara Théodore Papadopoulo
Objective: BCI (Brain-Computer Interface) technology operates in three modes:
online, offline, and pseudo-online. In the online mode, real-time EEG data is
constantly analyzed. In offline mode, the signal is acquired and processed
afterwards. The pseudo-online mode processes collected data as if they were
received in real-time. The main difference is that the offline mode often
analyzes the whole data, while the online and pseudo-online modes only analyze
data in short time windows. Offline analysis is usually done with asynchronous
BCIs, which restricts analysis to predefined time windows. Asynchronous BCI,
compatible with online and pseudo-online modes, allows flexible mental activity
duration. Offline processing tends to be more accurate, while online analysis
is better for therapeutic applications. Pseudo-online implementation
approximates online processing without real-time constraints. Many BCI studies
being offline introduce biases compared to real-life scenarios, impacting
classification algorithm performance. Approach: The objective of this research
paper is therefore to extend the current MOABB framework, operating in offline
mode, so as to allow a comparison of different algorithms in a pseudo-online
setting with the use of a technology based on overlapping sliding windows. To
do this will require the introduction of a idle state event in the dataset that
takes into account all different possibilities that are not task thinking. To
validate the performance of the algorithms we will use the normalized Matthews
Correlation Coefficient (nMCC) and the Information Transfer Rate (ITR). Main
results: We analyzed the state-of-the-art algorithms of the last 15 years over
several Motor Imagery (MI) datasets composed by several subjects, showing the
differences between the two approaches from a statistical point of view.
Significance: The ability to analyze the performance of different algorithms in
offline and pseudo-online modes will allow the BCI community to obtain more
accurate and comprehensive reports regarding the performance of classification
algorithms.