Personalized Brain State Targeting via Reinforcement Learning
Published in Neuromatch academy, 2020
We propose a novel use of reinforcement learning as an active closed-loop assistive system that learns in real time to lead any brain state to a given goal state. Previous open- and closed-loop systems to manipulate brain states are generally passive in the sense that they are trained offline from data collected from a population but are not tailored or adapted for any individuals. Offline adaptation per individual if at all is very slow. Adaptation, and the speed of adaptation, is critical in most applications where manipulation of brain states is performed because poor initial performance and long training periods are barriers to BCI adaptation and success. Reinforcement learning is a sequential decision-making paradigm in which the system learns to map situations to optimal actions via trial-and-error interactions with the world to maximize a reward signal. Crucially, this reward signal is a form of evaluative feedback, for instance, proportional to how far the current state is from the goal state. This is in contrast to instructive feedback in the supervised learning paradigm, where the correct action is assumed known. We propose modeling brain state manipulation as a sequential decision-making problem, wherein a system takes real-time EEG data as input and uses audio-visual cues to start from any brain state and reach a physiologically-objective goal state such as a particular oscillation frequency or a deep-sleep state. We show a proof of concept example using a consumer-based EEG device. We believe such an active closed-loop system would have a large impact in assistive applications ranging from helping critically-ill patients fall asleep to helping everyday stressed-out individuals relax.
Recommended citation: Personalized Brain State Targeting via Reinforcement Learning: Abhishek Naik, Amritpal Singh, Koushani Biswas,Harini Sudha, Matthew Schlegel, Kyle E. Mathewson, Neuromatch 3 / October 28th 2020, 21:15 EDT / Track 4 / Interactive talk