Mirror-therapy as a way to start BCI robot-assisted rehabilitation:a single case longitudinal study of a patient with hemiparesisRoman Rosipal(1,3), Natália Porubcová(1), Barbora Cimrová(2,3), Igor Farkaš(2)

(1) Institute of Measurement Science, Slovak Academy of Sciences, Bratislava (2) Faculty of Mathematics, Physics and Informatics, Comenius University in Bratislava, Slovakia(3) Pacific Development and Technology, LLC, CA, USAhttp://www.um.sav.sk/projects/BCI-RAS/

The Seventh International BCI Meeting, May 21 – 25, 2018, Pacific Grove, CA, USA

SummaryTo improve upper-limb neuro-rehabilitation in chronic stroke patients we apply new methods and tools of clinical training and machine learning for the design and development of an intelligent systemallowing the users to go through the process of self-controlled training of impaired motor pathways. We combine the brain–computer interface (BCI) technology with a robotic arm system into a compactsystem that can be used as a robot-assisted neuro-rehabilitation tool: (1) We use mirror therapy (MT) not only to improve motor functions but also to identify subject’s “atoms,” i.e. spectral-spatial EEG patternsassociated with imagined or real-hand movements, using parallel factor analysis. (2) We designed and tested a BCI-based robotic system using motor imagery in a patient with an impaired right upper limb.The novelty of this approach lies in the control protocol which uses spatial and spectral weights of the estimated sensorimotor atoms during the MT sessions.

Study DesignMirror therapy (9-months), is an innovative treatment approach

where an individual rehearses a specific limb movement byreflecting the movements of the non-paretic side in the mir-ror as if it were the affected side [1]. A link between motorimagery and passive action observation was found and as-sociated with the concept of mirror neurons [2].

PARAFAC - EEG “Atoms”, logarithmically transformed powerspectra densities (PSD) of EEG segments are analyzed bythree-way parallel factor analysis (PARAFAC) [3, 4]. Definea 3-dim. data matrix X (I×J×K) of PSD estimates at I timesegments, J electrodes and K frequencies. Then, three load-ing matrices, A, B, and C with elements a(f)i (time scores),b(f)j (spectral wights) and c

(f)k (spatial weights) define the

PARAFAC model which decomposes X as

xijk =

F∑f=1

a(f)i b

(f)j c

(f)k + εijk

where xijk are elements of X, εijk are the residual errors andF stands for a number of components (atoms). The loadingelements are found by minimizing the sum of squares of εijk

mina(f)i b

(f)j c

(f)k

‖xijk −F∑

f=1

a(f)i b

(f)j c

(f)k ‖

BCI-Controlled Robotic Splint (18 months), following the rec-ommendations of a clinical expert, we designed and con-structed a robotic splint (with one degree of freedom). Thesplint is controlled using the time scores of the selectedatoms extracted from EEG recorded during the MT sessions.Flexible score thresholds can be set.

Oscillatory Rhythms

5 10 15 20

0.40.8

Fre

quency

weig

hts Mu

FC3 C1 C3 C5 CP3 FC4 C2 C4 C6 CP4

0.250.5

Spatial

weig

hts Mu

5 10 15 20

0.40.8

Fre

quency

weig

hts SMR1

FC3 C1 C3 C5 CP3 FC4 C2 C4 C6 CP4

0.250.5

Spatial

weig

hts SMR1

5 10 15 20

0.40.8

Fre

quency

weig

hts SMR2

FC3 C1 C3 C5 CP3 FC4 C2 C4 C6 CP4

0.250.5

Spatial

weig

hts SMR2

5 10 15 20

0.40.8

Fre

quency

weig

hts Beta1

FC3 C1 C3 C5 CP3 FC4 C2 C4 C6 CP4

0.250.5

Spatial

weig

hts Beta1

5 10 15 20

0.40.8

Fre

quency

weig

hts Beta2

FC3 C1 C3 C5 CP3 FC4 C2 C4 C6 CP4

0.250.5

Spatial

weig

hts Beta2

5 10 15 20

0.40.8

Fre

quency

weig

hts Alpha

FC3 C1 C3 C5 CP3 FC4 C2 C4 C6 CP4

Electrode

0.250.5

Spatial

weig

hts Alpha

5 10 15 20

Frequency [Hz]

0.40.8

Fre

quency

weig

hts Theta

FC3 C1 C3 C5 CP3 FC4 C2 C4 C6 CP4

Electrode

0.250.5

Spatial

weig

hts Theta

Mean values of the PARAFAC spectral (left column) and spatial (right column) weight vec-tors obtained during the Mirror Therapy (MT, black) and training with the Robotic Splint(red). Each row represents the means for one of the seven extracted oscillatory rhythms(atoms). Shaded area represents the standard deviation of MT averages.

Mu SMR1 SMR2 Beta1 Beta2 Alpha Theta40

60

80

100

120

Ave

rag

ed

ato

m s

co

res

Mirror Therapy EC - VLH - L

*** * ** *

pre-training

post-training

Mu SMR1 SMR2 Beta1 Beta2 Alpha Theta40

60

80

100

120Mirror Therapy EO - VLH - L

*** *** *** *** ***

Mu SMR1 SMR2 Beta1 Beta2 Alpha40

60

80

100

120

Ave

rag

ed

ato

m s

co

res

Robotic Splint EC - VLH - L

** *** *** **

Mu SMR1 SMR2 Beta1 Beta2 Alpha40

60

80

100

120Robotic Splint EO - VLH - L

*** ** ** *** ***

Mean values of the PARAFAC atom scores computed during the resting period with eyesclosed. The blue bars represent pre-training session, yellow bars post-training. Significantdifferences between the two sessions are highlighted (∗∗∗: p < 0.001, ∗∗: p < 0.01, ∗:p < 0.05)

To study and test changes of oscillatory rhythms at each EEG electrode separately, the irregular-resampling auto-spectral analysis (IRASA)method was applied to separate fractal (representing background EEG) and harmonic (representing oscillatory EEG) components in thepower spectrum of EEG segments [5].

0

1

2

3

4

Mean

PSD

[µV

2/Hz]

FC3 **

0

1

2

3

4

C1 ***

0

1

2

3

4

C3 *

0

1

2

3

4

C5

0

1

2

3

4

CP3

0

1

2

3

4

O1

0

1

2

3

4

FC4 ***

0

1

2

3

4

C2 ***

0

1

2

3

4

C4 ***

0

1

2

3

4

C6 **

0

1

2

3

4

CP4 **

0

1

2

3

4

Mean

PSD

[µV

2/Hz]

FC3

0

1

2

3

4

C1

0

1

2

3

4

C3

0

1

2

3

4

C5

0

1

2

3

4

CP3

0

1

2

3

4

O1

0

1

2

3

4

FC4

0

1

2

3

4

C2

0

1

2

3

4

C4

0

1

2

3

4

C6

0

1

2

3

4

CP4

08/14 04/15

Date

-2

-1

0

1

2

Mean

PSD

Diff.[µV

2/Hz]

FC3 *

08/14 04/15

Date

-2

-1

0

1

2C1 **

08/14 04/15

Date

-2

-1

0

1

2C3 **

08/14 04/15

Date

-2

-1

0

1

2C5 **

08/14 04/15

Date

-2

-1

0

1

2CP3 *

08/14 04/15

Date

-2

-1

0

1

2O1

08/14 04/15

Date

-2

-1

0

1

2FC4 *

08/14 04/15

Date

-2

-1

0

1

2C2 *

08/14 04/15

Date

-2

-1

0

1

2C4 *

08/14 04/15

Date

-2

-1

0

1

2C6

08/14 04/15

Date

-2

-1

0

1

2CP4

Mirror Therapy: Averaged Mu rhythm harmonic part of the EEG power spectrum for theeyes closed condition. Each value is an average computed for a session (day). The first andsecond rows represent averages of pre-training and post-training periods. The third row rep-resents the post- and pre-training difference. Solid lines represents linear fit to data, 95%confidence interval for each point is represented by dotted lines. Linear fits with significantnon-zero slopes are denoted by stars (∗∗∗: p < 0.001, ∗∗: p < 0.01, ∗: p < 0.05).

0

1

2

3

4

Mean

PSD

[µV

2/Hz]

FC3

0

1

2

3

4

C1 *

0

1

2

3

4

C3 *

0

1

2

3

4

C5 *

0

1

2

3

4

CP3

0

1

2

3

4

O1 **

0

1

2

3

4

FC4 ***

0

1

2

3

4

C2 *

0

1

2

3

4

C4 **

0

1

2

3

4

C6 **

0

1

2

3

4

CP4 ***

0

1

2

3

4

Mean

PSD

[µV

2/Hz]

FC3

0

1

2

3

4

C1

0

1

2

3

4

C3

0

1

2

3

4

C5

0

1

2

3

4

CP3

0

1

2

3

4

O1 **

0

1

2

3

4

FC4

0

1

2

3

4

C2

0

1

2

3

4

C4

0

1

2

3

4

C6

0

1

2

3

4

CP4 *

06/16 10/17

Date

-2

-1

0

1

2

Mean

PSD

Diff.[µV

2/Hz]

FC3

06/16 10/17

Date

-2

-1

0

1

2C1

06/16 10/17

Date

-2

-1

0

1

2C3

06/16 10/17

Date

-2

-1

0

1

2C5 *

06/16 10/17

Date

-2

-1

0

1

2CP3

06/16 10/17

Date

-2

-1

0

1

2O1

06/16 10/17

Date

-2

-1

0

1

2FC4

06/16 10/17

Date

-2

-1

0

1

2C2 *

06/16 10/17

Date

-2

-1

0

1

2C4 *

06/16 10/17

Date

-2

-1

0

1

2C6 *

06/16 10/17

Date

-2

-1

0

1

2CP4 *

Robotic Splint: Averaged Mu rhythm harmonic part of the EEG power spectrum for theeyes closed condition. Each value is an average computed for a session (day). The first andsecond rows represent averages of pre-training and post-training periods. The third row rep-resents the post- and pre-training difference. Solid lines represents linear fit to data, 95%confidence interval for each point is represented by dotted lines. Linear fits with significantnon-zero slopes are denoted by stars (∗∗∗: p < 0.001, ∗∗: p < 0.01, ∗: p < 0.05).

5 10 15 20

Frequency [Hz]

0

5

µV

2/H

z

FC3

5 10 15 20

Frequency [Hz]

0

5

µV

2/H

z

C1

5 10 15 20

Frequency [Hz]

0

5

µV

2/H

z

C3

5 10 15 20

Frequency [Hz]

0

5

µV

2/H

z

C5

5 10 15 20

Frequency [Hz]

0

5

µV

2/H

z

CP3

5 10 15 20

Frequency [Hz]

0

5

µV

2/H

z

O1

5 10 15 20

Frequency [Hz]

0

5

µV

2/H

z

FC4

5 10 15 20

Frequency [Hz]

0

5

µV

2/H

z

C2

5 10 15 20

Frequency [Hz]

0

5

µV

2/H

z

C4

5 10 15 20

Frequency [Hz]

0

5

µV

2/H

z

C6

5 10 15 20

Frequency [Hz]

0

5

µV

2/H

z

CP4

Mirror Therapy: Pre-training (black) vs. post-training (red) averaged harmonic part ofthe EEG power spectrum for the eyes-closed condition, pre-trainng (black). Each plot is anaverage computed over 50 sessions (days).

5 10 15 20

Frequency [Hz]

0

2

µV

2/H

z

FC3

5 10 15 20

Frequency [Hz]

0

2

µV

2/H

z

C1

5 10 15 20

Frequency [Hz]

0

2

µV

2/H

z

C3

5 10 15 20

Frequency [Hz]

0

2

µV

2/H

z

C5

5 10 15 20

Frequency [Hz]

0

2

µV

2/H

z

CP3

5 10 15 20

Frequency [Hz]

0

2

µV

2/H

z

O1

5 10 15 20

Frequency [Hz]

0

2

µV

2/H

z

FC4

5 10 15 20

Frequency [Hz]

0

2

µV

2/H

z

C2

5 10 15 20

Frequency [Hz]

0

2

µV

2/H

zC4

5 10 15 20

Frequency [Hz]

0

2

µV

2/H

z

C6

5 10 15 20

Frequency [Hz]

0

2

µV

2/H

z

CP4

Robotic Splint: Pre-training (black) vs. post-training (red) averaged Mu rhythm harmonicpart of the EEG power spectrum for the eyes-closed condition. Each plot is an average com-puted over 132 sessions (days).

Clinical & Behavioural ResultsA clinical evaluation of the subject’s upper limb movement abili-ties was carried out. The undertaken clinical tests indicate a slightimprovement in movement and spasticity of the arm but without adetectable progress for wrist and fingers. It is worth noting that thesubject entered the study as late as two years after stroke with severeplegic hand. Subjectively, we observed an improvement in subject’sspeech and social communication, but this was not clinically tested.The subject showed strong enthusiasm to participate.

For the rehabilitation trainingwith the robotic splint we com-pared 3 different protocols. Weobserved that it was the most dif-ficult when the subjects was in-structed to keep eyes open dur-ing the robotic splint control.

ConclusionsTo our knowledge, this is the first longitudinal study (over 9 months long) of the mirror-box therapy showing effects on the modulation of sensorimotor EEG oscillatory rhythms. We observed significantshort-term (a single session pre- versus post-training) and longer-term EEG effects lasting from day-to-day as well as spanning the whole period of the experiment. Analysis of the EEG data recorded duringthe mirror therapy sessions reveals stable day-to-day space-spectral atomic EEG representation of dominant sensorimotor oscillatory rhythms. The atomic representation of EEG allowed us to develop andtest an efficient and flexible BCI protocol for the control of the constructed robotic splint for neuro-rehabilitation. Longitudinal robotic splint training of a 58-years-old man who had a right-hand hemiplegiadue to an ischemic stroke is promising and although it is a single case study, it is used as a proof of concept, not as a population based statistical proof. Clinical efficiency of this procedure requires furtherevaluation by considering a wider, clinically heterogeneous population of patients with motor impairment.

References[1] Ramachandran V. S., Rogers-Ramachandran D. C., Cobb S. Touching the phantom. Nature, 377:489–490, 1995.[2] Mulder T. Motor imagery and action observation: cognitive tools for rehabilitation. Journal of Neural Transmission. , 114:1265–1278, 2007.[3] Bro R. PARAFAC. Tutorial and applications. Chemometrics and Intelligent Laboratory Systems, 28:149–171, 1997.[4] Rosipal R., Trejo L.J., Nuñez P.L. Application of multi-way EEG decomposition for cognitive workload monitoring. In Proc. of the 6th ICPRM, Beijing, pp. 145-149, 2009.[5] Wen H, Liu Z. Separating Fractal and Oscillatory Components in the Power Spectrum of Neurophysiological Signal. Brain Topography. , 29:13–26, 2016.

AcknowledgementThe work was supported by the Slovak Research and DevelopmentAgency (projects APVV-16-0202 and APVV-0668-12) and by the Slo-vak Grant Agency for Science (projects VEGA-2/0011/16 and VEGA2/0151/18).

1