Journal of Rehabilitation Research & Development (JRRD)

Quick Links

  • Health Programs
  • Protect your health
  • Learn more: A-Z Health
Veterans Crisis Line Badge
 

Volume 48 Number 4, 2011
   Pages 367 — 386

Psychological state estimation from physiological recordings during robot-assisted gait rehabilitation

Alexander Koenig, MSc;1-2* Ximena Omlin, MSc;1 Lukas Zimmerli, MSc;1,3 Mark Sapa, MD;4 Carmen Krewer, MSc;4 Marc Bolliger, PhD;1-2 Friedemann Müller, MD, MSc;4 Robert Riener, PhD1-2

1Sensory-Motor Systems Laboratory, Department of Mechanical and Process Engineering, Eidenössische Technische Hochschule Zurich, Zurich, Switzerland; 2Spinal Cord Injury Center, Balgrist University Hospital, University of Zurich, Zurich, Switzerland; 3Hocoma AG , Volketswil, Switzerland; 4Neurologische Klinik Bad Aibling, Bad Aibling, Germany

Abstract — Robot-assisted treadmill training is an established intervention used to improve walking ability in patients with neurological disorders. Although it has been shown that attention to the task is a key factor for successful rehabilitation, the psychological state of patients during robot-assisted gait therapy is often neglected. We presented 17 nondisabled subjects and 10 patients with neurological disorders a virtual-reality task with varying difficulty levels to induce feelings of being bored, excited, and overstressed. We developed an approach to automatically estimate and classify a patient's psychological state, i.e., his or her mental engagement, in real time during gait training. We used psychophysiological measurements to obtain an objective measure of the current psychological state. Automatic classification was performed by a neural network. We found that heart rate, skin conductance responses, and skin temperature can be used as markers for psychological states in the presence of physical effort induced by walking. The classifier achieved a classification error of 1.4% for nondisabled subjects and 2.1% for patients with neurological disorders. Using our new method, we processed the psychological state data in real time. Our method is a first step toward real-time auto-adaptive gait training with potential to improve rehabilitation results by optimally challenging patients at all times during exercise.

Key words: automatic classification, gait robot, Lokomat, mental engagement, physiological measurements, psychophysiology, rehabilitation, spinal cord injury, stroke, virtual reality.

Abbreviations: ANS = autonomic nervous system, CI = confidence interval, CNS = central nervous system, DGO = driven gait orthosis, ECG = electrocardiogram, EMG = electromyogram, GSR = galvanic skin response, HR = heart rate, HRV = heart rate variability, PC = principal component, PCA = principal component analysis, RMSE = root-mean-square error, RMSSD = root-mean-square successive difference, SAM = Self-Assessment Manikin, SCI = spinal cord injury, SCL = skin conductance level, SCR = skin conductance response, SD = standard deviation, T = thoracic, VR = virtual reality.
*Address all correspondence to Alexander Koenig, MSc; Sensory-Motor Systems Laboratory, ETH Zurich, Tannenstrasse 1, E4, 8092 Zurich, Switzerland; +41-76-213-77-88; fax: +41-44-632-18-76. Email: Koenig@mavt.ethz.ch
DOI:10.1682/JRRD.2010.03.0044
INTRODUCTION

Treadmill training is an established treatment for gait rehabilitation in patients with neurological disorders, such as stroke, spinal cord injury (SCI), or traumatic brain injury [1-2]. To increase rehabilitation outcome in those patients, an increasing number of driven gait orthoses (DGOs) are available that automate gait training, such as the Lokomat (Hocoma AG; Volketswil, Switzerland), the Autoambulator (HealthSouth; Birmingham, Alabama), the LOPES (Lower-extremity Powered Exoskeleton, Laboratory Biomechanical Engineering, University of Twente; Enschede, the Netherlands), and the Gait Trainer (developed by Hesse et al. [4]) [3-7].

Active biomechanical engagement of patients in rehabilitation training has shown to be an important factor in successful rehabilitation results [8]. The patient's biomechanical effort can be quantified by torque and force sensors. This information is used to assess the patient's level of activity [9].

While many studies investigated biomechanical engagement, active mental engagement, which has also been shown to be a key factor in successful rehabilitation [10], cannot be assessed easily and has been previously neglected. The goal of our work is to determine if a patient is mentally engaged during the training in order to maximize motor learning during rehabilitation. From motor learning theory, it is known that the learning rate is maximal at a task difficulty level that positively challenges and excites subjects while not being too stressful or boring [11]. A task that is too easy for the subject will be perceived as boring and a task that is too difficult will overstress the subject, while an optimally challenging task should induce maximal mental engagement and optimal physical participation.

We developed an approach using psychophysiological signals to automatically estimate and classify a patient's psychological state, i.e., his or her mental engagement, during rehabilitation. We used measurements of heart rate (HR), breathing frequency, galvanic skin response (GSR), and skin temperature. To our best knowledge, estimation of psychophysiological states has never been performed, either during walking or in patients with neurological disorders. Using our new method, we process the psychological state data in real time. We introduce our method as a first step toward real-time, auto-adaptive gait training with management of subject engagement and potential to improve rehabilitation results by optimally challenging the subject at all times during exercise.

BACKGROUND

Active mental engagement has been shown to be a key factor for successful rehabilitation [10]. In our experiment, we have defined three different levels of mental engagement according to the circumplex model of affect [12] (Figure 1), in which emotions are defined by two dimensions: valence (ranging from unpleasant to pleasant) and arousal (ranging from deactivation to activation). We used virtual-reality (VR) environments during robot-assisted gait training to induce different levels of mental engagement in subjects. Challenging tasks in VR environments were shown to have a positive, motivating effect during rehabilitation [13]. In this context, boring, too stressful, or optimally challenging tasks can be the result of a VR task that is too easy, too difficult, or appropriate for the patient's abilities, respectively. A VR task that is too easy or underchallenging for the patient will be perceived as boring and a task that is too difficult will overstress the patient, while an optimally challenging task should excite and motivate the patient and cause maximal mental engagement and optimal physical participation.


Figure 1. Three conditions were defined according to the circumplex model of affect. Solid circle represents underchallenged condition, dotted circle represents challenged condition, and dashed circle represents over-challenged condition. Adapted from Russell JA. A circumplex model of affect. J Pers Soc Psychol. 1980;39(6):1161–78. DOI:10.1037/h0077714

Figure 1.

Three conditions were defined according to the circumplex model of affect. Solid circle represents underchallenged condition, dotted circle represents challenged condition, and dashed circle represents over-challenged condition. Adapted from Russell JA. A circumplex model of affect. J Pers Soc Psychol. 1980;39(6):1161–78. DOI:10.1037/h0077714

Click Image to Enlarge. View as PowerPoint Slide

In the present state of the art, mental engagement of patients is quantified by questionnaires-motivation, for example, can be quantified by the Intrinsic Motivation Inventory [14]. During gait rehabilitation, questionnaires are not appropriate for continuous, objective assessment of the psychological state of the patient. In addition, patients with neurological disorders with severe cognitive deficits or aphasia might not be able to understand and respond appropriately to the questions.

Therefore, we used physiological measurements as a proxy to assess and determine mental engagement of patients, because it has been shown that each behavioral, cognitive, emotional, and social interaction may be reflected in physiological processes [15-16]. The physiological recordings reflect reactions that result from activities of the central nervous system (CNS) and the autonomic nervous system (ANS). Examples for measures of the CNS are the electroencephalogram or near infrared spectroscopy; examples for measurements of the ANS are electrocardiogram (ECG), GSR, breathing frequency, or skin temperature. We focused on measuring the ANS because real-time measurement and analysis of signals from the CNS during walking in a robotic device are generally not feasible due to noise and motion artifacts.

Signals from the ANS that could indicate mental engagement are primarily signals that respond to mental stress or relaxation [15]. In addition to psychological processes, physical effort such as walking on a treadmill can influence the psychophysiological measurements. From the ECG , HR and HR variability (HRV) can be computed. When recorded during a VR task with a goal, HR was shown to be an indicator of physical as well as mental load [17]. Physiological effort and psychological stress have an influence on the short-term variation of HR. HRV was shown to decrease during physical effort [18] and mental stress [19]. GSR is used as a direct measure for arousal [20]. From the GSR, skin conductance responses (SCRs) measured as a number and the skin conductance level (SCL) are computed. The SCR is a sensitive indicator for emotional strain [21]. In recent research, SCL was found to increase during demanding tasks compared with a rest period [22]. The breathing frequency was found to increase during stress [23], mental effort [24], and physical activity [25]. Skin temperature decreased during mental work stress in a study by Ohsuga et al. [26] but increased with physical activity [27]. Other physiological recordings from the peripheral nervous system have been used as indicators of the psychophysiological state of a subject. Among these were facial electromyogram (EMG) recordings as indicators for emotional responses to pleasant or unpleasant stimuli [28-29].

We hypothesize that in our experiments, we should therefore expect an increase in HR and a decrease in HRV with increasing difficulty level of a task. In addition, an increase in SCR is expected to occur as a consequence of an increase in difficulty level of a VR task. An increase is also expected in SCL and breathing frequency for the task conditions compared with baseline. A decrease in skin temperature as a consequence of a decrease in the difficulty level could therefore be expected if the physical activity remained constant.

METHODS

We conducted this study at two locations. We conducted measurements with nondisabled subjects at the Spinal Cord Injury Center Balgrist, Zurich, Switzerland. We conducted measurements with patients who had experienced a stroke at the Neurologische Klinik Bad Aibling, Bad Aibling, Germany. We presented all subjects with a VR task that was used to induce three different levels of mental engagement. The mental engagement was subjectively quantified by subjects via questionnaires. To objectively quantify the effects of different levels of mental engagement, we recorded ECG , breathing, GSR, and skin temperature. Our goal was to determine the potential to identify the current state of mental engagement directly from the physiological signals by using descriptive statistics.

Attaching sensors for physiological recordings on the subject's body was time and labor intensive, requiring attention of the therapist and reducing the time a patient could exercise in the Lokomat. To improve clinical applicability of our approach, we investigated which physiological signals contained most of the information and which sensors might not be necessary for future applications. Principal component analysis (PCA) allowed identification of the signals that explained most of the variance in the data.

We then trained a neural network to investigate the possibility of identifying the current state of mental engagement directly from physiology. This was done with all physiological signals and with a reduced data set of signals, which were shown to be dominant in the PCA.

Participants

Seventeen nondisabled subjects (eight male and nine female, aged 24.1 ± 2.0 years [mean ± standard deviation (SD) unless otherwise specified]) and ten patients with neurological disorders (seven male and three female, aged 52.4 ± 18.9 years) participated in the study. All 10 patients who neurological gait impairment due to their pathology. Table 1 summarizes characteristics of the 10 patients who participated in the study. A clinical expert selected and approved all patients for participation in the study to ensure that they were able to follow the instructions and respond accordingly.


Table 1.
Characteristics of patients with neurological disorders.

Patient
Sex
Age
(yr)
Time Since Incident
(mo)
Lesion
FAC
WISCI II
Beta
Blocker

1
M
68
2.5
Right middle cerebral artery stroke
4
NA
Yes
2
M
62
3.0
Right middle cerebral artery stroke
1
NA
No
3
M
51
6.5
Right middle cerebral artery and posterior cerebral artery stroke
3
NA
Yes
4
M
64
1.0
SCI ASIA A (S1)
NA
13
No
5
F
19
21.0
SCI ASIA C (L1)
NA
20
No
6
M
75
2.5
Myelopathy (C3-C6)
NA
13
No
7
F
57
1.5
Guillain-Barré syndrome
0
NA
No
8
M
37
4.5
Hypoxic encephalopathy
1
NA
Yes
9
M
26
38.0
Subarachnoid hemorrhage Hunt and Hess grade 4-5
0
NA
No
10
F
65
3.0
Right basal ganglia hemorrhage with intraventricular extension
0
NA
No

ASIA = American Spinal Injury Association, C = cervical, F = female, FAC = functional ambulation category, L = lumbar, M = male, NA = not available, S = sacral, SCI = spinal cord injury, WISCI = Walking Index for Spinal Cord Injury.


Training Environment and Virtual-Reality Task

We used the DGO Lokomat for the locomotion training. The DGO is an exoskeleton with drives on hip and knee joints. The device allows assisted locomotion on a treadmill by guiding the subjects' legs along a predefined trajectory. We fixed subjects into the DGO with a harness around the hip and cuffs around the legs. The subjects' feet were passively lifted by elastic foot straps to prevent foot drop. Subjects were connected to a body-weight support system. During walking, we kept the speed constant at 2 km/h. Cadence had to be slightly adjusted to the individual leg length of each subject. Subjects walked with the assistance of 30 percent body-weight support. At Balgrist University Hospital, we projected the VR environment onto a 3 × 2 m back-projection screen, which was mounted in front of the DGO with a Dolby 5.1 sound system (Dolby Laboratories; San Francisco, California) for auditory feedback. At the Neurologische Klinik Bad Aibling, we used a 42-inch flat screen television with stereo sound.

We provided subjects with a VR task that included two distinct actions at the same time, a biomechanical task and a cognitive task (Figure 2). Subjects had to change walking direction in the VR environment to collect items by walking into them (biomechanical task). To change the walking direction, subjects had to perform an active push off in the terminal stance phase. To turn left, the subjects had to increase activity in the right leg during stance [30]. Because the required physical effort to change walking direction was set individually, the challenge was to navigate through the VR environment and collect items. In addition, subjects had to jump over barrels that rolled toward them by clicking a computer mouse button (cognitive task). Collected items added points to a counter, missed items and non-jumped barrels subtracted points.


Figure 2. Virtual task with x-axis representing distance between middle line and items and y-axis representing distance between items or barrels. Axes are adjustable to create different difficulty levels.

Figure 2.

Virtual task with x-axis representing distance between middle line and items and y-axis representing distance between items or barrels. Axes are adjustable to create different difficulty levels.

Click Image to Enlarge. View as PowerPoint Slide

To create different task difficulty levels, the distance between items was adjustable. Furthermore, the distance between the barrels and their speed was adjustable (Figure 2). Because physical effort influenced the psychophysiological recordings, we chose the VR task as a combination of coordination (change walking direction) and cognition (jump over the barrels). This allowed for creating subject-specific variation in task difficulty levels while keeping the physical effort necessary for successful task completion as low as possible.

We set task difficulty individually for each nondisabled subject and patient with neurological disorders during an initial practice time (Figure 3) by adjusting the task difficulty (5 min). In the underchallenged condition, we placed the items such that subjects could collect 100 percent of them without major changes in walking direction. We defined the challenged condition by setting the distance between items and their distribution over the whole scenario in such a way that subjects could collect 80 to 90 percent of the objects. In the overchallenged condition, we distributed the objects in such a way that subjects achieved less than 10 percent of the possible maximum.


Figure 3. Experimental protocol. SAM = Self-Assessment Manikin.

Figure 3.

Experimental protocol. SAM = Self-Assessment Manikin.

Click Image to Enlarge. View as PowerPoint Slide

Inducing Different Levels of Mental Engagement

We hypothesized that three different levels of mental engagement could be introduced during exercise by providing subjects with different levels of task difficulty. The target mental engagement levels were (1) a feeling of boredom, (2) a feeling of being motivated and excited, and (3) a feeling of being overstressed (Figure 4). We expected subjects to be bored when the VR task was easy (underchallenged condition) and when no particular biomechanical or cognitive effort was necessary to successfully complete the task. When setting task difficulty such that the task was difficult but feasible (challenged condition), we expected subjects to be motivated and excited. When setting the task difficulty to be too difficult (overchallenged condition), we expected them to be overstressed (Figure 1). In the arousal-valence space (Figure 1), the underchallenged condition would have a low level of arousal and a low level of valence, the challenged condition would have a high level of arousal and valence, and the overchallenged condition would have a high level of arousal and a low level of valence.


Figure 4. Inducing different states of mental engagement during gait training using task level difficulty of virtual-reality task. Subjectively, ques-tionnaires are used to establish connection between task difficulty and mental engagement. Objectively, psychophysiological measurements are used. *Russell JA. A circumplex model of affect. J Pers Soc Psy-chol. 1980;39(6):1161–78. DOI:10.1037/h0077714

Figure 4.

Inducing different states of mental engagement during gait training using task level difficulty of virtual-reality task. Subjectively, ques-tionnaires are used to establish connection between task difficulty and mental engagement. Objectively, psychophysiological measurements are used. *Russell JA. A circumplex model of affect. J Pers Soc Psy-chol. 1980;39(6):1161–78. DOI:10.1037/h0077714

Click Image to Enlarge. View as PowerPoint Slide

We used the Self-Assessment Manikin (SAM) questionnaire to verify the hypothesis that the three conditions in the VR task really resulted in feelings of boredom, excitement, and stress (Figure 4). The SAM is used to measure emotional response to different stimuli, in particular the emotional responses arousal and valence [31]. The arousal dimension ranged from "relaxed and sleepy" to "excited and extremely aroused." The valence dimension ranged from "unhappy" to "very happy" [32]. We asked subjects to respond to a 5-point scale by selecting the figure that best represented their current emotion. The value of 1 represented the lowest valence (unhappy) and arousal (sleepy) and 5 represented the highest valence (very happy) and arousal (excited). After each task condition, we asked the subject to respond to the SAM. Furthermore, we asked the subjects to rate their physical effort on a scale from 1 to 5, where 1 = not physically exhausting and 5 = extremely physical exhausting. This nonverbal, pictorial questionnaire was chosen so subjects did not disturb the breathing frequency analysis by speaking and to reduce the complexity of responding to the questionnaire for patients with aphasic stroke or cognitive impairments. We tested all dimensions of the SAM using the Friedman test. We performed a Wilcoxon test in combination with Bonferroni correction for a paired comparison. We set the significance level at p < 0.05.

Estimating Mental Engagement from Physiological Recordings
Physiological Recordings

In addition to the subjective questionnaires, we objectively quantified the effects of different levels of mental engagement through recordings of ECG , breathing, GSR, and skin temperature. We measured the ECG with three surface electrodes. One electrode was affixed 2 cm below the right clavicula between the first and second ribs, one was affixed at the fifth intercostal space on the midaxillary line on the left side of the body, and a ground electrode was affixed to the right acromion. We computed HR from ECG using a real-time R-wave detection algorithm (adapted from Christov [33]); we computed HRV as a discrete time-series of consecutive R waves, the so-called "RR intervals." We computed the square root of the mean squared differences of successive normal-to-normal intervals (RMSSD) from RR intervals for analysis of HRV in the time domain. According to the recommendations of Malik [34], we performed the frequency analysis of HRV using the quotient of low-frequency components over high-frequency components. The respective frequency bands can be found in "Heart rate variability. Standards of measurement, physiological interpretation, and clinical use" [34]. Using a thermistor flow sensor placed underneath the nose, we recorded breathing and computed breathing frequency using a peak detection algorithm. We measured changes in GSR using two electrodes attached to the proximal phalanx of the second and fourth fingers on the left hand or the unaffected hand in patients with neurological disorders. We detected one SCR from the skin conductance signal, when signal amplitude changed by at least 0.05 Siemens in less than 5 seconds [22]. SCL was high-pass filtered with a 20 Hz Butterworth filter to remove sensory noise. We measured skin temperature on the distal phalanx of the fifth finger of the left hand or the unaffected hand in patients with neurological disorders and did not process it further. We decided to exclude facial EMG , since some patients with neurological disorders did not tolerate additional sensors attached to their faces while other patients with neurological disorders had half-sided paralysis of their facial muscles. We amplified the signals with the g.USBamp (Guger Technologies; Graz, Austria). We sampled signals at 512 Hz according to the recommendations of Malik [34]. All signal processing software was written in MATLAB (The MathWorks; Natick, Massachusetts).

Evaluation of Physiological Recordings

Using descriptive statistics, we investigated which physiological signals changed significantly between the different task-level conditions. We only analyzed the last minute of each 5-minute condition to ensure that steady state had been reached. We tested all conditions using the Friedman test followed by a Wilcoxon test for paired comparison. Bonferroni correction corrected multiple errors caused by the paired comparison. We set the significance level at p < 0.05.

The attachment of sensors for physiological recordings on the body was time consuming for a clinical application, demanding resources of the therapist and also reducing exercise time with the Lokomat. To improve clinical applicability of our approach, we were interested in understanding whether all recorded physiological signals were necessary to perform classification of mental engagement or whether the recorded data contained information from dependent variables. It could, for example, be possible that HR and breathing frequency would show a strong correlation. In this case, one of these signals could then be omitted in future recordings without degrading classification performance.

Therefore, we investigated which signals contained the most information in terms of variance explained and could be seen as major markers for changes in psychological states. We performed a PCA for each nondisabled subject and each patient with neurological disorders individually. In this analysis, we combined 5 minutes of data for each of the four conditions of each subject before the PCA was performed. Inputs to the PCA were HR, a discrete time series of HRV, a discrete time series of the number of SCR events, SCL, skin temperature, and a discrete time series of breathing frequency. PCA is a linear, orthogonal rotation, which projects the original data into a new coordinate system. In this new coordinate system, the first axis (or first principal component [PC]) explains most of the variance. The second PC explains the second most important variance, etc. The PCA is computed as

Equation 1.

where D = the original data, n = number of data points, m = number of physiological recordings, F = the loading factors, and A¢ = the activation coefficients. In our case, the original data consisted of a time series with n data points and m = 6 physical recordings (dimensions). The activation coefficients were a time series with six dimensions. The matrix of loading factors was the rotation matrix and defined the new coordinate system.

We computed how many factors (k) were necessary to explain more than 80 percent of the variance in all subjects (k[1,n]), where n = the dimensionality of original data (i.e., 6). We performed a factor rotation on these first k PCs to obtain a clearer picture, where input signals provided the largest variance. Factor rotation is a mathematical transformation that does not alter the subspace spanned by the PCs, but shifts the weight of an input, e.g., from the first PC to the second, while maintaining the orthogonality between the components.

Neural Network for Automatic Classification of Mental Engagement

For automatic classification of mental engagement from physiological recordings, we evaluated the effectiveness of a neural network. As a classifier, we trained a data-fitting neural network (Neural Network Fitting Tool [MATLAB]) containing 30 hidden-layer neurons. The neural network provided an estimation of the current state of mental engagement, based on the physiological recordings. Of the data, we took 20 percent as training data, 20 percent as validation, and 60 percent as testing data. As neural networks require labeled data during the training phase, we labeled the training data as 1 = baseline, 2 = underchallenged, 3 = challenged, or 4 = overchallenged. Learning was performed with the Levenberg-Marquardt back-propagation algorithm [35]. We selected the data necessary for training of the network such that an identification phase would not take longer than 20 percent of the whole training time in the DGO; we targeted reducing complexity of the methods in order to simplify and benefit the clinical process.

We assessed whether all six physiological signals (HR, HRV, SCL, SCR, breathing frequency, and skin temperature) were needed for classification of mental engagement and investigated which sensors might be omitted without significantly decreasing the classifier performance. The goal of this exercise was to improve clinical applicability of our approach by optimizing the number of signals required during clinical sessions; attaching the sensors to the patient's body was time consuming, requiring a lot of attention and effort from the therapist, reducing the time left for Lokomat training. Also, sensors such as facial electrodes were impractical for the patient population and the breathing sensors were perceived to be disturbing by the patients; patient comfort during the training could be increased without the breathing sensor and the total number of sensors needed could be minimized.

We compared the performance of the classifier by means of the mean squared classification error for two sets of input data. The neural network classified the mental engagement from all (n = 6) raw physiological data streams. We then performed classification of the current psychological state with a reduced data set that contained only the physiological recordings that were dominant in the first k PCs (k < n). We selected the number k of PCs by means of variance explained and targeted a value >80 percent.

Experimental Protocol

We subdivided each measurement into a practice time with the VR environment, a baseline recording without the VR environment, and the three task conditions (underchallenged, challenged, and overchallenged) (Figure 3). Although we designed the task such that it was possible for the subject to manage with little physical effort, it still involved a nonnegligible physical effort. Because patients with neurological disorders, a heterogeneous population, are already likely to exhibit large intersubject variability due to impairments of cognitive and motor ability (as well as the potential effects of medications), we decided in the experiment planning phase to avoid task-order randomization to keep the influence of physical effort comparable. Therefore, we decided to present the conditions in the just mentioned order. We started with the bored condition, which required the least control effort to succeed, and finished with the overchallenged condition, which required the most effort to accomplish the task.

The sequence of one measurement was-

· Practice time: subjects became acquainted with the effects of their movements on the system (controlling the system). The walking speed for the baseline and the balance of the measurement interval was maintained at 2 km/h. The task difficulty levels were set individually for each subject.
· Five-minute walking baseline: physiological signals were recorded in the Lokomat with 30 percent body-weight support and without the VR tasks.
· Three task conditions in the VR environment: the three task conditions were arranged in increasing levels of difficulty, each with a duration of 5 minutes. Five minutes was determined as a trade-off between the time required to reach a steady state in the physiological signals and to keep the exercise portion of the experiment time below 45 minutes for patients with neurological disorders, since it has been informally reported by physiotherapists to be the maximum time for patients to exercise in the Lokomat.

After the walking baseline and after each scenario, we requested that the subjects respond to the SAM. During the questionnaire response time, we turned off the VR environment.

RESULTS
Questionnaires
Nondisabled Subjects

According to the SAM, we were able to elicit the desired psychological state (bored, excited, overstressed) in the majority of nondisabled subjects. Arousal in the four conditions (baseline, underchallenged, challenged, and overchallenged) was significantly different among each of the conditions and increased monotonously with task levels. We found similar results for perceived physical effort. The valence dimension demonstrated significant differences only in the overchallenged condition when compared with all other conditions. The values from the overchallenged condition were significantly lower than all other conditions (Table 2).


Table 2.
Results of nondisabled subjects for Self-Assessment Manikin (SAM) questionnaire.

Condition
Arousal
Valence
Physical Effort

Baseline
1* (1-2)
4 (4-4)
1* (1-1)
Underchallenged
2* (2-2)
4 (4-4)
2* (1-2)
Challenged
4* (3-4)
4 (3-4)
3* (3-3)
Overchallenged
4* (4-4)
3* (2-4)
3* (3-4)

Note: Median (95% CI) of the SAM dimensions (arousal, valence, and physical effort).
*Significantly different from other conditions (baseline, underchallenged, challenged, overchallenged) in this dimension (p < 0.05).
CI = confidence interval.


Patients with Neurological Disorders

We found no significant changes between the different conditions in the SAM, either for arousal, valence, or physical effort.

Statistical Analysis of Recorded Physiological Signals
Nondisabled Subjects

In nondisabled subjects, we found statistically significant differences in several physiological signals. Table 3 summarizes all results.


Table 3.
Statistical results of physiological recordings in nondisabled subjects.

Physiological Recording
Baseline
Underchallenged
Challenged
Overchallenged

Heart Rate (bpm)
73.3*†‡ (60.3-81.8)
81.3†‡§ (68.0-91.2)
94.1 (77.8-103.0)
96.4 (76.3-102.9)
Breathing Frequency (cpm)
21.6*†‡ (20.3-24.5)
23.0†‡§ (22.2-26.1)
27.5 (24.9-30.3)
27.8 (25.0-29.4)
Skin Conductance Response
(No. of SCR/min)
0.2*†‡ (0.0-0.6)
1.0ठ(0.2-3.7)
3.1§ (0.6-5.3)
3.3 (0.4-6.2)
RMSSD (ms)
30.0†‡ (19.4-49.9)
27.5 (6.8-37.3)
25.5§ (5.0-60.2)
15.3 (4.7-63.0)
Skin Temperature (°C)
32.5*† (31.7-32.9)
30.8ठ(29.1-32.1)
31.5ठ(30.5-32.5)
32.0*† (31.3-32.7)

Note: Median (95% CI) of heart rate, breathing frequency, SCR, RMSSD, and skin temperature.
*Significantly different from underchallenged condition (p < 0.05).
Significantly different from challenged condition (p < 0.05).
Significantly different from overchallenged condition (p < 0.05).
§Significantly different from baseline (p < 0.05).
bpm = beats per minute, CI = confidence interval, cpm = cycles per minute, RMSSD = square root of mean squared differences of successive normal-to-normal intervals, SCR = skin conductance response.


HR increased significantly from baseline for all conditions with the VR task and for the challenged and overchallenged conditions compared with the underchallenged condition. We found the same significant changes for breathing frequency.

We also found similar results for the number of SCR. For all VR task conditions, the number of SCR increased significantly compared with baseline. In addition, the number of SCR increased significantly for the overchallenged condition compared with the underchallenged condition.

In the time domain of HRV, RMSSD decreased significantly from baseline for the challenged and overchallenged conditions. Furthermore, we found a significant decrease for the overchallenged condition compared with the underchallenged condition. We found no significant changes in the frequency domain.

We also found a significant decrease in skin temperature. The skin temperatures during the underchallenged and challenged conditions were significantly decreased when compared with the baseline and overchallenged condition.

Patients with Neurological Disorders

Compared with the very robust and variable physiological signals in nondisabled subjects, we found only three significant changes in patients with neurological disorders. HR increased significantly for the challenged condition (7.6%) and for the overchallenged condition (6.2%) compared with baseline (median 89.7 beats per minute, confidence interval [CI]: 77.5-103.2). Breathing frequency decreased significantly for the overchallenged condition (5.0%) compared to the challenged condition (median 27.7 cycles per minute, CI: 24.8-29.8).

Principal Component Analysis of Recorded
Physiological Signals

The first two PCs explained more than 70 percent of the variance in all subjects, nondisabled as well as patients with neurological disorders; the first three PCs explained more than 80 percent in all but one nondisabled subject and all patients with neurological disorders (Figure 5).


Figure 5. Variance explained by principal components (PCs) for (a) nondisabled subjects (n = 17) and (b) patients with neurological disorders (n = 10). As PC analysis (PCA) performs without loss of information, correlation between original signal and PCA decomposed signal must be 100 percent if all components are combined.

Figure 5.

Variance explained by principal components (PCs) for (a) nondisabled subjects (n = 17) and (b) patients with neurological disorders (n = 10). As PC analysis (PCA) performs without loss of information, correlation between original signal and PCA decomposed signal must be 100 percent if all components are combined.

Click Image to Enlarge. View as PowerPoint Slide

By PCA, we were able to directly separate the four conditions (baseline, underchallenged, challenged, and overchallenged) and visually display the results in a simple and easy-to-understand manner, whereas the results of the statistical analysis of the physiological data were unclear. Simplifying the display of important data reflecting the response of the patient to the exercise condition and the level of mental engagement will improve ease of use in the clinical setting and allow personalization of the exercise experience for the patient. A typical example is shown in Figure 6 for a nondisabled subject and in Figure 7 for a patient with neurological disorders. Loading factor 1 of this particular plot relies mainly on skin temperature, while loading factor 2 is a combination of HR and SCL (Tables 4 and 5). Each mental engagement condition may be visually distinguished from the other (as depicted by ellipsoidal boundaries in Figures 6 and 7), providing objective information to the therapist relevant for patient therapy. The temporal evolution of the PC activation coefficients is displayed by changes in color. The lighter the color of each condition, the earlier in the condition we recorded the data. The black arrows mark the general evolution of the activation coefficients over the time course of one condition. Although we performed classification on three PCs, we only plotted two dimensions, as a three-dimensional plot would be difficult to display.


Figure 6. First two activation coefficients of the principal component analysis (PCA) exemplarily shown for one nondisabled subject (subject 17), sepa-rated for conditions “baseline,” “underchallenged,” “challenged,” and “overchallenged,” plotted for whole length of each condition (5 minutes). Within one color, darkness of color symbolizes later time during condition. While PCA was performed at once on whole data set, colors illustrate temporal evolution of data over conditions. PC = principal component.

Figure 6.

First two activation coefficients of the principal component analysis (PCA) exemplarily shown for one nondisabled subject (subject 17), sepa-rated for conditions "baseline," "underchallenged," "challenged," and "overchallenged," plotted for whole length of each condition (5 minutes). Within one color, darkness of color symbolizes later time during condition. While PCA was performed at once on whole data set, colors illustrate temporal evolution of data over conditions. PC = principal component.

Click Image to Enlarge. View as PowerPoint Slide


Figure 7. First two activation coefficients of the principal component analysis (PCA) exemplarily shown for one patient with neurological disorders (patient 3), separated for conditions “baseline,” “underchallenged,” “challenged,” and “overchallenged,” plotted for whole length of each condi-tion (5 minutes). Within one color, darkness of color symbolizes later time during condition. While PCA was performed at once on whole data set, colors illustrate temporal evolution of data over conditions. PC = principal component.

Figure 7.

First two activation coefficients of the principal component analysis (PCA) exemplarily shown for one patient with neurological disorders (patient 3), separated for conditions "baseline," "underchallenged," "challenged," and "overchallenged," plotted for whole length of each condi-tion (5 minutes). Within one color, darkness of color symbolizes later time during condition. While PCA was performed at once on whole data set, colors illustrate temporal evolution of data over conditions. PC = principal component.

Click Image to Enlarge. View as PowerPoint Slide


Table 4.
Loading factors of first principal components (PCs) of nondisabled subject 17.

Physiological Recording
PC1
PC2

Heart Rate (bpm)
0.2471
0.6025
Heart Rate Variability (ms)
0.0168
0.0585
Skin Conductance Response (No. of SCR/min)
0.0002
0.0607
Skin Conductance Level (µm)
-0.1987
-0.7271
Breathing Frequency (cpm)
-0.0093
-0.0668
Skin Temperature (°C)
-0.9482
0.3111

Note: Physiological recordings used to extract PCs.
bpm = beats per minute, cpm = cycles per minute.



Table 5.
Loading factors of first principal components (PCs) of patient with neurological disorders 3.

Physiological Recording
PC1
PC2

Heart Rate (bpm)
-0.0221
0.0284
Heart Rate Variability (ms)
0.0465
-0.0602
Skin Conductance Response (No. of SCR/min)
-0.3028
-0.8259
Skin Conductance Level (µm)
-0.6127
0.0150
Breathing Frequency (cpm)
0.0236
-0.4651
Skin Temperature (°C)
0.7278
-0.3112

Note: Physiological recordings used to extract PCs.
bpm = beats per minute, cpm = cycles per minute.


Evaluation of the loading factors (i.e., the rotation matrix obtained from the PCA) of all nondisabled subjects and all patients with neurological disorders revealed that PC1 was dominated by skin temperature and SCL in both groups (Figure 8). HRV played a minor role in the first three PCs for both groups. While breathing frequency was not dominant in nondisabled subjects, data variability of patients with neurological disorders showed a major dependency of breathing frequency in PC3.


Figure 8. Comparison between loading factors of first three principal components (PCs) of (a) nondisabled subjects (n = 17) and (b) patients with neuro-logical disorders (n = 10). Physiological recordings used to extract PCs are heart rate (HR), heart rate variability (HVR), skin conductance response (SCR), skin conductance level (SCL), breathing frequency (BF), and skin temperature (ST).

Figure 8.

Comparison between loading factors of first three principal components (PCs) of (a) nondisabled subjects (n = 17) and (b) patients with neuro-logical disorders (n = 10). Physiological recordings used to extract PCs are heart rate (HR), heart rate variability (HVR), skin conductance response (SCR), skin conductance level (SCL), breathing frequency (BF), and skin temperature (ST).

Click Image to Enlarge. View as PowerPoint Slide

Online Classification of Task Level Difficulty

Although we only found two significant differences in all physiological recordings over all conditions of patient data, the classification of the different psychological states using a neural network was possible for nondisabled subjects and also for patients with neurological disorders. As described in the "Methods" section, we evaluated the classification results on a neural network for two different sets of input data: on one side with six physiological parameters extracted and on the other side using only the physiological signals dominant in the first three PCs. Mean classification error was 1.4 percent for the full and 2.5 percent for the reduced data set in nondisabled subjects and 2.1 percent for the full and 4.7 percent for the reduced data set for patients with neurological disorders (Figure 9).


Figure 9. Classification error by means of root-mean-square error (RMSE) for all 17 nondisabled subjects (left) and all 10 patients with neurological dis-orders (right), using all six physiological signals (full data set) and using only three signals that were dominant in first three principal components (reduced data set). Light gray = full data set, white = reduced data set.

Figure 9.

Classification error by means of root-mean-square error (RMSE) for all 17 nondisabled subjects (left) and all 10 patients with neurological dis-orders (right), using all six physiological signals (full data set) and using only three signals that were dominant in first three principal components (reduced data set). Light gray = full data set, white = reduced data set.

Click Image to Enlarge. View as PowerPoint Slide

In 10 of 17 nondisabled subjects, the baseline and three different task conditions could be visually classified by plotting the first two PCs (Figure 6). In these subjects, automatic classification by a neural network was possible with a very low level of classification error measured by the root-mean-square error (RMSE). In 5 of 17 nondisabled subjects, a graphical separation (as in Figure 6) was not possible, while the neural network still performed with less than 3 percent RMSE. Classification of the remaining 2 of 17 nondisabled subjects was possible with 5 and 13 percent RMSE, respectively (Figure 9).

In 8 of 10 patients with neurological disorders, classification was possible with less than 5 percent error when using the full set of all physiological signals. Similar to the results from nondisabled subjects, visual separation was possible in these patients from a plot of the first two PCs (Figure 7). Figure 9 displays the classification results for the different sets of inputs. In patients with neurological disorders, the classification was successful, but to a lesser extent. Particularly in patient 10, the classification error was 18 percent for a reduced set of input data. Using all recorded physiological data, an overall classification error of less than 7 percent was possible for all patients. Training of the neural network was possible in <1 minute on a standard Pentium processor (1.4 GHz, 2 GB RAM [Intel; Santa Clara, California]).

DISCUSSION

We performed VR-enhanced robot-assisted treadmill training in nondisabled subjects and patients with neurological disorders, and we were able to demonstrate that psychophysiological measurements alone were sufficient to detect current psychological state. We parametrically manipulated the difficulty of a VR task during the rehabilitation training; we established three levels of difficulty comprised of underchallenging, challenging, and overchallenging to solicit experiences of boredom, excitement, or stress. We recorded ECG  , breathing frequency, skin temperature, and the GSR. The psychological state of a subject could be classified using a combination of PCA with a neural network. We found that HR, SCL, and skin temperature can be used as markers for psychological states in the presence of physical effort (corresponding very well with the gold standard instrument to measure psychological state, the SAM), associated with the exercise typically used for patients with neurological injury. We believe that patients will benefit from the provision of an objective measure to automatically assess psychological state during rehabilitation so that measures can be taken to adjust the exercise conditions accordingly.

Objective Classification of Mental Engagement

Evaluation of questionnaires from nondisabled subjects confirmed that VR tasks of different difficulty levels can indeed result in different levels of mental engagement, i.e., of being bored, excited, or overstressed. Also in nondisabled subjects, descriptive statistics would suffice to distinguish between different levels of mental engagement: the automatic classification worked in all but two subjects with less than 2 percent classification error.

In patients with neurological disorders, neither questionnaires nor physiological signals showed a picture as clearly as in nondisabled subjects. While therapists anecdotally reported that the VR task bored, exciting, or overstressed the patients with neurological disorders, the questionnaires did not confirm this observation. The Lokomat is typically used in gait rehabilitation of patients with little to no walking ability. In order to ensure that our approach is clinically applicable in the future, we explicitly recorded data from severely affected patients. A possible explanation of the poor results from questionnaires is that patients with cerebral lesions might have cognitive deficits, which might prevent them from assessing, expressing, and verbalizing their level of mental engagement during rehabilitation. This was consistent with reports by the therapists. In addition, patients did not usually admit if they experienced the task as too difficult, perhaps because they were very ambitious and also determined to successfully solve the task. Although the questionnaire showed only few changes, every patient with neurological disorders achieved the predefined scores (success rate of 100% for the underchallenged condition, of 80% to 90% for the challenged condition, and of 10% to 20% for the overchallenged condition). Another explanation might be the fact that walking with the help of the DGO was an experience that is positively perceived by patients with neurological disorders who may otherwise be unable to walk well on their own.

Despite the heterogeneous and unclear nature of the picture in the descriptive statistical analysis of the physiological data of patients with neurological disorders, the classification of the various conditions in the VR environment was possible with less than 8 percent classification error in all patients (Figure 9). In this context, a real-time automatic classification algorithm applied to physiological recordings seems to allow an objective estimation of mental engagement for the benefit of the patient in clinical applications, and in particular, when compared with the sometimes conflicting and often unreliable subjective information obtained from other sources.

Influence of Physical Activity, Neurological Deficits, and Medication

To our best knowledge, no one has previously performed estimation of psychophysiological states in patients with neurological disorders during walking. We expected our results to be altered compared with previous research because of the influence of physical activity caused by walking and changes in autonomous responses of the CNS in patients with neurological disorders.

HRV was previously used as a major marker for stress [36] and has also been shown to decrease with increased physical effort [37], which is the case in treadmill walking. Physical effort of walking might have occluded psychological influence of stress on HRV, which is why HRV might not have changed significantly in our subjects. Although we tried to keep the physical effort necessary for controlling the VR task as low as possible, the statistical evaluation confirmed that HRV did not vary significantly.

The lack of statistically significant changes in physiological responses of patients might be explained by the effects of the lesion. In patients with stroke, a disturbance in the autonomic functions caused by the lesion was often described to affect physiological processes in cerebrovascular diseases [38-40]. In the context of a decrease in HRV, i.e., SD of RR intervals, we found low frequency and high frequency for patients with stroke [38,40]. In addition, Korpelainen et al. showed that skin temperature is lower on the contralesional side after stroke [41] and Naver et al. showed that the sympathetic GSR were altered in amplitude and delay [42]. Furthermore, medication of patients with stroke can influence physiological signals, for example beta blockers, which influence the cardiovascular response. In patients with SCI with cervical lesions, ANS control might be disturbed as well, because the sympathetic nerve fibers that innervate the heart leave the CNS between thoracic (T) 1 and T4, with implications for the design of applications of these measures in real time for such patient populations.

Medication might have altered ANS control in addition to changes caused by the lesion. Of our 10 patients with neurological disorders (Table 1), 3 received beta-blocking medication at the time of recording, which is known to limit HR adaptation to increased physical stress. The average classification error (Figure 9) might have been lower if we had excluded patients on beta-blocking medication. Nevertheless, we decided to include such patients, because exclusion would have weakened the usability of our system in a clinical setting outside a laboratory environment.

Relevant Physiological Signals for Classification of Mental Engagement

We applied a PCA to identify the minimum set of physiological signals that would be necessary to perform classification of mental engagement while not degrading the performance of the classifier. In both nondisabled subjects and patients with neurological disorders, skin temperature and SCL were the main psychophysiological responders to our intervention. Also, HRV did not contribute significantly to the first three PCs. As discussed, HRV might have been reduced because of the physical effort involved in walking.

Relevant for clinical use are the results in patients with neurological disorders: the loading factor of HR only occurred with a noticeable magnitude in the third PC. Because HRV and HR, which are both computed from the ECG , did not significantly contribute to the variance of the recorded data in patients, the ECG might not be required in future experiments exploring mental engagement. Classification results of a data set containing only the data dominant in the first three PCs confirmed that classification performance dropped by 1 percent. While skin temperature and SCL sensors are easily attached to the subjects' unaffected finger, attachment of the ECG electrodes requires more time. In clinical day-to-day use of the system, reducing the complexity of data collection in this manner would shorten the time necessary to prepare a patient for therapy.

Real-Time Capability of Approach

Previously, VR environments in rehabilitation did not provide patient-specific and adaptive features, with the ability to adjust difficulty levels that correlated with mental engagement. The long-term goal of our work is to perform closed-loop control of psychological states during robot-assisted rehabilitation-to objectively determine the psychological state of the patient, and then automatically adjust or enable the therapist to adjust the attributes of the system to stimulate a desired level of mental engagement during rehabilitation. The goal of approaching optimal mental engagement during exercise in rehabilitation is consistent with evidence that attention to task and mental engagement improve outcomes of rehabilitation efforts in the training of motor skills. Previously, researchers have used closed-loop control of psychophysiological measurements to control stress. Rani et al. performed closed-loop control of mental stress in sitting nondisabled subjects and used changes in HRV and GSR for classification of the stress level [36]. Prinzel et al. used the electroencephalogram to moderate an operator's level of task engagement through a closed-loop system [43]. Yamamoto and Isshiki used galvanic skin reflex to control the awake-level of a subject [44].

During the design phase, we ensured the real-time capability of our approach. Processing of physiological signals was done in real time; we evaluated skin temperature using a moving average filter, with an average time of 1 s, and processed ECG with an adaptive steep slope beat detection algorithm [33] to extract HR and HRV with a delay of <100 ms. We transferred an adaptation of an offline GSR extraction algorithm from SCR to perform in real time with a 15 s delay and detected breathing frequency online using a peak detection algorithm with a <100 ms delay. The adaptation of the task level difficulty in the VR environment could be done in real time as well, such that the subject will not observe the changes-to be implemented through adaptation algorithms-as they happen.

The classifier network currently needs an initial training phase, after which MATLAB provides a Simulink model of the network that is real-time capable. The length of the identification phase must be determined separately, but the temporal changes in the first two PCs as shown in Figure 6 indicate that the different levels of difficulty resulted in rapid changes in physiological data. We expect the identification phase to last only a few minutes, which will be reasonable in a clinical setting when compared with a normal Lokomat exercise session time of 45 minutes. Approaches like reinforcement learning could also be implemented to perform classification of the psychophysiological state of subjects. Reinforcement learning would require a stable physiological response to a stimulating input. Vidaurre et al. found a direct correlation between the personally perceived stress level and SCL in nondisabled subjects performing a VR task while sitting on a chair [45]. In such a case, the SCL could be exploited by a reinforcement learning algorithm for classification and subsequent control of stress.

Usability in Clinical Setting

We recommend minimizing or avoiding the currently required identification phase of approximately 10 minutes when the system is transferred from laboratory use to day-to-day clinical use for real-time assessment of psychological states during robot-assisted gait therapy. Several strategies may be used. Supervised learning algorithms such as reinforcement learning would require labeled data. Labeled data would only be available if the state of arousal and valence of the patient is rated during the identification phase (either by the patient or by the therapist). This means that the system in its current state of development can be used only by patients that are cognitively capable of answering the SAM questionnaires

Shortening the identification phase is possible by using a simplified questionnaire: for example, asking only if the task is "too easy," "just right," or "too hard." This would allow labeling the data even for severely cognitively impaired patients. Alternatively, it might suffice to only perform identification for each patient once, during the first rehabilitation session, and then to use this patient-individual classifier for consecutive training days. Whether or not the physiological responses recorded on consecutive days are stable and repeatable will require further investigation.

Alternatively, future implementations could make use of adaptive algorithms that can learn in an unsupervised manner. An idea proposed by Vidaurre et al. suggests use of a classifier in combination with a Kalman filter to train the classifier at run time [45]. The classifier, such as a neural network, could be trained on data from previous experiments and a Kalman filter would adapt the classifier parameters to the patient. With this technique, an initialization and patient responses from a questionnaire would not be necessary, because by using this strategy the system would learn while the gait training is already in progress.

Extension of Our Approach to Upper Limbs

Our approach is not limited to a particular gait orthosis and is also not limited to rehabilitation of the lower limbs. In robot-assisted arm rehabilitation, as performed with the ARMin (Eidenössische Technische Hochschule Zurich; Zurich, Switzerland) [46-47], the Moog FCS HapticMASTER (FCS Robotics; Schipols, the Netherlands) [48], or the MIT Manus (Massachusetts Institute of Technology; Cambridge, Massachusetts) [49], the lower level of physical effort (as compared with walking) might even improve the results obtained from nondisabled subjects as well as patients with neurological disorders.

Study Limitations

There are three limitations to our study: lack of task-order randomization, the heterogeneous group of patients, and differences between the VR setups for patients with neurological disorders and nondisabled subjects. First, the order in which the nondisabled control and patient groups performed the tasks was not randomized. Hence, the results of the physiological recordings may have been influenced by the task order. We are aware that from a methodological viewpoint, a fully randomized task order would have been preferable but we consciously decided to use a nonrandomized order. This ensured maintaining a comparable level of physical effort among subjects. In addition, patients with neurological gait impairments are assumed to already show a large intersubject variability because of their motor and cognitive impairments. Although we chose a task in which higher task level difficulty could be achieved with little increase in physical effort, a task order randomization would have certainly further increased the variability in performance measures among subjects. Thus we suspected that a further increased variability could have made a comparison between subjects impossible and might hinder uncovering potentially important results.

Second, the patient group was comprised of patients with different neurological gait impairments, some of them taking medication such as beta-blocking medicine that altered physiological responses. For statistically significant physiological data, a homogeneous group of patients with stroke who were not taking beta-blocking medication would have been preferable. We decided to establish mental state classification in a broad variety of patient groups reflecting the heterogeneous population of realistic clinical patients since we are concerned with designing and developing a practical tool for use in a clinical setting. Therefore, we included a variety of subjects in terms of lesion (stroke, SCI) and medication (beta-blocking medication) to cover the whole range of characteristics of patients who might be present in a rehabilitation environment and might benefit from a combination of a DGO with VR environments.

Third, we had to use a larger screen for display of the VR environment for the nondisabled subjects than for the patients with neurological disorders, because the back-projection screen was installed in front of the Lokomat at Balgrist University Hospital, where we conducted recordings with nondisabled subjects. The larger screen size potentially elicited stronger responses in nondisabled subjects compared with patients with neurological disorders, who saw the VR environment on a 42-inch screen. This lower experience of immersion could explain the fewer significantly different results in physiological recordings of patients with neurological disorders compared with nondisabled subjects.

CONCLUSIONS

The key result of our study is that real-time objective assessment of mental engagement is possible using psychophysiological measurements. For the first time, this assessment of engagement can be performed in patients with neurological disorders during gait training. The next step will be implementation of real-time control of the psychophysiological state of a patient during rehabilitation training by adjusting the stimulus of the task being performed. Future work includes investigating modification of algorithms. Neural networks are nonadaptive after training them. We will implement other classifiers such as Bayesian networks, reinforcement learning, or fuzzy-C-mean clustering, which have already been used in automatic classification of emotional states.

A major contribution to the day-to-day clinical applicability of our approach will come from the use of Kalman filters for subject-specific adaptation of classifier parameters. These methods are currently under development and will be tested in the near future.

Closed-loop control of mental engagement has the potential to improve robot-assisted rehabilitation in the future. Methods of implementing robot-assisted rehabilitation to overcome challenges of the clinical environment and to provide engaging patient-specific therapies are in an early stage of evolution. Particularly in the field of gait rehabilitation, the effectiveness of robot-assisted intervention is not yet fully realized. The Lokomat gait orthosis provides the advantage of reducing the burden on clinical staff during weight-supported gait training and has been found by some researchers to have advantages over conventional therapies [50-51]. By addressing important psychological aspects of mental engagement in technology-assisted therapies, innovations may be made that help bridge the gap found by some researchers with conventional therapies [52-53]. Our approach might help to improve some of the features of technology-assisted rehabilitation-taking a step toward meeting more of the needs of the patient and therapist.

ACKNOWLEDGMENTS
Author Contributions:
Study concept and design: A. Koenig, R. Riener.
Acquisition of data: A. Koenig, X. Omlin, M. Sapa, C. Krewer.
Analysis and interpretation of data: A. Koenig, X. Omlin,
M. Bolliger.
Drafting of manuscript: A. Koenig, X. Omlin.
Statistical analysis: X. Omlin, M. Bolliger.
Technical, clinical, or material support: M. Sapa, C. Krewer,
F. Müeller.
Study supervision: M. Bolliger, F. Müller, R. Riener.
Financial Disclosures: Mr. Zimmerli is employed by Hocoma AG , Volketswil, Switzerland, manufacturer of the Lokomat. However, Hocoma as a company was not involved in the planning, writing, finalizing, or approval process of the publication of this article. Mr. Zimmerli's contributions to this article were based on his independent scientific motivation and scientific background. In addition, Mr. Zimmerli's contributions to this article resulted from the long-term scientific collaborations and research partnerships among Eidenössische Technische Hochschule Zurich, the Balgrist University Hospital, and Hocoma.
Funding/Support: This material was based on work supported in part by the Swiss National Science Foundation, National Centers of Competence in Research on Neural Plasticity and Repair under Project P8 Rehabilitation Technology Matrix and by the European Union Project Multimodal Immersive Motion Rehabilitation with Interactive Cognitive Systems funded by the European Community's Seventh Framework Program (FP7/2007-2013, grant 215756).
Additional Contributions: We thank Domen Novak and Marko Munih for methodological help. Furthermore, we want to thank Katherine G . August, Whitaker International Scholar, for her exceptional support in the organization of structure and language of this article. Dr. Sapa is now with the Lech-Mangfall-Klinken gGmbH, Hausham, Germany.
Institutional Review: We obtained approval for both studies from local ethics committees, and all subjects or their legal representative gave written informed consent before data collection.
Participant Follow-Up: The authors do not plan to inform participants of the publication of this study.
REFERENCES
1. Dietz V, Duysens J. Significance of load receptor input during locomotion: A review. Gait Posture. 2000;11(2):102-10.
[PMID: 10899663]
DOI:10.1016/S0966-6362(99)00052-1
2. Dietz V, Wirz M, Curt A, Colombo G . Locomotor pattern in paraplegic patients: Training effects and recovery of spinal cord function. Spinal Cord. 1998;36(6):380-90.
[PMID: 9648193]
DOI:10.1038/sj.sc.3100590
3. Colombo G , Joerg M, Schreier R, Dietz V. Treadmill training of paraplegic patients using a robotic orthosis. J Rehabil Res Dev. 2000;37(6):693-700. [PMID: 11321005]
4. Hesse S, Sarkodie-Gyan T, Uhlenbrock D. Development of an advanced mechanised gait trainer, controlling movement of the centre of mass, for restoring gait in non-ambulant subjects. Biomed Tech (Berl). 1999;44(7-8):194-201.
[PMID: 10472726]
DOI:10.1515/bmte.1999.44.7-8.194
5. Stauffer Y, Allemand Y, Bouri M, Fournier J, Clavel R, Metrailler P, Brodard R, Reynard F. The WalkTrainer-A new generation of walking reeducation device combining orthoses and muscle stimulation. IEEE Trans Neural Syst Rehabil Eng. 2009;17(1):38-45. [PMID: 19211322]
DOI:10.1109/TNSRE.2008.2008288
6. Veneman JF, Kruidhof R, Hekman EE, Ekkelenkamp R, Van Asseldonk EH, Van der Kooij H. Design and evaluation of the LOPES exoskeleton robot for interactive gait rehabilitation. IEEE Trans Neural Syst Rehabil Eng. 2007; 15(3):379-86. [PMID: 17894270]
DOI:10.1109/TNSRE.2007.903919
7. Winchester P, Querry R. Robotic orthoses for body weight-supported treadmill training. Phys Med Rehabil Clin N Am. 2006;17(1):159-72. [PMID: 16517349]
DOI:10.1016/j.pmr.2005.10.008
8. Lotze M, Braun C, Birbaumer N, Anders S, Cohen LG . Motor learning elicited by voluntary drive. Brain. 2003; 126(Pt 4):866-72. [PMID: 12615644]
DOI:10.1093/brain/awg079
9. Banz R, Bolliger M, Colombo G , Dietz V, Lünenburger L. Computerized visual feedback: An adjunct to robotic-assisted gait training. Phys Ther. 2008;88(10):1135-45.
[PMID: 18772279]
DOI:10.2522/ptj.20070203
10. Maclean N, Pound P. A critical review of the concept of patient motivation in the literature on physical rehabilitation. Soc Sci Med. 2000;50(4):495-506. [PMID: 10641802]
11. Guadagnoli MA, Lee TD. Challenge point: A framework for conceptualizing the effects of various practice conditions in motor learning. J Mot Behav. 2004;36(2):212-24.
[PMID: 15130871]
DOI:10.3200/JMBR.36.2.212-224
12. Russell JA. A circumplex model of affect. J Pers Soc Psychol. 1980;39(6):1161-78. DOI:10.1037/h0077714
13. Holden MK. Virtual environments for motor rehabilitation: Review. Cyberpsychol Behav. 2005;8(3):187-211.
[PMID: 15971970]
DOI:10.1089/cpb.2005.8.187
14. McAuley E, Duncan T, Tammen VV. Psychometric properties of the Intrinsic Motivation Inventory in a competitive sports setting: A confirmatory factor analysis. Res Q Exerc Sport. 1989;60(1):48-58. [PMID: 2489825]
15. Andreassi JL. Psychophysiology: Human behavior and physiological response. 5th ed. Hillsdale (NJ): Lawrence Erlbaum Associates; 2007.
16. Hugdahl K. Psychophysiology: The mind-body perspective. Cambridge (MA): Harvard University Press; 1995.
17. Mulder G , Mulder L, Veldman J, Van Roon A. A psychophysiological approach to working conditions. In: Backs RW, Boucsein W, editors. Engineering psychophysiology: Issues and applications. Mahwah (NJ): Lawrence Erlbaum Associates; 2000. p. 139-59.
18. De la Cruz Torres B, López López C, Naranjo Orellana J. Analysis of heart rate variability at rest and during aerobic exercise: A study in healthy people and cardiac patients. Br J Sports Med. 2008;42(9):715-20. [PMID: 18199627]
DOI:10.1136/bjsm.2007.043646
19. Delaney JP, Brodie DA. Effects of short-term psychological stress on the time and frequency domains of heart-rate variability. Percept Mot Skills. 2000;91(2):515-24.
[PMID: 11065312]
DOI:10.2466/PMS.91.6.515-524
20. Cacioppo JT, Tassinary LG  , Berntson GG . Handbook of psychophysiology. 2nd ed. New York (NY): Cambridge University Press; 2000.
21. Boucsein W. Electrodermal measurement. In: Stanton N, Hedge A, Brookhuis K, Salas E, Hendrick H, editors. Handbook of human factors and ergonomics methods. Boca Raton (FL): CRC Press; 2005. p. 182-90.
22. Dawson ME, Schell AM, Filion DL. The electrodermal system. In: Cacippo JT, Tassinary LG , Berntson GG , editors. Handbook of psychophysiology. 2nd ed. Cambridge (UK): Cambridge University Press; 2000. p. 200-223.
23. Suess WM, Alexander AB, Smith DD, Sweeney HW, Marion RJ. The effects of psychological stress on respiration: A preliminary study of anxiety and hyperventilation. Psychophysiology. 1980;17(6):535-40. [PMID: 7443919]
DOI:10.1111/j.1469-8986.1980.tb02293.x
24. Carroll D, Turner JR, Prasad R. The effects of level of difficulty of mental arithmetic challenge on heart rate and oxygen consumption. Int J Psychophysiol. 1986;4(3):167-73. [PMID: 3793559]
DOI:10.1016/0167-8760(86)90012-7
25. Rousselle JG , Blascovich J, Kelsey RM. Cardiorespiratory response under combined psychological and exercise stress. Int J Psychophysiol. 1995;20(1):49-58. [PMID: 8543484]
DOI:10.1016/0167-8760(95)00026-O
26. Ohsuga M, Shimono F, Genno H. Assessment of phasic work stress using autonomic indices. Int J Psychophysiol. 2001;40(3):211-20. [PMID: 11228348]
DOI:10.1016/S0167-8760(00)00189-6
27. Mancuso DL, Knight KL. Effects of prior physical activity on skin surface temperature response of the ankle during and after a 30-minute ice pack application. J Athl Train. 1992;27(3):242-49. [PMID: 16558168]
28. Brown SL, Schwartz GE. Relationships between facial electromyography and subjective experience during affective imagery. Biol Psychol. 1980;11(1):49-62.
[PMID: 7248403]
DOI:10.1016/0301-0511(80)90026-5
29. Larsen JT, Norris CJ, Cacioppo JT. Effects of positive and negative affect on electromyographic activity over zygomaticus major and corrugator supercilii. Psychophysiology. 2003;40:776-85. [PMID: 14696731]
DOI:10.1111/1469-8986.00078
30. Zimmerli L, Duschau-Wicke A, Riener R, Mayr A, Lunenburger L. Virtual reality and gait rehabilitation. Proceedings of the Virtual Rehabilitation International Conference; 2009 Jun 29-Jul 2; Haifa, Israel. Los Alamitos (CA): IEEE; 2009. p. 150-53. DOI:10.1109/ICVR.2009.5174223
31. Bradley MM, Lang PJ. Measuring emotion: The Self-Assessment Manikin and the semantic differential. J Behav Ther Exp Psychiatry. 1994;25(1):49-59. [PMID: 7962581]
DOI:10.1016/0005-7916(94)90063-9
32. Morris JD. SAM: The Self-Assessment Manikin. An efficient cross-cultural measurement of emotional response. J Advert Res. 1995;35:63-68.
33. Christov I. Real time electrocardiogram QRS detection using combined adaptive threshold. Biomed Eng Online. 2004; 3(1):28. [PMID: 15333132]
DOI:10.1186/1475-925X-3-28
34. Malik M. Heart rate variability. Standards of measurement, physiological interpretation, and clinical use. Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology. Eur Heart J. 1996;17:28.
35. Moré JJ. The Levenberg-Marquardt algorithm: Implementation and theory. Numerical Analysis. 1978;630:105-16.
DOI:10.1007/BFb0067700
36. Rani P, Sims J, Brackin R, Sarkar N. Online stress detection using psychophysiological signals for implicit human-robot cooperation. Robotica. 2002;20:673-85.
DOI:10.1017/S0263574702004484
37. Bernardi L, Valle F, Coco M, Calciati A, Sleight P. Physical activity influences heart rate variability and very-low-frequency components in Holter electrocardiograms. Cardiovasc Res. 1996;32(2):234-37. [PMID: 8796109]
DOI:10.1016/0008-6363(96)00081-8
38. Korpelainen JT, Sotaniemi KA, Huikuri HV, Myllylä VV. Abnormal heart rate variability as a manifestation of autonomic dysfunction in hemispheric brain infarction. Stroke. 1996; 27(11):2059-63. [PMID: 8898816]
39. Meyer S, Strittmatter M, Fischer C, Georg T, Schmitz B. Lateralization in autonomic dysfunction in ischemic stroke involving the insular cortex. Neuroreport. 2004;15(2):357-61.
[PMID: 15076768]
DOI:10.1097/00001756-200402090-00029
40. Tokgözoglu SL, Batur MK, Top Uoglu MA, Saribas O, Kes S, Oto A. Effects of stroke localization on cardiac autonomic balance and sudden death. Stroke. 1999;30(7):1307-11.
[PMID: 10390300]
41. Naver H, Blomstrand C, Ekholm S, Jensen C, Karlsson T, Wallin G . Autonomic and thermal sensory symptoms and dysfunction after stroke. Stroke. 1995;26(8):1379-85.
[PMID: 7631341]
42. Korpelainen JT, Tolonen U, Sotaniemi KA, Myllylä VV. Suppressed sympathetic skin response in brain infarction. Stroke. 1993;24(9):1389-92. [PMID: 8362436]
43. Prinzel LJ, Freeman FG , Scerbo MW, Mikulka PJ, Pope AT. A closed-loop system for examining psychophysiological measures for adaptive task allocation. Int J Aviat Psychol. 2000;10(4):393-410. [PMID: 11762443]
DOI:10.1207/S15327108IJAP1004_6
44. Yamamoto Y, Isshiki H. Instrument for controlling drowsiness using galvanic skin reflex. Med Biol Eng Comput. 1992;30(5):562-64. [PMID: 1293449]
DOI:10.1007/BF02457838
45. Vidaurre C, Schlögl A, Cabeza R, Scherer R, Pfurtscheller G . Study of on-line adaptive discriminant analysis for EEG-based brain computer interfaces. IEEE Trans Biomed Eng. 2007;54(3):550-56. [PMID: 17355071]
DOI:10.1109/TBME.2006.888836
46. Nef T, Guidali M, Riener R. ARMin III-Arm therapy exoskeleton with an ergonomic shoulder actuation. Appl Bionics Biomech. 2009;6:127-42.
DOI:10.1080/11762320902840179
47. Nef T, Mihelj M, Riener R. ARMin: A robot for patient-cooperative arm therapy. Med Biol Eng Comput. 2007; 45(9):887-900. [PMID: 17674069]
DOI:10.1007/s11517-007-0226-6
48. Houtsma JA, Van Houten FJ. Virtual reality and a haptic master-slave set-up in post-stroke upper-limb rehabilitation. Proc Inst Mech Eng H. 2006;220(6):715-18.
[PMID: 16961191]
DOI:10.1243/09544119H06104
49. Aisen ML, Krebs HI, Hogan N, McDowell F, Volpe BT. The effect of robot-assisted therapy and rehabilitative training on motor recovery following stroke. Arch Neurol. 1997; 54(4):443-46. [PMID: 9109746]
50. Husemann B, Müller F, Krewer C, Heller S, Koenig E. Effects of locomotion training with assistance of a robot-driven gait orthosis in hemiparetic patients after stroke: A randomized controlled pilot study. Stroke. 2007;38(2):349-54.
[PMID: 17204680]
DOI:10.1161/01.STR.0000254607.48765.cb
51. Mayr A, Kofler M, Quirbach E, Matzak H, Fröhlich K, Saltuari L. Prospective, blinded, randomized crossover study of gait rehabilitation in stroke patients using the Lokomat gait orthosis. Neurorehabil Neural Repair. 2007;21(4):307-14.
[PMID: 17476001]
DOI:10.1177/1545968307300697
52. Hidler J, Nichols D, Pelliccio M, Brady K, Campbell DD, Kahn JH, Hornby TG . Multicenter randomized clinical trial evaluating the effectiveness of the Lokomat in subacute stroke. Neurorehabil Neural Repair. 2009;23(1):5-13.
[PMID: 19109447]
DOI:10.1177/1545968308326632
53. Hornby TG , Campbell DD, Kahn JH, Demott T, Moore JL, Roth HR. Enhanced gait-related improvements after therapist- versus robotic-assisted locomotor training in subjects with chronic stroke: A randomized controlled study. Stroke. 2008;39(6):1786-92. [PMID: 18467648]
DOI:10.1161/STROKEAHA.107.504779
Submitted for publication March 26, 2010. Accepted in revised form October 6, 2010.
This article and any supplementary material should be cited as follows:
Koenig A, Omlin X, Zimmerli L, Sapa M, Krewer C, Bolliger M, Müller F, Riener R. Psychological state estimation from physiological recordings during robot-assisted gait rehabilitation. J Rehabil Res Dev. 2011; 48(4):367-86.
DOI:10.1682/JRRD.2010.03.0044
Crossref

Go to TOP

Go to the Table of Contents of Vol. 48 No. 4

Last Reviewed or Updated  Friday, May 13, 2011 9:36 AM

Valid HTML 4.01 Transitional