Journal of Rehabilitation Research & Development (JRRD)

Quick Links

  • Health Programs
  • Protect your health
  • Learn more: A-Z Health
Veterans Crisis Line Badge
 

Volume 53 Number 6, 2016
   Pages 959 — 972

Utility of a multimodal neurophysiological assessment tool in distinguishing between individuals with and without a history of mild traumatic brain injury

Martin Baruch, PhD;1* Jeffrey T. Barth, PhD;2 David Cifu, MD;3 Martin Leibman, PhD4

1Empirical Technologies Inc, Charlottesville, VA; 2University of Virginia School of Medicine, Charlottesville, VA; 3Virginia Commonwealth University School of Medicine, and Hunter Holmes McGuire Department of Veterans Affairs Medical Center, Richmond, VA; 4Kenner Army Health Clinic, Fort Lee, VA

Abstract — This was a preliminary validation study of a multimodal concussion assessment battery incorporating eye-tracking, balance, and neurocognitive tests using a new hardware platform, the Computerized Brain Injury Assessment System. Using receiver-operating characteristics analyses, (1) we identified a subset of the most discriminating neurophysiological assessment tests, involving smooth pursuit eye movement tracking errors, corrective saccade counts, a balance score ratio sensitive to vestibular balance performance, and two neurocognitive tests of response speed and memory/incidental learning; (2) we demonstrated the enhancement in discriminatory capability of detecting concussion-related deficits through the combination of the identified subset of assessments; and (3) we demonstrated the effectiveness of a robust and readily implemented global scoring approach for both eye track and balance assessment tests. These results are significant in introducing a comprehensive solution for concussion assessment that incorporates an economical, compact, and mobile hardware system as well as an assessment battery that is multimodal and time efficient and whose efficacy has been demonstrated on a preliminary basis. This represents a significant step toward the goal of a system capable of making a dependable return-to-play/duty determination based on concussion likelihood.

Key words: assessment, balance assessment, concussion, eye tracking, mild traumatic brain injury, multimodal, neurocognitive, neurophysiological, saccade count, stability.

Abbreviations: AUC = area-under-the-curve, CBIAS = Computerized Brain Injury Assessment System, CSI = Cognitive Stability Index, IR = infrared, LED = light-emitting diode, mTBI = mild traumatic brain injury, ROC = receiver-operator-characteristics, SD = standard deviation, S/N = signal to noise ratio, SPEM = smooth pursuit eye movement, TBI = traumatic brain injury, VAMC = Department of Veterans Affairs Medical Center.
*Address all correspondence to Martin Baruch, PhD; Empirical Technologies, 3042A Berkmar Dr, Charlottesville, VA 22901; 434-296-7000; fax: 434-975-0080.
Email: mcbaruch@comcast.net
http://dx.doi.org/10.1682/JRRD.2015.06.0120
INTRODUCTION

Mild traumatic brain injury (mTBI), or concussion, is an important source of morbidity for athletes, civilians, and the military, with approximately 3.5 million mTBIs occurring annually in the United States [1]. Attention to acute concussions in contact sports is on the rise, in addition to the growing concern over the possible long-term, cumulative, degenerative effects of multiple concussive and subconcussive injuries in both athletes and the military. mTBIs have also been termed the "signature injury" of the conflicts in the Persian Gulf, with more than 200,000 reported between 2000 and 2013 [2]. The overall prevalence of concussions is likely far higher than reported figures since many concussions go unrecognized during and after sporting events, military deployments, falls in the elderly, recreational accidents, and motor vehicle collisions. This oversight occurs because of a number of factors, including the transient nature of most acute symptoms, the greater attention paid to more visible concomitant injuries (e.g., lacerations, fractures), and the limited awareness of the common cognitive and behavioral deficits manifest in the acute period. Critical issues in this mTBI population are the need to make timely, accurate diagnoses and to have meaningful assessments to understand and monitor phases of recovery after injury for better-informed clearance for a return-to-play, school, duty, and other activities, and for metrics to determine the efficacy of individualized management protocols for concussions. The major obstacle to the acute and long-term detection, monitoring of recovery, clearance to activity, and assessment of efficacy is the lack of a standardized, objective, multimodal system that can accurately and consistently assess physiologic functioning after concussion.

While a number of neurophysiological assessment approaches to understand mTBI have been able to distinguish between groups of concussed and nonconcussed populations [3–7], no single assessment modality has demonstrated efficacy in diagnosing mTBI on an individual basis. An evidence-based, single-metric decision point for an individual would be ideal for making a dependable return-to-activity determination. Two key factors that affect the accuracy of neurophysiological assessments and thus make this goal elusive are (1) potential large baseline variations in neurophysiological function and (2) the presence of transient interferences, such as learning effects, fatigue, anxiety, and unrelated states of mental alertness or illnesses. One approach to overcoming these obstacles is the use of a tool that includes an integration of several testing modalities. A multimodal approach is more likely to uncover deficits related to concussion that a single modality would miss. This conceptual approach is beginning to be explored and utilized [8–9].

The study was designed to explore the hypothesis that the sensitivity and specificity of mTBI detection can be enhanced by using a multimodal, neurophysiological assessment approach. Based on preliminary data, the modalities chosen were computerized formats of eye-tracking, balance and stability, and neurocognitive assessments. While all three modalities have been individually validated as being able to distinguish between concussed and nonconcussed populations [10–13], the goal of this study was to examine the enhancement in individual test discriminatory capability through the combination of all three assessment modality scores into a single, integrated multimodal system score. A secondary goal of the research was to assess the feasibility of developing a field-deployable system, with the compressed assessment, score compilation time, and portability that this application requires.

METHODS

After obtaining the appropriate institutional review board approvals, individuals who had either no history of concussion (n = 28) or a history of one or more concussions (n = 28) were recruited from a freestanding traumatic brain injury (TBI) specialty clinic affiliated with an academic medical center (Virginia Neurocare; Charlottesville, Virginia), primary care and TBI specialty clinics at a tertiary care Department of Veterans Affairs Medical Center (VAMC) (Hunter Holmes McGuire VAMC; Richmond, Virginia), and a military treatment facility (Kenner Army Health Clinic; Fort Lee, Virginia). Diagnosis of mTBI was confirmed through clinical interviews and a review of medical records. Information on the subjects' demographics, TBI history, and medical history were recorded. All subjects underwent comprehensive eye tracking and balance testing using the Computerized Brain Injury Assessment System (CBIAS). The CBIAS is an integrated neurophysiological tool that incorporates state-of-the-art technology to rapidly and consistently assess eye tracking and posturography. The CBIAS consists of a head-mounted unit, a sensorized platform that the subject stands on, and a laptop computer to collect the sensor system data.

A subset of the subjects was also assessed with the Headminder Cognitive Stability Index (CSI). CSI is an Internet-based, neurocognitive measure that assesses visual memory, response speed, attention/executive functions, and visual-motor speed [14–15].

Equipment Description
Computerized Brain Injury Assessment System

The CBIAS is an integrated sensory assessment system designed to provide a multimodal evaluation of neurologic condition. For this study, sensory inputs included eye tracking, posturography, and neurocognitive evaluation. The test battery comprised a total of 27 assessment tests, including three different eye movement tests, six different stance assessment tests, and the cognitive assessment test battery. At the core of the CBIAS system is the computer-based software, which coordinates the execution of the assessment test sequence, the test-dependent target/stimulus presentation, and the time-synched collection of the six data inputs, two from the eye tracker and four from the platform. The different components of the system have different timing accuracies. While the Measurement Computing analog-to-digital converter (Norton, Massachusetts), which digitizes the voltage signals coming from the four platform sensors, has a specified accuracy on the order of 10 µs, the computer-initiated eye target movements have a timing accuracy with a standard deviation (SD) of 5.8 ms, for the specific computer used. This uncertainty was measured by comparing the computer time-stamp call to the target movement function and the photodiode-measured transition corresponding to the actual target movement on the computer's screen. Both the eye-tracking and the stance assessment test sequences are reconfigurable because they are submitted to the software as an editable ASCII file. Each assessment test is characterized by a set of parameter entries. For example, a horizontal sinusoidal smooth pursuit eye movement (SPEM) test is characterized by its amplitude and frequency, while a random movement test is characterized by an ASCII file of time and position coordinates. Stance assessment tests are characterized by their time duration and the type of concurrent eye target movement pattern, if applicable. At the end of a session, the program saves the raw sensor data, the time-stamped target movements where applicable, and a log file of the executed test sequence and timing markers for the entire eye movement and stance assessment sequence. The data are stored in RAM during the session to prevent program execution delays due to disk writing tasks.

Eye Tracking

The head-mounted unit, which is shown in Figure 1, consists of a heads-up display, which is a modified i3PC (I-O Display Systems; Sacramento, California), and a custom-designed eye movement tracking module. The display has a resolution of 800 × 600 pixels, establishing an image size of 70 in. at 13 ft, with an update rate of 100 Hz. The eye relief is at a distance of 1 in. from the eye, leaving enough access space for the eye-tracking hardware to be mounted below the eye. The eye tracker uses the principle of "limbus tracking" [16], which is based on tracking the difference in reflection of infrared (IR) light by the sides of the limbus margin between sclera and iris. For each eye, one IR LED (light-emitting diode) and two photodiodes, operating in the 850 to 950 nm wavelength range, are used in a linear array with a length of about 3 cm, with the photodiodes located at the ends of the array and the LED centered. The optical array is located about 2 cm away from the eyeball below the line of sight.


Figure 1. View of the inverted head-mounted prototype unit of the Computerized Brain Injury Assessment System. In the foreground, the optical elements of the eye trackers are visible, consisting, for each eye, of one center infrared light-emitting diode (LED) (darker element) and two flanking photodiodes. The viewing screens of the i3PC display system for each eye are visible behind the eye tracking elements. Part of the forehead support is visible in the lower left foreground. Electronics to drive the LEDs and amplify/demodulate the photodiodes’ signal are contained in the box in the upper right background.

Figure 1.

View of the inverted head-mounted prototype unit of the Computerized Brain Injury Assessment System. In the foreground, the optical elements of the eye trackers are visible, consisting, for each eye, of one center infrared light-emitting diode (LED) (darker element) and two flanking photodiodes. The viewing screens of the i3PC display system for each eye are visible behind the eye tracking elements. Part of the forehead support is visible in the lower left foreground. Electronics to drive the LEDs and amplify/demodulate the photodiodes' signal are contained in the box in the upper right background.

Click Image to Enlarge. View as PowerPoint Slide

Care was taken in the design of the driving circuitry of the LED that the eye damage threshold, about 10 mW/cm2, is not approached within a factor of 10. The IR LED is driven using an amplitude-modulated current at 5.0 kHz, making it possible to use lock-in or phase-sensitive detection techniques to improve signal to noise ratio (S/N), particularly relative to 1/f noise. Since the pulsation of the IR LED lowers its duty cycle, it is possible to increase the driving current relative to constant operation, increasing S/N, while lowering overall current consumption, an important consideration for battery-powered field assessment scenarios.

In order to extract the eye movement signal, the differential voltage from a given photodiode pair is obtained, which contains both the eye movement as well as the sinusoidal signal of the LED modulator. Phase-sensitive detection, which amounts to demodulating the signal by beating it with the LED modulator frequency, provides a direct current-like voltage that is the eye movement signal, as well as higher-order harmonics of the driver signal, which are eliminated by low-pass filtering at a break frequency of 100 Hz with a roll-off of about –10 dB/decade. With this arrangement, acquisition rates of 500 Hz can be readily supported, which is sufficiently fast to track the fastest saccades components. The angular resolution capability of the system is approximately 0.03°. The limbus tracking approach has been used extensively in numerous eye-tracking studies, for example in commercial implementations such as the IRIS system by Skalar (Breda, the Netherlands), and made the recording of eye movements with a temporal resolution of milliseconds possible before the advent of high-speed video recording systems. In eye-tracking applications requiring mobility and compact size, the approach is still attractive because of its small footprint, use of readily available and low-cost optical components, and simplified data acquisition/processing requirements [16].

The three eye movement tests were horizontal smooth pursuit, circular smooth pursuit, and random horizontal smooth. Each eye test was run twice with different excursion amplitudes and movement timing/frequency. To ensure proper alignment of the eye tracker, a horizontal sinusoidal scan (amplitude: 15°, frequency: 0.2 Hz, time: 10 s) is performed as a calibration at the beginning of a test session. The CBIAS graphical user interface is set up so that the tester can visually compare the results of the scan with a benchmark. In the event of asymmetries in the scan, the eye trackers can be individually adjusted with set screws and/or the head mount can be moved as a whole using the adjustable support straps. This calibration scan was followed by five SPEM tests of varying difficulty, two circular tests (0.8 Hz, 12° and 18° amplitude), one horizontal SPEM (1.2 Hz, 18° amplitude), and two random horizontal SPEM tests, generated using the standard sum of sines method. The two random tests are of different difficulty levels with principal spectral component, respectively, at 0.35 Hz (12° amplitude) and 0.80 Hz (13° amplitude).

Posturography

The structural components of the platform unit are a 2 ft × 2 ft × 0.5 in. carbon fiber platform, with Divinycell H100 foam core (DragonPlate; Elbridge, New York) and four corner supports, which are double-convoluted air springs (model 2B6–530, Goodyear; Akron, Ohio) that allow the platform to be operated in a stable and an air-cushioned support mode. The platform is sensorized with four piezoelectric sensors (3/4 in. PZT disks sandwiched between silvered electrodes) that are centered on each side. Since the bending axis of each piezoelectric sensor is external to it, the effect of the bending as the platform surface is displaced is to subject the piezoelectric sensor to internal stress or strain. Amplification of the signal established by the piezoelectric sensors is provided by a so-called "transimpedance" amplifier that inputs current and outputs voltage. The transimpedance configuration clamps the voltage on the piezo-element to zero and provides an output proportional to the current produced by the movement of charge in the element. The output is thus proportional to the rate of strain in the piezo-element or to the time derivative of its displacement. The benefits of this circuit include fast recovery, about 0.01 s, and high sensitivity to small signals. The output signals of the four sensors are low-pass filtered (3 dB point: 500 Hz with a roll-off of 6 dB/decade) and each is digitized at 1,024 Hz using a Measurement Computing 16-bit analog/digital converter.

The six stance tests involved three different assessments for two different platform modes, stable and air-cushioned. The first three stance assessments, performed on a stable platform, were standing with eyes open (S1), standing with eyes closed (S2), and standing while observing a randomly moving target on the heads-up display (S3). The same stance assessments, performed on an unstable platform, constituted S4, S5, and S6. Each stance assessment test was repeated three times in random order. We present receiver-operator-characteristics (ROC) analyses and area-under-the-curve (AUC) calculations to demonstrate the detection specificity/sensitivity of individual assessment metrics as well as that of sets of metrics. The ROC analyses, which involve plotting the true positive rate against the false positive rate, are an effective approach to assessing the classification capability of diagnostic tests, i.e., mTBI yes/no, where the generally accepted AUC for an effective test is larger than 0.7, with perfect discrimination corresponding to AUC = 1.0, while a test with no discriminatory capability will have an AUC in the vicinity of 0.5. All calculations were performed using the statistical module of the Origin 8.5 analysis software (OriginLab Corporation; Northampton, Massachusetts).

Neurocognitive Assessment–Headminder Cognitive Stability Index

The CSI is a standardized and validated web-enabled computerized test protocol that utilizes, among other procedures, a classic visual memory paradigm in which subjects, after one exposure, must remember the nature and position of objects that are subsequently hidden behind cabinet doors. The CSI also employs a traditional continuous performance test format in computerized form to measure attention, response speed, and multitasking (executive function) [12]. The CSI provides rapid and repeatable neurocognitive screenings with instantly accessible results. Designed for longitudinal assessment, the CSI has alternate forms to minimize practice effects and uses the individual's baseline assessment to detect decline, improvement, or stability of cognitive functioning across time. Subject performance can also be compared with normative data by age, education, sex, and ethnicity. Individuals can be tested in a variety of settings, including military theaters, even where there is no access to neuropsychological services. It has been used successfully in theater (e.g., Balad Air Force Base Hospital) and has been validated against the Repeatable Battery for the Assessment of Neuropsychological Status, which is also actively used in military treatment settings. The CSI, which also compared well with ImPACT (Immediate Postconcussion Assessment and Cognitive Test) and ANAM (Automated Neuropsychological Assessment Metrics) in sports assessments [17], is sensitive to the subtle neurocognitive deficits associated with concussion and was used in this study to complement the eye tracking and balance assessment of neurophysiological function in the concussed and nonconcussed subject populations.

Statistical Analyses
Eye Tracking

The SPEM analysis approach focused on the generalized quantification of the overlap of target motion and eye movement. In addition, traditional measures such as the number and amplitude of corrective saccades were also used to parameterize eye tracking accuracy. While Contreras et al. developed analysis formalisms for overlap quantification that are intimately related to the type of motion used, such as circular motion [13], the goal in this study was to develop a formalism that would lend itself to the analysis of any smooth pursuit movement. SDs on five different time scales, specifically 0.5, 1.0, 2.0, 4.0, and 8.0 s, were chosen as the parameterization measures of overlap of target and eye movement track. This approach was chosen because it (1) provides an approach to temporally resolve tracking capability, (2) is algorithmically readily implemented, and (3) lends itself to the calculation of a single score for a given eye test. Each of these properties is critical for the development of a time-sensitive concussion assessment methodology in mobile settings. While small SDs would be expected at the shortest time scales, small SDs at longer time scales will indicate prolonged overlap or the ability of the subject's eye-tracking to follow the target in a sustained fashion.

The first step in the data preprocessing involves obtaining the optimum overlap between the target track and the eye-track data streams by time-shifting them relative to each other. The optimum time shift is obtained through the cross-correlation of both signals. The second step involves obtaining the difference of the two optimally time-aligned signals. From this difference curve, data sequences corresponding to a certain time window, for example 0.5 s, are extracted, and the SD is calculated for the data points in this window. Perfect tracking would result in an all-zeros difference curve, and the SD would therefore be zero. The window is then moved along to the next data point, and the SD is again obtained. The process yields an array of SDs for the entire length of the difference curve for the specified time window. The process is repeated for five different windows: 0.5, 1.0, 2.0, 4.0, and 8.0 s; the hypothesis being that longer windows would better differentiate neurocognitive deficits because they require longer-term tracking accuracy. The resulting arrays of SDs for the stated time windows are compiled into distributions, a process that effectively categorizes the angular tracking errors according to count. These distributions are further distilled by determining their quantiles. To maintain resolution, nine quantiles were chosen, providing a resolution in 10 percent increments and a possible parameter set of 40 quantiles, 9 for each of the 5 time windows. In order to identify the most sensitive subset with regard to discriminating differences between concussed and nonconcussed subjects within the window/quantile plane, an ROC analysis was performed using each of the 40 quantile parameters as the sole assessment parameter. The circular SPEM test at 1.2 Hz and 18° amplitude, considered the most challenging, was used for this comparative assessment. From the ROC analysis, the AUC was obtained. The process was repeated for each of the 40 quantile parameters. The results, in the form of a three-dimensional waterfall graph that displays the AUC for each quantile/window pair, are presented in Figure 2. The graph reveals a "ridge" of high AUC values across time windows corresponding to the seventh through eighth quantiles, suggesting that this subset of quantiles has the highest discriminating capability. For this reason, the seventh through eighth quantiles of the 1.0, 2.0, and 4.0 s windows were averaged. The average of this subset of parameters was used in all of the subsequent generalized angular tracking error calculations for SPEM tests.


Figure 2. Three-dimensional waterfall area-under-the-curve (AUC) graph of the 40 quantile/window pairs for the circular smooth pursuit eye movement test at 1.2 Hz and 18° amplitude. Discrimination sensitivity increases with window size and quantile number until a ridge is reached (7–8th quantile) beyond which sensitivity falls off.

Figure 2.

Three-dimensional waterfall area-under-the-curve (AUC) graph of the 40 quantile/window pairs for the circular smooth pursuit eye movement test at 1.2 Hz and 18° amplitude. Discrimination sensitivity increases with window size and quantile number until a ridge is reached (7-8th quantile) beyond which sensitivity falls off.

Click Image to Enlarge. View as PowerPoint Slide

The number and size of corrective saccades were used as another eye tracking score. As part of an automated analysis, saccades were detected if the amplitude of the movement was greater than ±0.6°, the duration of the saccade fell within a predetermined minimum and maximum time limit, and the calculated velocity and acceleration values (based on a two-point central difference method) were greater than ±20 °/s and ±400 °/s2, respectively. Amplitudes in excess of 6° were discarded as blinks and a data range extending 30 ms before and after the event was excluded from subsequent analysis. For any saccadic eye movement, the time, location, and amplitude of the saccade were determined. Saccades were categorized by size in single degree increments and evaluated regarding their discriminatory power using the ROC/AUC analysis.

Posturography

Stance scores for the individual balance tests, identified as S1 through S6 as previously described, were obtained by digitally integrating the signals of the sensors measuring posterior and anterior sway and then obtaining their difference. This difference signal was again analyzed using the generalized overlap/SD at different time scales, as previously described. The time scales in this case were 0.125, 0.500, 1.000, and 4.000 s. Nine quantiles in this case resulted in a characterizing set of 36 parameters for each of the 6 balance tests. As in the SPEM case, a ROC analysis was performed to identify the most sensitive subset of the 36-parameter set with regard to discriminating differences between concussed and nonconcussed subjects. In this case, the score difference between S5 and S2, involving balance tests with eyes closed on a stable and on an unstable platform, was used. As before, within the window/quantile plane, a ROC analysis was performed using each of the 36 quantile parameters as the sole assessment parameter. The results are presented in Figure 3. As in the optical case, a "ridge" of highest AUC values is observed across all time windows for the seventh and eighth quantiles. The average over this subset of parameters was used in the subsequent analysis of all the balance data.


Figure 3. Three-dimensional waterfall area-under-the-curve (AUC) graph of the 36 quantile/window pairs for the score difference between S5 (eyes closed, unstable platform) and S2 (eyes closed, stable platform). Discrimination sensitivity increases mostly with quantile number until a ridge is reached (7th–8th quantile), beyond which sensitivity falls off.

Figure 3.

Three-dimensional waterfall area-under-the-curve (AUC) graph of the 36 quantile/window pairs for the score difference between S5 (eyes closed, unstable platform) and S2 (eyes closed, stable platform). Discrimination sensitivity increases mostly with quantile number until a ridge is reached (7th-8th quantile), beyond which sensitivity falls off.

Click Image to Enlarge. View as PowerPoint Slide

To minimize the effect of individual balance capability as well as to isolate somatosensory, visual, and vestibular balance inputs, the following ratios of test scores were calculated: S4/S1, S5/S1, and S6/S1. ROC analyses were performed on each ratio to determine the discriminatory capability of each assessment metric.

NeuroCogntive Assessment

Each of the Headminder CSI constituent tests was analyzed analogously in order to determine the most promising candidates for inclusion in the final composite score.

RESULTS

A total of 56 subjects were tested, including 28 subjects (27 male, mean age = 32.68 yr, SD = 11.62) with a diagnosis of mTBI and 28 subjects with no history of mTBI (25 male, mean age = 30.96 yr, SD = 16.16). The Headminder CSI assessment was performed on a subset of 30 subjects (18 with a history of mTBI). Of the 28 subjects with mTBI, 12 were tested within 2.5 yr of injury (mean testing time after injury = 1.08 yr), and 16 subjects were tested within 5 yr of injury (mean testing time after injury = 4.2 yr). The etiology of concussion included blasts (25.8%), bullet-related (6.4%), blunt force (3.2%), motor vehicle accidents (3.2%), falls (5.0%) and sports-related (35.5%).

Individual Assessment Tests

Table 1 summarizes the results for the individual eye tracking and posturography assessments, listing the respective means for both populations, the AUCs, the statistical significances, and the T-scores for the horizontal, circular, and random motion SPEM tests as well as the three stance assessment metrics. Table 2 lists the corresponding information for the different Headminder CSI tests.


Table 1. 
p-Value
T-Score
0.38
0.64
0.75
0.07
–2.30
8.30
10.50
0.66
0.08
–1.84
5.30
6.90
0.58
0.13
–1.03
3.70
6.30
0.63
0.08
–2.09
1.18
2.78
0.78
0.03
–2.78
17.50
22.10
0.75
0.03
–1.48
15.10
25.10
0.64
0.04
–1.61
6.80
12.10
0.64
0.20
–2.09
1.32
2.68
0.88
0.06
-4.71
30.30
40.30
0.72
0.08
–0.62
25.20
34.50
0.79
0.04
–2.83
7.90
16.40
0.79
0.07
–3.84
0.21
0.95
0.65
0.045
–0.65
10.60
11.10
0.63
0.08
–0.34
6.00
7.20
0.62
0.12
–1.26
4.30
6.55
0.74
0.08
–2.39
0.41
1.07
0.79
0.06
–3.56
10.10
12.80
0.64
0.08
–1.70
12.60
15.80
0.62
0.009
–2.08
3.88
10.60
0.76
0.10
–2.07
0.0023
0.0129
0.74
0.005
–3.03
0.0047
0.0263
0.78
0.02
–2.06
0.0025
0.0300
0.75
0.07
–1.69

Table 2. 
p-Value
T-Score
101.1
91.6
0.65
0.14
1.52
115.5
83.7
0.86
0.004
3.27
110.2
86.8
0.84
0.007
3.12
89.6
78.8
0.65
0.02
1.25
103.2
95.6
0.63
0.15
1.46
103.0
80.0
0.79
0.02
2.56
104.6
89.9
0.64
0.02
1.35
113.8
91.8
0.82
0.01
3.19
115.1
91.1
0.76
0.04
2.54
117.2
103.6
0.78
0.03
2.27
104.4
93.5
0.74
0.05
1.97
101.5
81.4
0.78
0.03
2.31
86.9
83.9
0.55
0.14
0.61
92.0
86.9
0.57
0.20
0.84
Multimodal Assessment Test

The ROC analysis approach can be readily extended to examine the cumulative effect on sensitivity/specificity of including the results of several discriminatory tests. Since the previous analyses established statistically significant differences in the discriminatory capability of the different assessments, those with the highest AUCs were selected for inclusion in the multimodal analysis. In Figures 4 and 5 we present the results of combining the scores of test combinations for the entire cohort of subjects. Figure 4 displays the ROC curve obtained using the angular tracking error of the fast circular motion eye test as the sole assessment metric. In Figure 5, the comparative results are presented for the case when, in addition to the angular tracking error, the saccade count between 1° and 2° and the S5/S1 balance score are included as metrics.


Figure 4. Receiver-operator-characteristics curve for fast circular motion eye test only: 56 subjects. AUC = area-under-the-curve.

Figure 4.

Receiver-operator-characteristics curve for fast circular motion eye test only: 56 subjects. AUC = area-under-the-curve.

Click Image to Enlarge. View as PowerPoint Slide


Figure 5. Receiver-operator-characteristics curve for angular tracking error, the saccade count between 1° and 2°, and the S5/S1 balance score as metrics. 56 subjects. AUC = area-under-the-curve.

Figure 5.

Receiver-operator-characteristics curve for angular tracking error, the saccade count between 1° and 2°, and the S5/S1 balance score as metrics. 56 subjects. AUC = area-under-the-curve.

Click Image to Enlarge. View as PowerPoint Slide

In order to generate comparative results that included the most effective CSI scores, the analysis was repeated for the subset of 30 subjects who were assessed with all three modalities. In Figures 6–8, we present the comparative results. Figures 6 and 7 present the ROC curves for the angular tracking error of the fast circular motion eye test as the assessment metric (AUC = 0.84) and the angular tracking error as well as the saccade count between 1° and 2° and the S5/S1 balance score (AUC = 0.92), respectively. Figure 8 presents the ROC curve for the calculation that incorporates the effect, along with the previous metrics, of two of the most effective CSI constituent tests (response speed and memory/incidental learning 1). The AUC now equals 0.97, indicating a very high ability to discriminate concussed from nonconcussed subjects. While inclusion of additional constituent tests would have enhanced discriminatory capability further, the motivation with regard to a fieldable system is to minimize the time required for each modality, and consequently also the neurocognitive assessment aspect, as much as possible.


Figure 6. Receiver-operator-characteristics curve obtained using the angular tracking error of the fast circular motion eye test as the assessment metric only for the subset of 30 subjects assessed with Cognitive Stability Index. AUC = area-under-the-curve.

Figure 6.

Receiver-operator-characteristics curve obtained using the angular tracking error of the fast circular motion eye test as the assessment metric only for the subset of 30 subjects assessed with Cognitive Stability Index. AUC = area-under-the-curve.

Click Image to Enlarge. View as PowerPoint Slide


Figure 7. Receiver-operator-characteristics curve for angular tracking error, the saccade count between 1° and 2°, and the S5/S1 balance score as metrics for 30 subject subset. AUC = area-under-the-curve.

Figure 7.

Receiver-operator-characteristics curve for angular tracking error, the saccade count between 1° and 2°, and the S5/S1 balance score as metrics for 30 subject subset. AUC = area-under-the-curve.

Click Image to Enlarge. View as PowerPoint Slide


Figure 8. Receiver-operator-characteristics curve for angular tracking error, saccade count between 1° and 2°, S5/S1 balance score and two Cognitive Stability Index constituent tests (response speed and memory/learning). AUC = area-under-the-curve.

Figure 8.

Receiver-operator-characteristics curve for angular tracking error, saccade count between 1° and 2°, S5/S1 balance score and two Cognitive Stability Index constituent tests (response speed and memory/learning). AUC = area-under-the-curve.

Click Image to Enlarge. View as PowerPoint Slide

DISCUSSION

This investigation represents the first prospective assessment of the utility of a multimodal mTBI diagnostic tool, the CBIAS. The results support a number of significant findings, providing initial quantitative validation for (1) a new portable assessment system for the neurophysiological identification of mTBI, (2) a new and generalized SPEM analysis method that provides eye tracking deficit information, and (3) a new and generalized posturography analysis method that provides resolution of balance deficits. These findings demonstrate the enhanced potential of integrating several objective, physiologic measures of brain functioning, instead of relying on a single measure, to more readily identify individuals who have demonstrate persistent evidence of mTBI. The ability of the CBIAS to differentiate between individuals with and without a history of mTBI, combined with its portability makes it an assessment tool with significant clinical utility.

In this study, the effectiveness of the CBIAS-based eye-tracking and balance assessments in resolving differences between the concussed and nonconcussed groups varied significantly. As noted by Contreras et al. [13] and Cifu et al. [10], two-dimensional SPEM tests demonstrated the highest effectiveness (AUC = 0.88). Furthermore, more challenging horizontal tests, such as the faster random sinusoidal assessment (AUC = 0.79), resolved differences better than less challenging assessments, specifically the simple sinusoidal as well as the slow random horizontal tests. While the effectiveness of the angular tracking error parameter was good to very good for all SPEM tests except the slow random sinusoidal test (AUC = 0.65 vs 0.75–0.88 for the other SPEM tests), the effectiveness of tracking corrective saccades was largely limited to the circular motion SPEM. The effectiveness was further concentrated toward the larger-angle saccades, with only the 1–2° and 2–3° corrective saccades demonstrating efficacy (AUC = 0.79 for both vs 0.72 for 0.4–1.0 ). One reason for this comparative underperformance may be the inherent granularity, or limited dynamic range, of the method. Put differently, the number of larger-scale corrective saccades in a 20 to 25 s scan will usually be in the single-digit to perhaps two dozen range, except in rare cases. This compares in the angular tracking error case to a calculation that involves 10,000 data points for a 20 s scan collected at 500 Hz. The increased granularity likely increases the ability to detect differences in tracking accuracy.

The results of the SPEM-related quantile/window analysis with regard to the concussed/nonconcussed discrimination sensitivity of individual quantile/window pairs appear to support the hypothesis that longer time windows have better discriminatory capability, presumably because of the more sustained effort required to maintain tracking, which evidently resolves concussion-related deficits better. The dynamic range of the waterfall plot shown in Figure 2 is significant, ranging from good to very good discrimination (0.78–0.91). Of particular interest, however, is the fact that the sensitivity falls off slightly for the longest window (8.0 s), likely because the largest window is more likely to include errors such as from residual blinks, etc., that are unrelated to concussion but instead are due to fatigue, inattention, etc. The same mechanism is likely responsible for the increasing sensitivity response that is observed with increasing quantiles numbers until a ridge is reached in the range of the seventh and eighth quantile, after which there is a distinct fall-off. Exclusion of the highest quantile appears to exclude the larger error signals that evidently are less concussion-related.

The effectiveness scores of the balance tests were fairly tightly grouped in an AUC range, indicating unexceptional discriminatory capability (0.74–0.78). Examination of the waterfall graph in Figure 3 with regard to the different discrimination sensitivities of the individual quantile/window pairs reveals that (1) the dynamic range of the AUC range is comparable to that in the SPEM case, ranging from poor discrimination (AUC = 0.68) to good discrimination (AUC = 0.79), and (2) the discrimination capability displays a distinct maximum "ridge" in the 70 to 80 percentile range, with a peak along that ridge for the 0.5 and 1.0 s windows. However, the distinct time window dependence that is observed in the lower quantiles in the SPEM test case is not evident here, i.e., sensitivity increases primarily with quantile number until the ridge is reached. The ridge feature suggests, however, that exclusion of the largest balance errors is prudent because these are likely due to balance shifts unrelated to cognitive deficits and common to both populations. The mild peak feature for the central window lengths supports this notion since it is reasonable to hypothesize that (1) low balance errors for short time windows are less likely to have discriminatory power unless severe balance deficits are present and (2) long time windows are more likely to again include balance shifts unrelated to effects due to concussion.

Another result of this study is that the majority of eye-tracking and balance assessment tests did not show exceptional discriminatory ability. This is likely due to two factors: (1) the tests in question were not sufficiently challenging and (2) the significant time since injury for the majority of the tested cohort.

Regarding the challenge of the tests, comparison of the relative AUC results establishes the circular SPEM tests as standouts, with the one-dimensional SPEM tests having very poor discriminatory ability.

To increase the overall effectiveness of the battery, the challenge level of the tests can be increased (higher frequency, larger amplitude) or other tests, such as memory guided saccades, antisaccades, etc., can be substituted. There are similar concerns regarding the S6/S1 tests (observing a randomly moving object [unstable platform]/eyes open [stable platform]) and the S5/S1 tests (eyes closed [unstable platform]/eyes open [stable platform]). While both test vestibular function, the S6/S1 combination should presumably be more challenging because of the added tracking task. Yet it displayed less discriminatory power than the S5/S1 combination, which may have been because the actual field of view provided the subject spatiovisual clues that negated the desired testing condition. Another reason may also have been that the presented stimulus motion was not sufficiently captivating so as to engage the subject throughout. The other factor that likely affected discriminatory capability was time since injury. The overall mean time since injury was 4.2 yr (SD = 4.9 yr). This compares to, for example, the study cohorts of Cifu et al. [10], where the mean time since the most recent mTBI was 8.5 mo (SD = 6.58 mo) and Contreras et al. [8] (2.2 yr ± 1.8 yr). One of the forward-looking results is the further enhancement of the sensitivity of the CBIAS through the modification and incorporation of other eye tests as well as the incorporation of neurocognitive tests into the CBIAS battery. Options include a computerized version of the N-back test [18] and the Paced Auditory Serial Addition Task [19], which are relatively brief and sensitive tests of neurocognitive functions involving new and rapid problem-solving requiring learning and flexibility of thinking.

CONCLUSIONS

Preliminary validation results were obtained using a multimodal assessment battery incorporating eye-tracking, balance, and neurocognitive tests on a new hardware platform, the CBIAS. Using ROC analyses, the following conclusions were reached: (1) a subset of the most discriminating neurophysiological assessment tests, involving SPEM tracking errors, corrective saccade counts, a balance score ratio sensitive to vestibular balance performance, and two neurocognitive tests (response speed and memory/incidental learning), was identified; (2) the enhancement in discriminatory capability of detecting concussion-related deficits through the combination of the identified subset of assessments was demonstrated; and (3) the effectiveness of a robust and readily implemented global scoring approach was demonstrated for both eye track and balance assessment tests. These results are significant in introducing a comprehensive solution for concussion assessment that incorporates an economical, compact, and mobile hardware system and an assessment battery that is multimodal and time efficient and whose efficacy has been demonstrated on a preliminary basis. As such, it presents a significant step toward the goal of a system capable of making a dependable return-to-play/duty determination based on concussion likelihood.

ACKNOWLEDGMENTS
Author Contributions:
Study concept and design: M. Baruch, J. T. Barth.
Acquisition of data: M. Baruch, M. Leibman.
Analysis and interpretation of data: M. Baruch, J. T. Barth, D. Cifu.
Drafting of manuscript: M. Baruch, D. Cifu, M. Leibman.
Critical revision of manuscript for important intellectual content:
J. T. Barth, D. Cifu, M. Leibman.
Statistical analysis: M. Baruch.
Obtained funding: M. Baruch, J. T. Barth.
Administrative, technical, or material support: J. T. Barth, D. Cifu,
M. Leibman.
Study supervision: J. T. Barth, D. Cifu, M. Leibman.
Financial Disclosures: The authors have declared that no competing interests exist.
Funding/Support: This material was based on work supported by the U.S. Army Small Business Innovation Research program (contract W81XWH-09-C-0150).
Additional Contributions: It is a pleasure to acknowledge the contributions of Karen Vajda, Hillary Samples, and Ethan Baruch with regard to project administration and data collection.
Institutional Review: This work was approved by the institutional review boards of the University of Virginia, Hunter Holmes McGuire VAMC, and Walter Reed National Military Medical Center (for Kenner Army Health Clinic) and the New England Independent Review Board.
Participant Follow-Up: The authors do not plan to inform participants of the publication of this study.
Disclaimer: The views expressed in this article are those of the authors and do not reflect the official policy or endorsement of the Department of the Army, Department of Defense, or the U.S. Government.
REFERENCES
1.
Langlois JA, Rutland-Brown W, Wald MM. The epidemiology and impact of traumatic brain injury: A brief overview. J Head Trauma Rehabil. 2006;21(5):375–78.
[PMID:16983222]
http://dx.doi.org/10.1097/00001199-200609000-00001
2.
DoD worldwide numbers for TBI [Internet]. Washington (DC): Defense and Veterans Brain Injury Center; 2016 [updated 2016 Nov 23]. Available from:
http://dvbic.dcoe.mil/dod-worldwide-numbers-tbi
3.
Heitger MH, Anderson TJ, Jones RD, Dalrymple-Alford JC, Frampton CM, Ardagh MW. Eye movement and visuomotor arm movement deficits following mild closed head injury. Brain. 2004;127(Pt 3):575–90. [PMID:14736751]
http://dx.doi.org/10.1093/brain/awh066
4.
5.
Nuwer MR, Hovda DA, Schrader LM, Vespa PM. Routine and quantitative EEG in mild traumatic brain injury. Clin Neurophysiol. 2005;116(9):2001–25. [PMID:16029958]
http://dx.doi.org/10.1016/j.clinph.2005.05.008
6.
Bulut M, Koksal O, Dogan S, Bolca N, Ozguc H, Korfali E, Ilcol YO, Parklak M. Tau protein as a serum marker of brain damage in mild traumatic brain injury: Preliminary results. Adv Ther. 2006;23(1):12–22. [PMID:16644603]
http://dx.doi.org/10.1007/BF02850342
7.
Erlanger DM, Kaushik T, Cantu R, Barth J, Broshek D, Freeman J, Webbe F. Symptom-based assessment of concussion severity. J Neurosurg. 2003;98:477–84.
[PMID:12650417]
https://doi.org/10.3171/jns.2003.98.3.0477
8.
Contreras R, Ghajar J, Bahar S, Suh M. Effect of cognitive load on eye-target synchronization during smooth pursuit eye movement. Brain Res. 2011;1398:55–63.
[PMID:21620377]
http://dx.doi.org/10.1016/j.brainres.2011.05.004
9.
Markgraff P. ImPACT, King-Devick Test partner to support multifaceted concussion management [Internet]. Football Coach Daily. 2013 Oct 10. Available from: http://footballcoachdaily.com/2013/10/10/impact-king-devick-test-partner-to-support-multifaceted-concussion-management/#sthash.NkONgpJh.dpuf
10.
Cifu DX, Wares JR, Hoke KW, Wetzel PA, Gitchel G, Carne W. Differential eye movements in mild traumatic brain injury versus normal controls. J Head Trauma Rehabil. 2015;30(1):21–28. [PMID:24695263]
http://dx.doi.org/10.1097/HTR.0000000000000036
11.
Guskiewicz KM, Ross SE, Marshall SW. Postural stability and neuropsychological Deficits after concussion in college athletes. J Athetic Train. 2001;36(3):263–73.
[PMID:12937495]
12.
Erlanger D, Feldman D, Kutner K, Kaushik T, Festa J, Barth J, Freeman J, Broshek D. Development and validation of a web-based neuropsychological test protocol for sports-related return-to-play decision-making. Arch Clin Neuropsychol. 2003;18(3):293–316. [PMID:14591461]
ttps://doi.org/10.1093/arclin/18.3.293
13.
Contreras R, Kolster R, Voss HU, Ghajar J, Suh M, Bahar S. Eye-target synchronization in mild traumatic brain-injured patients. J Biol Phys. 2008;34(3–4):381–92.
[PMID:19669482]
http://dx.doi.org/10.1007/s10867-008-9092-1
14.
Grant G, Islet W, Erlanger D, Kaushik T. Feasibility of the HeadMinder CSI for screening of blast-injury related cognitive impairment in a combat field hospital. Arch Clin Neuropsychol. 2007;22(7):898–99.
15.
Erlanger DM, Kaushik T, Broshek D, Freeman J, Feldman D, Festa J. Development and validation of a web-based screening tool for monitoring cognitive status. J Head Trauma Rehabil. 2002;17(5):458–76. [PMID:12802255]
http://dx.doi.org/10.1097/00001199-200210000-00007
16.
Reulen JP, Marcus JT, Koops D, de Vries FR, Tiesinga G, Boshuizen K, Bos JE. Precise recording of eye movement: The IRIS technique. Part 1. Med Biol Eng Comput. 1988; 26(1):20–26. [PMID:3199896]
http://dx.doi.org/10.1007/BF02441823
17.
Schatz P, Putz BO. Cross-validation of measures used for computer-based assessment of concussion. Appl Neuropsychol. 2006;13(3):151–59. [PMID:17361667]
http://dx.doi.org/10.1207/s15324826an1303_2
18.
Schmiedek F, Lövdén, Lindenberger U. A task is a task is a task: Putting complex span, n-back, and other working memory indicators in psychometric context. Front Psychol. 2014;5:1475. [PMID:25566149]
https://doi.org/10.3389/fpsyg.2014.01475
19.
Diehr MC, Heaton RK, Miller W, Grant I. The Paced Auditory Serial Addition Task (PASAT): Norms for age, education, and ethnicity. Assessment. 1998;5(4):375–87.
[PMID:9835661]
http://dx.doi.org/10.1177/107319119800500407
This article and any supplementary material should be cited as follows:
Baruch M, Barth JT, Cifu D, Leibman M. Utility of a multimodal neurophysiological assessment tool in distinguishing between individuals with and without a history of mild traumatic brain injury. J Rehabil Res Dev. 2016; 53(6): 959–72.
http://dx.doi.org/10.1682/JRRD.2015.06.0120
iThenticateCrossref

Go to TOP

VHA

Last Reviewed or Updated Wednesday, February 22, 2017 12:29 PM

Valid HTML 4.01 Transitional