Development and assessment of a neuropsychological battery to aid in predicting driving performance

Janet P. Szlyk, PhD; Lara Myers, MA; YuanXia Zhang, PhD; Linda Wetzel, PhD; Rita Shapiro, DO
Department of Research and Development, Department of Veterans Affairs Chicago Health Care System, West Side Division; Department of Ophthalmology and Visual Sciences, University of Illinois at Chicago (UIC) Eye Center, University of Illinois at Chicago College of Medicine, Chicago, IL

Abstract: Purpose. This study was conducted to select a neuropsychological battery that correlated with driving simulator skills, thus enabling practitioners to provide information to older patients and their families about driving risks. Methods. The study was conducted in two phases. In Phase 1, a survey inquiring as to the kinds of neuropsychological tests currently used to screen patients for driving was sent to 292 licensed neuropsychologists. Of these 292 surveys, 125 (43%) were returned. We used the responses to develop a battery of nine tests, including eight neuropsychological tests and one other cognitive measure: (1) the Seashore Rhythm Test, (2) Logical Memory (Immediate [I] and Delayed [II]) of the Wechsler Memory Scale--Revised (WMS-R), (3) WMS-R Visual Reproduction (Immediate [I] and Delayed [II]), (4) Trails A and B, (5) Digit Span, (6) Digit Symbol, (7) Block Design, (8) Visual Form Discrimination, and (9) a Zoo Map Test. The complete battery included 12 measures. In Phase 2, 22 licensed drivers were recruited ranging in age from 67 to 91 years (14 males and 8 females). The Mini-Mental Status Exam (MMSE) was administered to all subjects. Scores on this test served as a criterion cutoff for placement into a group of subjects with suspected dementia (Group 1, MMSE score below 25) or a group of control subjects (Group 2, with a MMSE score of 25 or above). None of the patients had any gross motor difficulties. Following screening, subjects were administered the neuropsychological battery, a driving simulator test, and a Driving Habits Interview. Results. Data revealed a significant difference between the performance of Groups 1 and 2 on the driving simulator test in two distinct areas, staying within one's lane boundaries and speed. The suspected dementia subjects had significantly more lane boundary crossings than the control subjects and drove at significantly slower speeds. Ten neuropsychological measures correlated with driving simulator performance. The number of lane boundary crossings correlated with the greatest number of neuropsychological tests, with more lane boundary crossings correlated with poorer performance on the neuropsychological tests. In particular, Trails A, Trails B, and Logical Memory (Immediate) correlated with the largest number of driving measures. Conclusion. Preliminary findings show that commonly used neuropsychological tests correlated with driving simulator skills as measured with a driving simulator. Because the driving simulator has been shown to be correlated with actual on-road driving, one may hypothesize that these neuropsychological tests may be predictive of on-road driving. This research is important in defining an appropriate battery to screen for driving skills in patients with known or suspected dementia.

Key words: aging, dementia, driving, neuropsychological testing.

This material was based on work supported by a grant from the Rehabilitation Research and Development Service of the Department of Veterans Affairs, Washington, DC (JPS), and an unrestricted grant from Research to Prevent Blindness, Inc.
Address all correspondence and requests for reprints to Janet Szlyk, PhD; Research and Development Service, VA Chicago Health Care System, West Side Division, 820 South Damen M/C 151, Chicago, IL 60612; 312-996-1466; fax: 312-996-6685; email: janeszly@uic.edu.

INTRODUCTION

Approximately 4 million Americans are estimated to have Dementia of the Alzheimer's Type (DAT). As the population continues to grow, Americans are living longer and the number with DAT is estimated to be 14 million by 2050 [1]. There is mounting evidence that patients with early Alzheimer's disease have impairments in multiple domains, including cortical visual perception and processing, visual attention, and selective attention, as well as in memory, insight, risk-taking, and judgment [2]. As the disease progresses, cognitive abilities worsen, and persons with DAT find performing daily activities, such as driving, increasingly difficult [3].

The assessment of an individual's ability to continue to drive after developing dementia is an increasingly frequent clinical problem. Because of the increased rate of dementia and crash risk in older individuals, accurately assessing driving skills in older individuals becomes essential. One clinically convenient way to assess these individuals is to use neuropsychological tests. Previous studies on the effectiveness of neuropsychological tests to assess driving-related skills have had varying results. Trobe et al. examined neuropsychological test scores as predictors of adverse driving events [4]. They reviewed crash and violation rates of 143 licensed drivers with DAT and 715 licensed drivers who were control subjects and found that neuropsychological test scores did not predict future crashes or violations. Their findings may be because there were too few crashes in this group to obtain statistical significance.

In contrast, Fitten, Perryman, and Wilkinson examined two mild dementia groups, a diabetic control group, an older control group, and a young control group on an on-road driving test and on tests that measure attention, perception, and memory [5]. Short-term memory, visual tracking, and the Mini-Mental Status Exam (MMSE) scores had the highest correlations with an overall driving score [6]. These researchers concluded that the type and degree of cognitive impairment are better predictors of driving skills than age or medical diagnosis. Bieliauskas et al. found clear differences between patients with DAT and control subjects on neuropsychological perceptual and reaction-time test measures, and on-road driving performance [7]. With the use of neuropsychological tests, these data provide some preliminary suggestions toward predicting certain aspects of driving safety among older individuals.

As an alternative to on-road driving, driving simulator testing is a safe and efficient way to measure driving-related skills. It may provide unbiased evidence to aid in the difficult clinical decisions as to whether older individuals should continue to drive [8]. Bieliauskas et al. recommended the use of a driving simulator that will strongly challenge the driver to make active and informed driving decisions, such as measures of reaction to scenarios such as oncoming vehicles and a child running across the street [7]. The driving simulator has the potential to identify the neuropsychological tests that correlate with driving abilities. This is because a great deal of information about driving-related skills can be obtained in relatively short time periods, under controlled conditions. Because real-world accidents are infrequent events and the variability on this measure is so narrow, previous studies have been hindered by this restricted variance and the statistical power necessary to obtain correlations between real-world accidents and neuropsychological tests [4].

In the current study, we examined the relationships between a series of neuropsychological tests and driving simulator performance. We tested patients diagnosed with dementia related to Alzheimer's disease (using NINCDS criteria), and vascular dementia and compared their scores with normal control patients [9]. We hypothesized that patients with lower (poorer) MMSE scores would also have worse scores on many of the neuropsychological tests, which would in turn best correlate with poorer performance on the driving simulator test. Our goal was to determine which neuropsychological tests best predicted driving simulator performance in older subjects. Our neuropsychological test battery was based on the results of a questionnaire that was sent to 292 neuropsychologists asking them to rate which neuropsychological tests would best assess driving-related skills. We also included in our battery the Zoo Map Test, a cognitive measure that assesses executive functioning, and a self-reported Driving Habits Interview (provided by the National Highway Transportation Safety Administration) [10].

METHODS

Phase 1: Neuropsychological Test Battery Development
The neuropsychological test battery was developed from a survey questionnaire that was sent to 292 board-certified neuropsychologists. The first page of the questionnaire asked the neuropsychologists to indicate the characteristics that best described their patient population in terms of race, education, and occupation. From a list provided on the second page of the questionnaire, clinicians were asked to indicate tests they used most frequently in their evaluation of older drivers and to add any tests that were not listed. They were asked to rank the list of neuropsychological tests from one to five, where a score of one would be used to label the least discriminating test in terms of identifying problem drivers, and a five would be used to label the most discriminating test. Table 1 contains the results from the survey.

Table 1 . Tests used by surveyed neuropsychologists to assess driving skills.

Of the 292 neuropsychologists, 125 responded to the survey (43 percent response rate), and of those responses, 41 (33 percent) indicated that they did perform driving assessments on older individuals. Those 41 who performed assessments stated that of the populations they served, 77 percent were Caucasian, 14 percent were African American, 6 percent were Hispanic, and 3 percent were described as "Other." They reported seeing patients from all levels of education, the majority of their patients having 12 years of school. These neuropsychologists reported seeing patients from all occupations, the majority being managers, administrators, clerical workers, and salespersons.

We chose the neuropsychological tests that were used in Phase 2 of the present study based on the survey results, and conferences with a neuropsychologist, a neurologist, and a psychologist, who are all authors of the present study. The tests were selected after considering frequency of use by neuropsychologists and representativeness of visuocognitive motor skills related to driving. The final measures that we decided upon included--

1. The Seashore Rhythm Test that is a test of auditory attentional skills [11].
2. The Logical Memory I and II Subtests from the Wechsler Memory Scale--Revised (WMS-R) that measure immediate (I) and delayed (II) verbal memory [12].
3. The Visual Reproduction I and II Subtests (also from the WMS-R) that measure immediate (I) and delayed (II) visual memory.
4. Trails A and B Tests that measure mental tracking abilities and mental flexibility [13].
5. The Digit Symbol Subtest from the Wechsler Adult Intelligence Scale--Revised (WAIS-R) that measures speed of information processing [14].
6. The Digit Span Subtest from the WAIS-R that measures attention and mental manipulation.
7. The Block Design Subtest of the WAIS-R that assesses visual-spatial organization and visuomotor coordination.
8. The Visual Form Discrimination Test that measures attention to visual detail [15].

The Zoo Map Subtest of the Behavioral Assessment of the Dysexecutive Syndrome Test that measures executive planning skills and mental tracking was added to the neuropsychological battery [10]. We administered the MMSE to all subjects and used the scores as a criterion cutoff for placement into a group of subjects with suspected dementia (Group 1, score below 25) or a group of control subjects (Group 2, with a score of 25 or above) [6]. Increasing age and lower educational levels are known to decrease the normative scores for the MMSE. Therefore, based on the published age-related modifications of the MMSE norms, we chose 25 as a discriminating criterion for MMSE scores between Groups 1 and 2 [16].

Phase 2: Driving Simulator Assessment
Participants
We included 22 participants (14 men and 8 women), recruited from the Department of Veterans Affairs (VA) Chicago Health Care System, West Side Division Memory Clinic and the Geriatric Clinic, and from the University of Illinois College of Medicine Memory Clinic. These partici-pants were screened by a neurologist who is one of the authors (RS). Based upon the MMSE scores, the 22 subjects were placed into one of the two groups: Group 1 with suspected dementia (n = 8; 7 men and 1 woman), or Group 2, which served as the normal control group (n = 14; 7 men and 7 women). The age of the subjects from Groups 1 and 2 ranged from 67 to 85 years (mean [m] = 75.6 years; standard deviation [SD] = 7.0 years) and from 70 to 91 years (m = 77.0 years; SD = 6.2 years), respectively. The MMSE scores from Groups 1 and 2 ranged from 8 to 24 (m = 19.6; SD = 5.7) and from 26 to 30 (m = 27.9; SD = 1.2), respectively. Participants had driving experience ranging from 40 to 77 years (m = 57.2 years; SD = 9.1 years). All subjects were licensed drivers; 21 of the 22 subjects were currently driving. Participants were ambulatory with preserved motor coordination. All subjects had driving experience in the past 2 years. Individuals with severe cardiovascular disease, large strokes resulting in significant visual or motor impairment, seizure disorder, and visual impairments were excluded from the study.

Procedure
Neuropsychological Test Battery and Driving Habits Interview. The Neuropsychological Test Battery developed in Phase 1 was administered to all subjects. A Driving Habits Interview (provided by the National Highway Transportation Safety Administration), a 34-item self-report questionnaire that asks subjects about their driving behaviors, was also administered. The administration of the entire neuropsychological test battery, the Zoo Map Test, and the Driving Habits Interview took approximately 2 hours.

Driving Simulator Test. All participants underwent testing on an interactive driving simulator that was developed in collaboration with the Atari Corporation (Milipitas, California) and has been described in previous studies [17-19]. The simulator consisted of a seat, a steering wheel, gas and brake pedals, and an automatic transmission. The visual display system was composed of three 62.5-cm color monitors that were synchronized to display the appropriate view of a computer-generated environment. The visual display system provided 160° of horizontal viewing field and 35° of vertical viewing field to the subject seated 57.5 cm from the center screen. Testing was performed with the room lights off. We instructed subjects to operate the simulator, because they would normally drive their own car, and to obey all traffic signs and signals along the roadway. Following a 10- to 15-minute training session on a practice course (session length determined by patient competency), we collected data for subjects' responses during an 8-minute session of driving the course. The driving simulator provided the patients with challenges requiring them to make active driving decisions.

The simulator indexes that were analyzed included—

1. Mean speed (in miles per hour).
2. Brake pedal pressure (in arbitrary units, calculated as the SD in brake pedal pressure during the session).
3. Number of lane boundary crossings (defined as one of the four tires crossing over any of the lanes' boundaries).
4. Braking response time to a stop sign (defined as the elapsed time between the presentation of a stop sign and the initiation of a brake pedal response).
5. Braking response time to a traffic light (defined as the elapsed time between the presentation of a suspended red traffic light and the initiation of a brake pedal response).
6. Slope of the braking response curve (calculated as the change in speed divided by the change in time).
7. Horizontal and vertical eye movement (the SD of eye movement along the horizontal axis and along the vertical axis).
8. Number of times the subject ran stop signs and stoplights (calculated as the number of stop signs and stoplights run by the participant).
9. Number of near accidents (calculated as the number of times that a participant encounters a near accident).
10. Simulator accidents.

Previous studies have demonstrated a significant correlation between driving simulator measures and real-world driving in three major areas: lane boundary crossings, speed, and simulator accidents [20-21]. Fewer lane boundary crossings on the simulator were correlated with better scores on several subindexes of the road test, including a composite score for merging, maintaining proper lane position while making a left turn, adjusting speed properly while merging, visually checking blind spot while merging, determining sufficient distance to merge, and using a signal properly. Lower speeds on the simulator were correlated with worse scores for lane observance and worse scores for proper use of a signal before merging. More simulator accidents were related to worse scores for lane observance in the real-world condition. These studies provide data to demonstrate the validity of the driving simulator as it relates to real-world driving.

RESULTS

Neuropsychological Tests
The test results of Groups 1 and 2 on the neuropsychological tests are shown in Table 2. We performed parametric t-tests and nonparametric T-tests to analyze for differences between the groups. The following tests were significant at p ≤ 0.001: Logical Memory I, Logical Memory II, Trails B, Digit Symbol, and Digit Span. Trails A was significant at p ≤ 0.01. The following tests showed significant differences between the groups at p ≤ 0.05: Visual Memory (Immediate), Visual Form Discrimination, and Block Design. Visual Memory II, the Zoo Map Test, and the Seashore Rhythm Test did not show significant differences between the groups.

Table 2. Significant differences between Groups 1 (suspected dementia) and 2 (control) on neuropsychological measures. If mean is indicated, a t is presented; if median is indicated, a T is presented.

Driving Simulator and Neuropsychological Tests
The correlations between neuropsychological test scores and the driving simulator performance measures are shown in Table 3. We performed correlations on the tests using either a Pearson correlation with parametric data or a Spearman correlation with nonparametric data. The following relationship was significant at p ≤ 0.001: Trails A and number of lane boundary crossings. The following relationships were significant at p ≤ 0.01: Logical Memory I and number of lane boundary crossings, Logical Memory I and brake pedal pressure, Seashore Rhythm Test and number of lane boundary crossings, Trails A and brake pedal pressure, Trails B and number of lane boundary crossings, Trails B and speed, and Trails B and brake pedal pressure. The following correlations were significant at p ≤ 0.05: Logical Memory II and speed, Logical Memory II and brake pedal pressure, Visual Memory I and number of lane boundary crossings, Block Design and number of lane boundary crossings, Block Design and speed, Digit Span and brake pedal pressure, Digit Span and horizontal eye movement, Digit Symbol and number of lane boundary crossings, Trails A and speed, and Visual Form Discrimination and horizontal eye movement.

Table 3. Correlations between neuropsychological tests and driving simulator performance.

Driving Simulator Performance
The differences between the performance of Groups 1 and 2 on the driving simulator are shown in Table 4. We made comparisons using a parametric t-test or a nonparametric Mann Whitney Rank Sum test. Significant differences (p ≤ 0.05) were shown between the suspected dementia and control groups with the suspected dementia subjects having slower speeds and more lane boundary crossings.

Table 4. Driving simulator performance measures for Groups 1 (suspected dementia) and 2 (control). If mean is indicated, a t is presented; if median is indicated, a T is presented.

Driving Habits Interview
The differences between Groups 1 and 2 on the Driving Habits Interview are shown in Table 5. We analyzed data using a parametric t-test or a nonparametric Mann Whitney Rank Sum Test. The following differences were significant at p ≤ 0.05: distant towns, total passengers rode with subject over the past year, and total places driven to in an average week. The control Group 2 drove to more distant towns, with more passengers over the past year, and to more places in an average week than the suspected dementia Group 2.

Table 5. Results of driving habits interview for Groups 1 (suspected dementia) and 2 (control). If mean is indicated, a t is presented; if median is indicated, a T is presented.	Table 5 (continured)

We performed correlational analyses between the results of the MMSE and the Driving Habits Interview. The results are shown in Table 6. The following correlations were significant at p ≤ 0.05 with the MMSE scores: seat-belt use, total places driven in a week, total passengers rode with subjects over the past year, traveling to distant towns, traveling outside of Illinois, traveling outside of the Midwest, and number of traffic tickets. Participants with a higher (better) MMSE score wore their seat belts more often, drove to more places, and drove with more passengers than subjects with a lower (poorer) MMSE score. Participants with a higher MMSE score also drove more often to more distant towns, outside of Illinois, and outside of the Midwest than subjects with a lower MMSE score drove, and had fewer traffic tickets. No significant correlations were found between MMSE scores and number of accidents or number of times pulled over by the police.

Table 6. Correlations between MMSE and driving habits interview.