Journal of Rehabilitation Research & Development (JRRD)

Quick Links

  • Health Programs
  • Protect your health
  • Learn more: A-Z Health
Veterans Crisis Line Badge
 

Volume 53 Number 3, 2016
   Pages 379 — 390

Utility of the Validity-10 scale across the recovery trajectory following traumatic brain injury

Sara M. Lippa, PhD;1–2* Rael T. Lange, PhD;1–3 Jason M. Bailie, PhD;1,4 Jan E. Kennedy, PhD;1,5 Tracey A. Brickell, D.Psych;1–2,6 Louis M. French, PsyD1–2,6

1Defense and Veterans Brain Injury Center and 2National Intrepid Center of Excellence, Walter Reed National Military Medical Center, Bethesda, MD; 3Department of Psychiatry, University of British Columbia, Vancouver, Canada; 4Naval Medical Center, San Diego, San Diego, CA; 5San Antonio Military Medical Center, San Antonio, TX; 6Center for Neuroscience and Regenerative Medicine, Uniformed Services University of the Health Sciences, Bethesda, MD

Abstract — The Validity-10 scale was recently developed to screen for symptom exaggeration in patients following traumatic brain injury (TBI). However, it has only been validated on patients with TBI largely in the chronic phase of recovery. The influence of time since injury on the Validity-10 scale was investigated in 2,661 male servicemembers with TBI presenting to six U.S. Defense and Veterans Brain Injury Centers. Participants completed the Neurobehavioral Symptom Inventory (NSI). The Validity-10 scale and NSI total score were both weakly statistically significantly (1) positively correlated with time since injury, (2) negatively correlated with bodily injury severity, and (3) higher in participants undergoing medical board evaluations than in participants who returned to duty or were still hospitalized. Participants were statistically more likely to screen positive for possible symptom exaggeration on the Validity-10 scale as time since injury increased. However, the Validity-10 scale was only weakly related to time since injury, TBI severity, bodily injury severity, disposition, age, and return to duty status. That false positives are not increased in the acute phase of recovery and that the Validity-10 scale is not strongly related to clinical factors support the use of the Validity-10 scale in the acute recovery phase and across the TBI recovery trajectory.

Key words: military, Neurobehavioral Symptom Inventory, postconcussion symptoms, symptom exaggeration, symptom report, symptom validity, TBI, time since injury, traumatic brain injury, Validity-10 scale.

Abbreviations:ANCOVA = analysis of covariance, GWOT = Global War on Terror, ISS = Injury Severity Score, LOC = loss of consciousness, NSI = Neurobehavioral Symptom Inventory, PTA = posttraumatic amnesia, SVT = symptom validity test, TBI = traumatic brain injury.
*Address all correspondence to Sara M. Lippa, PhD; Defense and Veterans Brain Injury Center, Walter Reed National Military Medical Center, Building 19, Room 6003, 8901 Wisconsin Ave, Bethesda, MD 20889; 301-400-1872. Email: sara.lippa.ctr@mail.mil; saralippa@gmail.com
http://dx.doi.org/10.1682/JRRD.2015.01.0009
INTRODUCTION

Traumatic brain injury (TBI) has been a common occurrence in the Global War on Terror (GWOT) operations, with 15 to 23 percent of servicemembers sustaining a TBI during deployment [1–2]. Following a TBI, cognitive, emotional, physical, sensory, and/or sleep symptoms may be reported. Collectively known as postconcussive symptoms, these may be related to TBI itself or caused by a number of other factors unrelated to TBI (e.g., vestibular injury, depression, posttraumatic stress, anxiety, sleep disturbance, chronic headaches or bodily pain, personality characteristics, and diverse social psychological factors [3–5]). Regardless of etiology, the symptoms reported following TBI determine whether a servicemember is fit for duty, is deployable, requires a medical retirement, and/or requires a disability pension. These external factors may motivate under- or overreporting of symptoms. If patients are concerned that their history of TBI may limit their immediate or long-term career opportunities, they may be motivated to not report their TBI or minimize their symptoms. Underreporting of TBI and postconcussive symptoms is a particular concern in athletes [6–10]. In contrast, overreporting symptoms is often found in patients who are involved in litigation or otherwise have potential for secondary gain [11–15]. Performance validity test failure rates have previously been shown to be high in Active Duty and Veteran samples, ranging from 11 to 59 percent [16–24]. These failure rates vary by evaluation context. Higher rates of failure have been reported in servicemembers and Veterans undergoing medical board evaluations and compensation and pension evaluations, compared with evaluations that do not have clear financial consequences [18,21]. Elevated failure rates have also been found in servicemembers and Veterans undergoing research evaluations to be used clinically rather than strictly for research purposes [20].

Symptom Validity Assessment

Symptom overreporting has been reported at rates of 9.8 [25], 22 [26], 33 [24], and 38 percent [27] in GWOT Veteran and military TBI samples. Detection of symptom exaggeration is paramount. If patients exaggerate their symptoms and this goes undetected, providers may conclude that the patients have more problems than they do in reality and may initiate improper treatments and disability classifications. This not only has substantial financial consequences for the Government and taxpayers but also could increase iatrogenesis because patients may come to believe that they have been put in particular treatments and given certain disability ratings because they truly do have many problems related to their TBI. Beliefs regarding mild TBI have been shown to be related to return to work, functional capacity, and postconcussive symptom report [28–30].

Symptom validity tests (SVTs) are designed to assess under- or overreporting of the presence and/or severity of symptoms using self-report measures or structured interviews. Ideally, the possibility that a person has minimized or exaggerated symptoms is evaluated using well-validated measures designed for this purpose, such as the Minnesota Multiphasic Personality Inventory-2 [31], Personality Assessment Inventory [32], Structured Inventory of Malingered Symptoms [33], or Structured Inventory of Reported Symptoms [34]. Unfortunately, clinicians often do not have time to administer these lengthy assessments; however, embedded measures of symptom validity, which are derived from existing self-report questionnaires, offer the advantage of screening for veracity with minimal or no additional resources.

Validity-10 Scale

Recently, Vanderploeg et al. [26] developed and cross-validated a potential embedded measure of symptom exaggeration from a commonly administered postconcussion symptom scale, the Neurobehavioral Symptom Inventory (NSI) [35]. The NSI is administered to every Veteran who screens positive for TBI at a Department of Veterans Affairs hospital [36] and every servicemember who has a concussion medical encounter [37]. Vanderploeg et al. identified NSI items that are most strongly correlated with report of atypical symptoms (NIM-5) and the items that were most infrequently endorsed (LOW-6) and combined these items to form the Validity-10 scale [26]. The Validity-10 scale has been suggested as a screen to identify possible symptom exaggeration, with positive screens indicating the need for a more in-depth analysis of symptom veracity.

The Validity-10 scale has primarily been investigated in patients who were in the chronic phase of recovery from injury. The original study was cross-validated on 206 military servicemembers an average of 32 wk following a TBI using the Personality Assessment Inventory Negative Impression Management scale as an external criterion for defining genuine versus invalid responding [26]. A cutoff of >22 was identified as optimal, with a sensitivity of 0.61 and a specificity of 0.85. In a follow-up study, Lange et al. examined the Validity-10 scale in 63 military servicemembers an average of 78 wk postinjury [27]. They used the Minnesota Multiphasic Personality Inventory-2-Restructured Form to differentiate between genuine versus invalid responding [38]. This study concluded that a cutoff of >12 was optimal for when providers do not want to miss any potential symptom exaggeration, with a sensitivity of 0.63, specificity of 1.0, positive predictive power of 0.93, and negative predictive power of 0.83. Finally, in a replication of the original cross-validation study, Lange et al. investigated the Validity-10 scale in 272 military servicemembers an average of 33 wk following a TBI [25]. A cutoff of >18 was found to be optimal for differentiating possible symptom exaggeration from genuine responding but also supported the cutoff of >12 in situations in which it is particularly important not to miss potential invalidity. In this study, the cutoff of >18 had a sensitivity of 0.59, specificity of 0.89, positive predictive power of 0.74, and negative predictive power of 0.80. The cutoff of >12 had a sensitivity of 0.74, specificity of 0.71, positive predictive power of 0.58, and negative predictive power of 0.83.

Purpose

The Validity-10 scale has previously been studied in patients who are largely in the chronic phase of recovery, and it remains unclear whether the previously established cutoffs can be applied to patients in the acute recovery phase. The main aim of the current study was to determine whether there is a significant relationship between time since injury and the Validity-10 scale, and in particular, whether the previously suggested Validity-10 scale cutoffs result in similar classification rates across the acute, subacute, and chronic phases of recovery from TBI. This study also sought to determine whether other injury characteristics that may be related to postconcussion symptom reporting, such as age, TBI severity, bodily injury severity, and return to duty status, were related to the Validity-10 scale.

METHODS
Participants and Procedures

The final sample consisted of 2,661 U.S. military servicemembers who sustained a mild (87.8%), moderate (6.8%), or severe (5.4%) TBI that was either GWOT-related (85.4%) or non-combat-related (14.6%). These participants were evaluated by the Defense and Veterans Brain Injury Centers at the San Antonio Military Medical Center, San Antonio, Texas (36.0%); Walter Reed National Military Medical Center, Bethesda, Maryland (or its predecessor, the Walter Reed Army Medical Center) (24.8%); Naval Medical Center, San Diego, San Diego, California (14.7%); Naval Hospital Camp Pendleton, Oceanside, California (13.5%); Wilford Hall Ambulatory Surgical Center, San Antonio, Texas (7.1%); and Robert E. Bush Hospital, Twentynine Palms, California (3.8%). Participants were selected from a larger sample of 3,205 patients with mild to severe TBI. Participants were excluded if they were >6.5 yr postinjury (n = 2), did not complete or had an incomplete NSI (n = 251), were female (n = 155), had a penetrating head injury (n = 59), or had missing data on key variables (i.e., time since injury [n = 303], sex [n = 21], or whether TBI was penetrating [n = 23]). This study excluded females given previous findings that women tend to report more postconcussion symptoms than men [39–41].

Excluded participants did not differ from included participants in terms of time since injury, NSI total score, Validity-10 scale total score, or bodily injury severity (all p > 0.06). Age, sex, return to duty status, and TBI severity did significantly differ between the included and excluded participants. Excluded participants were older (mean ± standard deviation: 28.6 ± 8.2 yr) than included participants (27.47 ± 7.39 yr; F(1, 692.6), p = 0.002), were female (29.6% vs 0%; p < 0.001), were more likely to return to duty (43.1% vs 33.4%; χ2 (1, n = 2,842) = 14.335, p < 0.001), less likely to be on limited duty or a medical hold (36.5% vs 49.9%; χ2 (1, 2,842) = 25.415, p < 0.001), and more likely to be undergoing a medical board (7.5% vs 4.0%; χ2 (1, 2,842) = 9.825, p = 0.002). Excluded participants were more likely to meet criteria for severe TBI (10.2% vs 5.4%; χ2 (1, 2,980) = 15.097, p < 0.001) than included participants. Of the 2,661 participants included in this study, some were missing data regarding age (n = 2), TBI severity (n = 152), bodily injury severity (n = 415), and return to duty status (n = 244) and were therefore excluded from relevant analyses.

Measures
Demographics, Traumatic Brain Injury Characteristics, and Return to Duty Status

Demographic information, TBI characteristics, and return to duty status were collected through medical chart review (including in-theater medical records when available), interview of the patient and family, collection of other collateral information, and/or case conferencing. TBI severity was first divided into mild, moderate, and severe groups based on commonly used severity classification systems [42]. The mild TBI group was then divided into three groups based on the presence or absence of loss of consciousness (LOC) and/or posttraumatic amnesia (PTA) [43] and the presence or absence of neuroimaging findings [44]. The five final groups were (1) equivocal mild TBI (alteration of consciousness only), (2) uncomplicated mild TBI (absence of intracranial abnormality and LOC ≤30 min or PTA <24 h), (3) complicated mild TBI (presence of intracranial abnormality and LOC ≤30 min or PTA <24 h), (4) moderate TBI (LOC >30 min–24 h or PTA 1–7 d), and (5) severe TBI (LOC >24 h or PTA >7 d). Glasgow Coma Scale scores were not available. In order to differentiate between different phases of acute, subacute, and postacute recovery, time since injury was divided into seven a priori groups for most analyses: (1) ≤1 wk, (2) >1 wk to ≤1 mo, (3) >1 to ≤3 mo, (4) >3 to ≤6 mo, (5) >6 mo to ≤1 yr, (6) >1 to ≤3 yr, and (7) >3 to ≤6.5 yr. Return to duty status was divided into four groups for most analyses: (1) return to duty, (2) limited duty (medical hold), (3) pending medical board, and (4) still hospitalized.

Postconcussion Symptoms and Validity-10 Scale

The NSI is a 22-item questionnaire of postconcussion symptoms [35]. Whether the NSI is an accurate assessment of emotional and functional outcomes has yet to be established; however, it has been recommended as a common data elements outcome measure for TBI research [45]. It is also one of two core outcome measures within the military healthcare system [46]. Participants are asked to rate symptoms on a 5-point Likert scale ranging from 0 (none) to 4 (very severe), with higher scores related to increased symptom report. The Validity-10 scale is calculated by summing the scores on 10 items that are infrequently endorsed and/or related to atypical symptom report [26]. Scores on the Validity-10 scale range from 0 to 40. We investigated Validity-10 scale failure rates at three previously suggested cutoffs: >22 [26], >18 [25], and >12 [25,27].

Bodily Injury Severity

Bodily injury severity was measured using the Injury Severity Score (ISS) derived from the Abbreviated Injury Scale [47]. On the ISS, higher scores are associated with increased injury severity. For the current study, a modified ISS that excluded intracranial injuries was calculated. In addition to using the ISS as a continuous variable, in some analyses, ISS scores were collapsed into the following categories [48]: minor (1–3), moderate (4–8), serious (9–15), severe (16–24), and critical (25–75).

RESULTS
Demographics and Clinical Characteristics

Table 1 presents primary summary data for the total sample, including demographics, injury characteristics, and clinical characteristics.


Table 1. 
Age, yr (n = 2,659)
Validity-10 Scale and Neurobehavioral Symptom Inventory Total Score

Figure 1 presents the means and standard errors of the NSI total scores and the Validity-10 scale for each time since injury group, controlling for age, TBI severity, bodily injury severity, and return to duty status. Analyses of covariance (ANCOVAs) were conducted to assess how age, TBI severity, bodily injury severity, and return to duty status (all entered as covariates) and time since injury are related to the NSI total score and Validity-10 scale. All post hoc analyses were conducted with p < 0.01 to minimize experiment-wide type I error.


Figure 1. Scores (mean) on Neurobehavioral Symptom Inventory (NSI) total score and Validity-10 scale as function of time since injury, controlling for age, traumatic brain injury severity, bodily injury severity, and return to duty status. Note: Error bars represent standard error.

Figure 1.

Scores (mean) on Neurobehavioral Symptom Inventory (NSI) total score and Validity-10 scale as function of time since injury, controlling for age, traumatic brain injury severity, bodily injury severity, and return to duty status. Note: Error bars represent standard error.

Click Image to Enlarge. View as PowerPoint Slide

Validity-10 Scale

The overall ANCOVA model investigating factors related to the Validity-10 scale was significant (F(10, 2,011) = 18.292, p < 0.001). Age (F(1, 2,011) = 10.765, p = 0.001), TBI severity (F(1, 2,011) = 6.333, p = 0.01), bodily injury severity (F(1, 2,011) = 29.481, p < 0.001), return to duty status (F(1, 2,011) = 46.492, p < 0.001), and time since injury group (F(6, 2,011) = 18.552, p < 0.001) all significantly predicted the Validity-10 scale score. Post hoc tests revealed that the Validity-10 scale score increased with time since injury. Most time since injury groups had significantly higher Validity-10 scale scores than earlier time since injury groups. The only nonsignificant differences on Validity-10 scale scores were between the following participant groups: >1 to ≤3 mo versus >3 to ≤6 mo, >3 to ≤6 mo versus >6 mo to ≤1 yr, >1 to ≤3 yr versus >3 to ≤6.5 yr, >6 mo to ≤1 yr versus >1 to ≤3 yr, and >6 mo to ≤1 yr versus >3 to ≤6.5 yr.

To further investigate the unique contribution of each of the covariates toward the prediction of the Validity-10 scale, a series of five hierarchical linear regression analyses was performed. Each included all covariates and predictor variables and entered a single variable in the last step. Time since injury (R2 total = 0.80; R2 change = 0.047) contributed the most unique variance toward the prediction of the Validity-10 scale; however, it only explained an additional 4.7 percent of the variance. Return to duty status and bodily injury severity explained approximately 2 percent of variability in Validity-10 scale scores (R2 change = 0.019 and 0.013, respectively). Age (R2 change = 0.004) and TBI severity (R2 change = 0.003) all statistically improved the model but contributed less than 0.5 percent unique variance to the model; therefore, they were deemed not meaningful and were not included as covariates in subsequent analyses.

Neurobehavioral Symptom Inventory Total Score

The overall ANCOVA model investigating factors related to the NSI total score was significant (F(10, 2,011) = 26.165, p < 0.001). TBI severity (F(1, 2,011) = 3.941, p = 0.047), bodily injury severity (F(1, 2,011) = 34.475, p < 0.001), return to duty status (F(1, 2,011) = 54.345, p < 0.001), and time since injury group (F(6, 2,011) = 31.670, p < 0.001) were all significantly related to the NSI total score. In contrast, age was not related to NSI total score (F(1, 2,011) = 3.545, p = 0.06). Post hoc analyses revealed that the NSI total score increased with time since injury. Most time since injury groups had significantly higher NSI total scores than earlier time since injury groups. The only nonsignificant differences were between the following groups: >3 to ≤6 mo versus >6 mo to ≤1 yr, >6 mo to ≤1 yr versus >3 to ≤6.5 yr, and >1 to ≤3 yr versus >3 to ≤6.5 yr.

Validity-10 Scale Failure Rates

Table 2 presents Validity-10 scale failure rates, stratified by time since injury, bodily injury severity, and return to duty status (variables that explained over 1% additional variance to Validity-10 scale). Validity-10 scale failure rates are presented using three different cutoff scores (i.e., >22, >18, >12) as recommended by previous research [25–27]. Chi-square analysis indicated that the number of participants exceeding all three of the Validity-10 scale cutoffs differed across time since injury groups (i.e., >22 cutoff, χ2 (6, 2,661) = 19.230, p = 0.004; >18 cutoff, χ2 (6, 2,661) = 35.491, p < 0.001; and >12 cutoff, χ2 (6, 2,661) = 74.278, p < 0.001). The number of participants exceeding all three cutoffs increased as time since injury increased. Bodily injury severity was related to exceeding the Validity-10 scale >22 cutoff (χ2 (4, 2,246) = 10.164, p = 0.04) and >12 cutoff (χ2 (4, 2,246) = 32.295, p < 0.001). Similarly, bodily injury severity approached, but did not reach, significance at the >18 cutoff (χ2 (4, 2,246) = 9.243, p = 0.06). Participants with mild and moderate bodily injuries were more likely to exceed the >22 and >12 cutoffs than participants with serious, severe, and critical injuries. Return to duty status was related to exceeding the Validity-10 scale >22 cutoff (χ2 (3, 2,417) = 24.335, p < 0.001), >18 cutoff (χ2 (3, 2,417) = 49.113, p < 0.001), and >12 cutoff (χ2 (3, 2,417) = 55.954, p < 0.001). Participants in the limited duty (medical hold) and pending medical board groups fell above the cutoffs more frequently than participants who were still hospitalized or who had returned to duty.


Table 2. 

Next, the possibility of a moderating relationship between (1) time since injury and bodily injury severity or (2) time since injury and return to duty status on Validity-10 scale failure rates was examined. Each of these categories was collapsed into fewer groups to clarify the relationships and ensure adequate cell counts and power for analyses. Time since injury was collapsed into three groups representing acute, subacute, and chronic recovery: (1) ≤1 mo, (2) >1 mo to 1 yr, and (3) >1 yr. Return to duty status was also collapsed into three groups: (1) return to duty, (2) medical board or limited duty (medical hold) (these two groups were combined because they are both the most likely to have fitness for duty evaluations that might influence the incentive to exaggerate or minimize symptoms), and (3) still hospitalized. Bodily injury severity was collapsed into two groups to distinguish the most severe injuries from more mild injuries: (1) mild or moderate and (2) serious or higher. The three levels of time since injury were then combined with the three levels of return to duty status (e.g., ≤1 mo time since injury and returned to duty) and the two levels of bodily injury severity (e.g., ≤1 mo time since injury and mild or moderate bodily injury) to create mutually exclusive groups. Figure 2 presents failure rates at the >18 cutoff for each of the combined groups. In each group, failure rates increased with time since injury (except for the still hospitalized group, which had a very small sample size at >1 yr postinjury [n = 11]). The time since injury group was related to Validity-10 scale scores >18 in all three return to duty status groups (all χ2 > 9.356, p < 0.01). The time since injury group was related to Validity-10 scale scores >18 in participants with mild or moderate bodily injuries (χ2 (2, 1,649) = 14.515, p = 0.001) but not in participants with serious or higher bodily injuries (χ2 (2, 597) = 1.112, p = 0.57).


Figure 2. Percentage of Validity-10 scale failure at >18 cutoff in different groups. Note: Other Validity-10 scale cutoffs of >22 and >12 are available upon request from authors. Other cutoffs followed a similar pattern, with higher rates of failure at >12 cutoff and lower rates of failure at >22 cutoff for all groups.

Figure 2.

Percentage of Validity-10 scale failure at >18 cutoff in different groups. Note: Other Validity-10 scale cutoffs of >22 and >12 are available upon request from authors. Other cutoffs followed a similar pattern, with higher rates of failure at >12 cutoff and lower rates of failure at >22 cutoff for all groups.

Click Image to Enlarge. View as PowerPoint Slide

DISCUSSION

The main goal of the present study was to examine the relation between time since injury and the Validity-10 scale to determine whether the Validity-10 scale can be used in the acute recovery phase following a TBI without increasing the rates of screening positive for possible symptom exaggeration. The closer participants were to their date of injury, the less likely they were to fall above Validity-10 scale cutoffs. This suggests that the current proposed cutoffs will not result in an increased rate of identifying patients with acute TBI as exaggerating their symptoms. Additionally, although the Validity-10 scale was higher in participants who were evaluated further from their date of injury, this statistically significant relationship is likely the result of large sample size and does not appear to have significant clinical implications given that time since injury explained less than 5 percent of the variance in the Validity-10 scale. This suggests that the Validity-10 scale changes minimally over time and further supports the use of the Validity-10 scale in the acute phase and across the time since injury spectrum. Similarly, other clinical variables, such as TBI severity, bodily injury severity, return to duty status, and age each explained less than 2 percent of the unique variance in the Validity-10 scale, suggesting that the Validity-10 scale is not meaningfully affected by these clinical factors, supporting its use as an SVT. Both the relatively low rate of potential false positives in patients with acute TBI and the rather weak relationship between the Validity-10 scale and a variety of clinical factors support the use of the Validity-10 scale and its previously suggested cutoffs in acute TBI and across different phases of recovery.

Given that symptoms generally improve with time following a TBI, it was interesting that the NSI total score and Validity-10 scale scores were higher in participants assessed many years following their injury than in participants who were assessed days or weeks following their injury. Some studies have demonstrated the persistence of symptoms many years following mild TBI [49–51]; however, in the overwhelming majority of patients, symptom complaints following a mild TBI resolve completely within a year of the injury [52]. Although persisting symptoms are generally thought to be the result of other factors, such as psychological and medical comorbidities [52], the origin of persisting symptoms is unknown. The finding that symptom report was higher as time since injury increased is likely a result of sample bias. Military personnel who are many months or years post mild TBI and asymptomatic are presumably relatively unlikely to actively seek out or be referred for a TBI evaluation. Often, patients who are many months postinjury are referred to clinical care because they are reporting impaired functioning and also have a remote history of TBI; the symptoms and injury may not be related [53]. Further, patients may be referred for evaluation as part of the process for separation from the military such as medical retirement. As a result, the present study includes many participants, years from injury, who are highly symptomatic. These results may not apply to all people who have experienced a TBI (including those who would not normally present for evaluation and/or treatment). The findings are most applicable to patients presenting for evaluation and/or treatment of TBI at military treatment facilities. Additionally of note, this study did not reassess participants across the recovery trajectory but sampled each participant only once. Therefore, this study cannot speak to changes in symptom report or symptom exaggeration over time. It is possible that symptom exaggeration may be best assessed by the change (or lack thereof) in Validity-10 scale score over time rather than the Validity-10 scale score at a single point in time.

The current study found that return to duty status was related to overall postconcussion symptom report and rates of symptom exaggeration in a clinic-based TBI sample. Participants who were on limited duty (medical hold) or undergoing a medical board evaluation reported more symptoms and were at increased likelihood of screening positive for symptom exaggeration based on the Validity-10 scale than participants who had returned to duty or who were still hospitalized. Despite this statistically significant relationship, and after controlling for other variables, return to duty status only explained 1.9 percent of the variance in the Validity-10 scale. Previous research has shown that external incentives increase rates of invalid responding on neuropsychological tests [11–15,18,20–21]; however, the current results suggest only a very mild relationship between external incentives and Validity-10 scale. It is likely that the small increase in Validity-10 scale in participants undergoing medical board evaluations and on limited duty (medical hold) is driven by underlying psychological distress. Although the Validity-10 scale is a screener for symptom validity, it is important to acknowledge that it comprises nearly half of the NSI. Scores on the Validity-10 scale therefore likely increase not only with symptom exaggeration but with an increase of true symptomatology as well. This study did not assess the effect of psychological factors on the Validity-10 scale score. It is likely that psychological factors affect symptom validity; however, the purpose of this study was to examine the Validity-10 scale regardless of etiology of symptom exaggeration. It is difficult to clearly determine the strength of the relationship between symptom exaggeration and emotional distress because psychological distress is measured by self-report and therefore can be exaggerated as well.

Along with time since injury and return to duty status, bodily injury severity was also related to postconcussion symptom report. Possible explanations for this phenomenon have included that patients who have survived severe bodily injuries may view any postconcussion symptoms as minor inconveniences not worth being concerned about and may subsequently underreport these symptoms; patients may have more visible evidence of their recovery and/or adaptation, which may help to confirm that their brain is recovering as well; and patients may have increased social support [54].

The present study also examined three different Validity-10 scale cutoffs that have previously been supported in the literature. Though it did not employ an external criterion to compare Validity-10 scale results (and subsequently calculate sensitivity, specificity, and positive and negative predictive power), the large sample allowed exploration of the implications of using different cutoffs. The cutoff of >12, which has been suggested for use as a screen when providers do not want to miss any potential symptom exaggeration [25], identified a total of 31.9 percent of the participants as needing additional symptom validity testing. There is some concern that patients presenting within 3 mo of injury may have actual neurological symptoms that may be inflating their score above this cutoff. The originally suggested cutoff of >22 [26] identified only 7 percent of the participants as needing additional symptom validity testing and may not be sensitive enough because this is much lower than the majority of previous studies investigating rates of SVT failure in GWOT personnel [24,26–27]. These base rates suggest that the cutoff of >18, which generated a positive screen in 14.7 percent of participants in the current study and has most recently been suggested as the optimal cutoff for identifying possible symptom exaggeration [25], may be the most appropriate of the three suggested cutoffs.

CONCLUSIONS

In sum, the present study was the largest yet to investigate the Validity-10 scale and the first to specifically investigate the relationship between time since injury and Validity-10 scale performance. Validity-10 scale and NSI total score both increased with time since injury; however, the Validity-10 scale was more resistant to change over time than the NSI total score. Participants were significantly more likely to screen positive for possible symptom exaggeration on the Validity-10 scale the further they were from their injury date. This study provides evidence to support the use of the Validity-10 scale in patients with acute, subacute, and chronic TBI. Continued research on the Validity-10 scale and other efficient measures of symptom validity is necessary to improve the accuracy of these instruments because symptom exaggeration has numerous diagnostic, treatment, and financial implications.

ACKNOWLEDGMENTS
Author Contributions:
Study concept and design: S. M. Lippa, R. T. Lange, J. M. Bailie, J. E. Kennedy.
Acquisition of data: R. T. Lange, J. M. Bailie, J. E. Kennedy, L. M. French, T. A. Brickell.
Analysis and interpretation of data: S. M. Lippa, R. T. Lange, J. M. Bailie.
Drafting of manuscript: S. M. Lippa.
Critical revision of manuscript for important intellectual content: S. M. Lippa, R. T. Lange, J. M. Bailie, J. E. Kennedy, L. M. French, T. A. Brickell.
Statistical analysis: S. M. Lippa.
Financial Disclosures: The authors have declared that no competing interests exist.
Funding/Support: This material was unfunded at the time of manuscript preparation.
Institutional Review: The protocols under which these data were collected were approved by the Institutional Review Boards of Walter Reed National Military Medical Center, Bethesda, Maryland; San Antonio Military Medical Center, San Antonio, Texas; Naval Medical Center, San Diego, San Diego, California; Naval Hospital Camp Pendleton, Oceanside, California; Wilford Hall Ambulatory Surgical Center, San Antonio, Texas; and Robert E. Bush Naval Hospital, Twentynine Palms, California. This study was completed in accordance with the guidelines of the Declaration of Helsinki. For Naval Medical Center, San Diego, it is further noted that the voluntary informed consent of the subjects used in this research was obtained as required by SECNAVINST 3900.39D and the investigator(s) adhered to the policies regarding the protection of human subjects as prescribed by Code of Federal Regulations Title 45, Volume 1, Part 46; Title 32, chapter 1, Part 219; and Title 21, chapter 1, Part 50 (Protection of Human Subjects).
Participant Follow-up: The authors do not plan to inform the participants of the publication of this study.
Disclaimer: The views expressed in this article are those of the authors and do not reflect the official policy of the Department of the Navy, Department of the Army, Department of Defense, or U.S. Government.
REFERENCES
1.
Hoge CW, McGurk D, Thomas JL, Cox AL, Engel CC, Castro CA. Mild traumatic brain injury in U.S. Soldiers returning from Iraq. N Engl J Med. 2008;358(5):453–63.
[PMID:18234750]
http://dx.doi.org/10.1056/NEJMoa072972
2.
Terrio H, Brenner LA, Ivins BJ, Cho JM, Helmick K, Schwab K, Scally K, Bretthauer R, Warden D. Traumatic brain injury screening: Preliminary findings in a US Army Brigade Combat Team. J Head Trauma Rehabil. 2009; 24(1):14–23. [PMID:19158592]
http://dx.doi.org/10.1097/HTR.0b013e31819581d8
3.
Brenner LA, Vanderploeg RD, Terrio H. Assessment and diagnosis of mild traumatic brain injury, posttraumatic stress disorder, and other polytrauma conditions: Burden of adversity hypothesis. Rehabil Psychol. 2009;54(3):239–46.
[PMID:19702422]
http://dx.doi.org/10.1037/a0016908
4.
Iverson GL, Langlois JA, McCrea MA, Kelly JP. Challenges associated with post-deployment screening for mild traumatic brain injury in military personnel. Clin Neuropsychol. 2009;23(8):1299–1314. [PMID:19882473]
http://dx.doi.org/10.1080/13854040903153902
5.
Lange RT, Brickell TA, Kennedy JE, Bailie JM, Sills C, Asmussen S, Amador R, Dilay A, Ivins B, French LM. Factors influencing postconcussion and posttraumatic stress symptom reporting following military-related concurrent polytrauma and traumatic brain injury. Arch Clin Neuropsychol. 2014;29(4):329–47. [PMID:24723461]
http://dx.doi.org/10.1093/arclin/acu013
6.
Kaut KP, DePompei R, Kerr J, Congeni J. Reports of head injury and symptom knowledge among college athletes: Implications for assessment and educational intervention. Clin J Sport Med. 2003;13(4):213–21. [PMID:12855923]
http://dx.doi.org/10.1097/00042752-200307000-00004
7.
Llewellyn T, Burdette GT, Joyner AB, Buckley TA. Concussion reporting rates at the conclusion of an intercollegiate athletic career. Clin J Sport Med. 2014;24(1):76–79.
[PMID:24157468]
http://dx.doi.org/10.1097/01.jsm.0000432853.77520.3d
8.
McCrea M, Hammeke T, Olsen G, Leo P, Guskiewicz K. Unreported concussion in high school football players: Implications for prevention. Clin J Sport Med. 2004;14(1): 13–17. [PMID:14712161]
http://dx.doi.org/10.1097/00042752-200401000-00003
9.
Meier TB, Brummel BJ, Singh R, Nerio CJ, Polanski DW, Bellgowan PS. The underreporting of self-reported symptoms following sports-related concussion. J Sci Med Sport. 2015;18(5):507–11. [PMID:25150463]
http://dx.doi.org/10.1016/j.jsams.2014.07.008
10.
Williamson IJ, Goodman D. Converging evidence for the under-reporting of concussions in youth ice hockey. Br J Sports Med. 2006;40(2):128–32. [PMID:16431999]
11.
Binder LM, Rohling ML. Money matters: A meta-analytic review of the effects of financial incentives on recovery after closed-head injury. Am J Psychiatry. 1996;153(1):7–10.
[PMID:8540596]
http://dx.doi.org/10.1176/ajp.153.1.7
12.
Cook JB. The post-concussional syndrome and factors influencing recovery after minor head injury admitted to hospital. Scand J Rehabil Med. 1972;4(1):27–30.
[PMID:4283655]
14.
Paniak C, Reynolds S, Toller-Lobe G, Melnyk A, Nagy J, Schmidt D. A longitudinal study of the relationship between financial compensation and symptoms after treated mild traumatic brain injury. J Clin Exp Neuropsychol. 2002;24(2):187–93. [PMID:11992201]
http://dx.doi.org/10.1076/jcen.24.2.187.999
15.
Reynolds S, Paniak C, Toller-Lobe G, Nagy J. A longitudinal study of compensation-seeking and return to work in a treated mild traumatic brain injury sample. J Head Trauma Rehabil. 2003;18(2):139–47. [PMID:12802223]
http://dx.doi.org/10.1097/00001199-200303000-00005
16.
Lange RT, Pancholi S, Bhagwat A, Anderson-Barnes V, French LM. Influence of poor effort on neuropsychological test performance in U.S. military personnel following mild traumatic brain injury. J Clin Exp Neuropsychol. 2012; 34(5):453–66. [PMID:22273465]
http://dx.doi.org/10.1080/13803395.2011.648175
17.
Armistead-Jehle P. Symptom validity test performance in U.S. veterans referred for evaluation of mild TBI. Appl Neuropsychol. 2010;17(1):52–59. [PMID:20146122]
http://dx.doi.org/10.1080/09084280903526182
18.
Nelson NW, Hoelzle JB, McGuire KA, Ferrier-Auerbach AG, Charlesworth MJ, Sponheim SR. Evaluation context impacts neuropsychological performance of OEF/OIF veterans with reported combat-related concussion. Arch Clin Neuropsychol. 2010;25(8):713–23. [PMID:20923860]
http://dx.doi.org/10.1093/arclin/acq075
19.
Whitney KA, Shepard PH, Williams AL, Davis JJ, Adams KM. The Medical Symptom Validity Test in the evaluation of Operation Iraqi Freedom/Operation Enduring Freedom soldiers: A preliminary study. Arch Clin Neuropsychol. 2009;24(2):145–52. [PMID:19395348]
http://dx.doi.org/10.1093/arclin/acp020
20.
McCormick CL, Yoash-Gantz RE, McDonald SD, Campbell TC, Tupler LA. Performance on the Green Word Memory Test following Operation Enduring Freedom/Operation Iraqi Freedom-era military service: Test failure is related to evaluation context. Arch Clin Neuropsychol. 2013; 28(8):808–23. [PMID:23877970]
http://dx.doi.org/10.1093/arclin/act050
21.
Armistead-Jehle P, Buican B. Evaluation context and Symptom Validity Test performances in a U.S. military sample. Arch Clin Neuropsychol. 2012;27(8):828–39.
[PMID:23047953]
http://dx.doi.org/10.1093/arclin/acs086
22.
Whitney KA, Shepard PH, Mariner J, Mossbarger B, Herman SM. Validity of the Wechsler Test of Adult Reading (WTAR): Effort considered in a clinical sample of U.S. military veterans. Appl Neuropsychol. 2010;17(3):196–204.
[PMID:20799111]
http://dx.doi.org/10.1080/09084282.2010.499787
24.
Lippa SM, Pastorek NJ, Romesser J, Linck J, Sim AH, Wisdom NM, Miller BI. Ecological validity of performance validity testing. Arch Clin Neuropsychol. 2014; 29(3):236–44. [PMID:24681945]
http://dx.doi.org/10.1093/arclin/acu002
25.
Lange RT, Brickell TA, Lippa SM, French LM. Clinical utility of the Neurobehavioral Symptom Inventory validity scales to screen for symptom exaggeration following traumatic brain injury. J Clin Exp Neuropsychol. 2015; 37(8):853–62. [PMID:26245293]
http://dx.doi.org/10.1080/13803395.2015.1064864
26.
Vanderploeg RD, Cooper DB, Belanger HG, Donnell AJ, Kennedy JE, Hopewell CA, Scott SG. Screening for postdeployment conditions: Development and cross-validation of an embedded validity scale in the Neurobehavioral Symptom Inventory. J Head Trauma Rehabil. 2014;29(1): 1–10. [PMID:23474880]
http://dx.doi.org/10.1097/HTR.0b013e318281966e
27.
Lange RT, Brickell TA, French LM. Examination of the Mild Brain Injury Atypical Symptom Scale and the Validity-10 scale to detect symptom exaggeration in US military service members. J Clin Exp Neuropsychol. 2015;37(3):325–37.
[PMID:25849968]
http://dx.doi.org/10.1080/13803395.2015.1013021
28.
Hou R, Moss-Morris R, Peveler R, Mogg K, Bradley BP, Belli A. When a minor head injury results in enduring symptoms: A prospective investigation of risk factors for postconcussional syndrome after mild traumatic brain injury. J Neurol Neurosurg Psychiatry. 2012;83(2):217–23.
[PMID:22028384]
http://dx.doi.org/10.1136/jnnp-2011-300767
29.
Snell DL, Siegert RJ, Hay-Smith EJ, Surgenor LJ. Associations between illness perceptions, coping styles and outcome after mild traumatic brain injury: Preliminary results from a cohort study. Brain Inj. 2011;25(11):1126–38.
[PMID:21870903]
http://dx.doi.org/10.3109/02699052.2011.607786
30.
Whittaker R, Kemp S, House A. Illness perceptions and outcome in mild head injury: A longitudinal study. J Neurol Neurosurg Psychiatry. 2007;78(6):644–46.
[PMID:17507448]
http://dx.doi.org/10.1136/jnnp.2006.101105
31.
Butcher JN, Dahlstrom WG, Graham JR, Tellegan A, Kaemmer B. Manual for administration and scoring of the MMPI-2. Minneapolis (MN): University of Minnesota Press; 1989.
32.
Morey LC. The Personality Assessment Inventory: Professional manual. Lutz (FL): Psychological Assessment Resources; 1990.
33.
Widows MR, Smith GP. Structured Inventory of Malingered Symptomatology: Professional manual. Odessa (FL): Psychological Assessment Resources; 2005.
34.
Rogers R, Kropp PR, Bagby RM, Dickens SE. Faking specific disorders: A study of the Structured Interview of Reported Symptoms (SIRS). J Clin Psychol. 1992;48(5): 643–48. [PMID:1401150]
http://dx.doi.org/10.1002/1097-4679(199209)48: 5<643::AID-JCLP2270480511>3.0.CO;2-2
35.
Cicerone KD, Karlmar K. Persistent postconcussion syndrome: The structure of subjective complaints after mild traumatic brain injury. J Head Trauma Rehabil. 1995;10:1–17.
http://dx.doi.org/10.1097/00001199-199510030-00002
36.
Belanger HG, Uomoto JM, Vanderploeg RD. The Veterans Health Administration's (VHA's) Polytrauma System of Care for mild traumatic brain injury: Costs, benefits, and controversies. J Head Trauma Rehabil. 2009;24(1):4–13.
[PMID:19158591]
http://dx.doi.org/10.1097/HTR.0b013e3181957032
37.
Kaplan DA. Information paper: Neurobehavioral Symptom Inventory (NSI): Recommendations for scoring and serial administration for concussion health care outcomes standardization [Internet]. Silver Spring (MD): Defense and Veterans Brain Injury Center, Department of the Army; 2014 May 15. Available from: http://dvbic.dcoe.mil/sites/default/files/DVBIC_-_NSI_Information_Paper_Final.pdf
38.
Ben-Porath YS, Tellegen A. MMPI-2-RF (Minnesota Multiphasic Personality Inventory-2 Restructured Form): Manual for administration, scoring, and interpretation. Minneapolis (MN): University of Minnesota Press; 2008.
39.
Bazarian JJ, Blyth B, Mookerjee S, He H, McDermott MP. Sex differences in outcome after mild traumatic brain injury. J Neurotrauma. 2010;27(3):527–39.
[PMID:19938945]
http://dx.doi.org/10.1089/neu.2009.1068
40.
Fear NT, Jones E, Groom M, Greenberg N, Hull L, Hodgetts TJ, Wessely S. Symptoms of post-concussional syndrome are non-specifically related to mild traumatic brain injury in UK Armed Forces personnel on return from deployment in Iraq: An analysis of self-reported data. Psychol Med. 2009;39(8):1379–87. [PMID:18945380]
http://dx.doi.org/10.1017/S0033291708004595
41.
McCauley SR, Boake C, Levin HS, Contant CF, Song JX. Postconcussional disorder following mild to moderate traumatic brain injury: Anxiety, depression, and social support as risk factors and comorbidities. J Clin Exp Neuropsychol. 2001;23(6):792–808. [PMID:11910545]
http://dx.doi.org/10.1076/jcen.23.6.792.1016
42.
Iverson GL, Lange RT. Moderate and severe traumatic brain injury. In: Schoenberg MR, Scott JG, editors. The little black book of neuropsychology. New York (NY): Springer; 2011. p. 663–96.
43.
Report of the Quality Standards Subcommittee. Practice parameter: The management of concussion in sports (summary statement). Neurology. 1997;48(3):581–85.
[PMID:9065530]
http://dx.doi.org/10.1212/WNL.48.3.581
44.
McCrea MA. Influence of acute injury characteristics on recovery. Mild traumatic brain injury and postconcussion syndrome. New York (NY): Oxford University Press; 2008. p. 123–24.
45.
Wilde EA, Whiteneck GG, Bogner J, Bushnik T, Cifu DX, Dikmen S, French L, Giacino JT, Hart T, Malec JF, Millis SR, Novack TA, Sherer M, Tulsky DS, Vanderploeg RD, von Steinbuechel N. Recommendations for the use of common outcome measures in traumatic brain injury research. Arch Phys Med Rehabil. 2010;91(11):1650–60.e17.
[PMID:21044708]
http://dx.doi.org/10.1016/j.apmr.2010.06.033
47.
Baker SP, O'Neill B, Haddon W Jr, Long WB. The injury severity score: A method for describing patients with multiple injuries and evaluating emergency care. J Trauma. 1974;14(3):187–96. [PMID:4814394]
http://dx.doi.org/10.1097/00005373-197403000-00001
48.
Stevenson M, Segui-Gomez M, Lescohier I, Di Scala C, McDonald-Smith G. An overview of the injury severity score and the new injury severity score. Inj Prev. 2001; 7(1):10–13. [PMID:11289527]
http://dx.doi.org/10.1136/ip.7.1.10
49.
Scholten JD, Sayer NA, Vanderploeg RD, Bidelspach DE, Cifu DX. Analysis of US Veterans Health Administration comprehensive evaluations for traumatic brain injury in Operation Enduring Freedom and Operation Iraqi Freedom Veterans. Brain Inj. 2012;26(10):1177–84.
[PMID:22646489]
http://dx.doi.org/10.3109/02699052.2012.661914
50.
Lew HL, Pogoda TK, Baker E, Stolzmann KL, Meterko M, Cifu DX, Amara J, Hendricks AM. Prevalence of dual sensory impairment and its association with traumatic brain injury and blast exposure in OEF/OIF veterans. J Head Trauma Rehabil. 2011;26(6):489–96. [PMID:21386715]
http://dx.doi.org/10.1097/HTR.0b013e318204e54b
51.
Combs HL, Berry DT, Pape T, Babcock-Parziale J, Smith B, Schleenbaker R, Shandera-Ochsner A, Harp JP, High WM Jr. The effects of mild traumatic brain injury, post-traumatic stress disorder, and combined mild traumatic brain injury/post-traumatic stress disorder on returning veterans. J Neurotrauma. 2015;32(13):956–66.
[PMID:25350012]
http://dx.doi.org/10.1089/neu.2014.3585
52.
Cassidy JD, Cancelliere C, Carroll LJ, Côté P, Hincapié CA, Holm LW, Hartvigsen J, Donovan J, Nygren-de Boussard C, Kristman VL, Borg J. Systematic review of self-reported prognosis in adults after mild traumatic brain injury: Results of the International Collaboration on Mild Traumatic Brain Injury Prognosis. Arch Phys Med Rehabil. 2014;95(3 Suppl):S132–51. [PMID:24581902]
http://dx.doi.org/10.1016/j.apmr.2013.08.299
53.
Management of Concussion/mTBI Working Group. VA/DoD clinical practice guideline for management of concussion/mild traumatic brain injury. J Rehabil Res Dev. 2009; 46(6):CP1–68. [PMID:20108447]
54.
French LM, Lange RT, Iverson GL, Ivins B, Marshall K, Schwab K. Influence of bodily injuries on symptom reporting following uncomplicated mild traumatic brain injury in US military service members. J Head Trauma Rehabil. 2012;27(1):63–74. [PMID:21897286]
http://dx.doi.org/10.1097/HTR.0b013e3182248344
This article and any supplementary material should be cited as follows:
Lippa SM, Lange RT, Bailie JM, Kennedy JE, Brickell TA, French LM. Utility of the Validity-10 scale across the recovery trajectory following traumatic brain injury. J Rehabil Res Dev. 2016;53(3):379–90.
http://dx.doi.org/10.1682/JRRD.2015.01.0009
iThenticateCrossref

Go to TOP

Last Reviewed or Updated  Tuesday, May 10, 2016 2:49 PM

Valid HTML 4.01 Transitional