Journal of Rehabilitation Research & Development (JRRD)

Quick Links

  • Health Programs
  • Protect your health
  • Learn more: A-Z Health
Veterans Crisis Line Badge
 

Volume 53 Number 5, 2016
   Pages 571 — 584

Postconcussive symptom overreporting in Iraq/Afghanistan Veterans with mild traumatic brain injury

Sarah M. Jurick, MS;1–2 Elizabeth W. Twamley, PhD;3–4 Laura D. Crocker, PhD;5 Chelsea C. Hays, MS;1–2 Henry J. Orff, PhD;3–4 Shahrokh Golshan, PhD;2,4 Amy J. Jak, PhD3,5*

1San Diego State University/University of California San Diego Joint Doctoral Program in Clinical Psychology, San Diego, CA; 2Veterans Medical Research Foundation, San Diego, CA; 3Center of Excellence for Stress and Mental Health, Department of Veterans Affairs (VA) San Diego Healthcare System, San Diego, CA; 4Department of Psychiatry, University of California San Diego, San Diego, CA; 5Psychology Service, VA San Diego Healthcare System, San Diego, CA

Abstract — A comprehensive evaluation, including the assessment of neurobehavioral symptoms, has been instituted at the Department of Veterans Affairs (VA) healthcare system to address the large number of Operation Iraqi Freedom/Operation Enduring Freedom (OIF/OEF) Veterans returning with mild traumatic brain injuries (mTBIs). The Validity-10 is a measure of symptom overreporting embedded within the Neurobehavioral Symptom Inventory, a component of the comprehensive evaluation that assesses postconcussive symptom severity. The Validity-10 is composed of 10 unlikely/low-frequency items and a validated cutoff score to identify postconcussive symptom overreporting. We examined the items and cutoff used in the initial development and validation study of the Validity-10 through retrospective chart reviews of 331 treatment-seeking Veterans who sustained an mTBI. The Validity-10 exhibited significant relationships with psychiatric variables, VA service connection, and neuropsychological performance validity (all p < 0.01), but nonsignificant relationships with demographic and injury variables (all p > 0.05). Furthermore, the Validity-10 modestly predicted neuropsychological performance validity test failure over and above psychiatric comorbidities and VA service connection. The present study supports the use of the Validity-10 to assess symptom validity in treatment-seeking OIF/OEF Veterans with a history of mTBI.

Key Words: concussion, depression, mild traumatic brain injury, neurobehavioral symptoms, neuropsychology, Operation Enduring Freedom/Operation Iraqi Freedom, performance validity test, postconcussive symptoms, posttraumatic stress disorder, symptom validity test.

Abbreviations:ANOVA = analysis of variance, BDI-II = Beck Depression Inventory-Second Edition, CVLT-II = California Verbal Learning Test-Second Edition, DOD = Department of Defense, IQR = interquartile range, LOC = loss of consciousness, LOW6 = NSI Infrequent Scale, mBIAS = mild Brain Injury Atypical Symptoms scale, MMPI-2 = Minnesota Multiphasic Personality Inventory-Second Edition, mTBI = mild traumatic brain injury, NIM5 = Negative Impression Management Atypical Scale, NSI = Neurobehavioral Symptom Inventory, OIF/OEF = Operation Iraqi Freedom/Operation Enduring Freedom, PAI = Personality Assessment Inventory, PCL-C = PTSD Checklist–Civilian Version, PTA = posttraumatic amnesia, PTSD = posttraumatic stress disorder, PVT = performance validity test, SVT = symptom validity test, TBI = traumatic brain injury, TOMM = Test of Memory Malingering, VA = Department of Veterans Affairs, WRAT-4 = Wide Range Achievement Test-4.
*Address all correspondence to Amy J. Jak, PhD; VA San Diego Healthcare System, 3350 La Jolla Village Dr (151B), San Diego, CA 92161; 858-642-3742; fax: 858-642-6340. Email: ajak@ucsd.edu
http://dx.doi.org/10.1682/JRRD.2015.05.0094
INTRODUCTION

It is estimated that at least 30 percent of servicemembers engaged in active combat in Iraq or Afghanistan for 4 mo or more sustained a traumatic brain injury (TBI) [1]. Over 80 percent of military TBIs are classified as mild (mTBI), defined by an alteration or loss of consciousness (LOC) less than or equal to 30 min, posttraumatic amnesia (PTA) less than or equal to 24 h, and negative or equivocal neuroimaging findings [2–3]. Recovery after mTBI usually occurs within a period of days to weeks [2,4]; however, a sizable minority (18%–30%) of individuals in certain samples continues to report persistent postconcussive complaints, such as headache, light sensitivity, depressed mood, and reduced attention, many months to years after injury [5–7]. Persistent postconcussive symptoms are associated with poor functional outcomes [8–10], such as difficulty returning to work or school. Pain [11], secondary gain [12], and psychiatric comorbidities, such as posttraumatic stress disorder (PTSD) and depression, notably contribute to poor prognoses [13–17]. Various factors specific to Operation Iraqi Freedom/Operation Enduring Freedom (OIF/OEF) Veterans, such as number of injuries sustained, high rates of psychiatric and medical comorbidities, potential for secondary gain, and iatrogenic consequences, have likely affected the frequency of neurobehavioral complaints in this population [18–19].

In response to the large number of Veterans with mTBIs returning from Iraq and Afghanistan, a military TBI task force was convened to highlight the need to assess both neuropsychological performance and symptom validity and address the unique complications in interpreting neuropsychological and symptom assessments in this population [20]. The validity of performance on neuropsychological tests is assessed using performance validity tests (PVTs): either atypical performance on commonly used neuropsychological tests or impaired performance on stand-alone tests that severe neurological patients pass at near-perfect levels [21]. Conversely, symptom validity tests (SVTs), such as the validity scales within the Minnesota Multiphasic Personality Inventory-Second Edition. (MMPI-2) [22], measure the accuracy of self-reported symptom complaints. Thus, the development and validation of SVTs and PVTs for mTBI populations is warranted. Vanderploeg and colleagues [23] developed an embedded SVT in the Neurobehavioral Symptom Inventory (NSI) [24] to identify likely overreporting of postconcussive symptoms. The NSI is a part of the Comprehensive TBI Evaluation that is administered to every Veteran in Department of Veterans Affairs (VA) clinical settings who screen positive for TBI and opt for further evaluation of their symptoms. Vanderploeg et al. [23] identified items on the NSI that were most highly correlated with the mild Brain Injury Atypical Symptoms scale (mBIAS) [25] and the Negative Impression Management scale on the Personality Assessment Inventory (PAI) [26] to create the Negative Impression Management Atypical Scale (NIM5), which contained highly unlikely or bizarre symptoms. They then identified the least frequently endorsed NSI items across various mild to severe TBI clinical and nonclinical samples to create the NSI Infrequent Scale (LOW6). The combination of the NIM5 and LOW6 (with one overlapping item) comprises the 10 items of the Validity-10. Logistic regression using a clinical cross-validation sample and a cutoff score of 8 or more on the mBIAS [25] revealed that a cutoff score of greater than 22 on the Validity-10 was most effective at identifying symptom overreporting [23]. Since its publication, two studies have identified different cutoff scores for the Validity-10 in military servicemembers with mild [27] and all severities of TBI [28].

Rates of failure on neuropsychological PVTs range from 17 to 58 percent among individuals with a history of mild to moderate TBI [10,13,29–30], despite the finding that no neuropsychological deficits are expected by 3 mo after mTBI in prospective and unselected samples [5]. Elevations on PVTs and SVTs are related to compensation-seeking and higher levels of psychiatric comorbidities, such as PTSD and depression, but not demographic variables or injury characteristics in those with a history of mTBI [27,29,31–34]. The benefits of using an embedded SVT such as the Validity-10 are twofold. Firstly, it is a useful screener to alert providers to cautiously interpret self-report of postconcussive symptoms. Secondly, if an embedded SVT is able to predict PVT failure, it may save providers from administering a time-consuming and costly neuropsychological assessment that is uninterpretable. In a Veteran sample with various neurological conditions and mild to severe TBI, as well as in a civilian sample of mTBI only, various SVTs (i.e., validity scales of the MMPI-2) significantly predicted PVT failure [34–35]. However, in Veterans with a history of mTBI only, other SVTs (various subscales of the PAI) did not significantly predict PVT failure, whereas VA service connection and a previous diagnosis of depression did [29]. Thus, there is evidence that PVT failure corresponds to SVTs, but assessing symptom and performance validity separately is still essential as they are not completely overlapping constructs and failure of one type of measure does not necessarily invalidate the other set of measures [36]. The discrepant findings regarding the relationship between PVTs and SVTs may be a result of the use of heterogeneous samples (e.g., all severities of TBI, various neurological disorders), the use of different measures across studies (e.g., PAI vs MMPI-2, Medical Symptom Validity Test [37] vs Test of Memory Malingering (TOMM) [38]), and the use of different methods to assess response bias (e.g., atypical vs infrequent symptom reporting). To elucidate the role of SVT performance in predicting PVT failure, which is especially relevant in individuals with pre- and postinjury psychiatric comorbidities [39–44], the study of embedded SVTs in disorder-specific inventories with homogenous samples is warranted [45].

The Validity-10 has not yet been used, nor have the initial results been replicated outside of the initial development and validation sample in a homogenous sample of Veterans with mTBI [23]. One study replicated the initial development and validation study in servicemembers with mTBI but identified a much lower cutoff score [27]. Additionally, no studies to date have examined the Validity-10's relationship to other variables of interest in this population such as demographics, injury variables, psychiatric characteristics, VA service connection, or PVT performance. This study examined the items and cutoff score used in the initial development and validation study [5] in a homogenous sample of Veterans with a history of mTBI. Specifically, we aimed to compare the low–frequency items (LOW6) and the percentage of Veterans scoring above (SVT-fail group) versus below (SVT-pass group) the cutoff score of 22 on the Validity-10 to the findings of Vanderploeg et al. [23]. We also extended those findings by comparing the SVT-fail and SVT-pass groups on variables of interest such as demographics, injury characteristics (LOC, PTA, number of lifetime TBIs), psychiatric variables, VA service connection, and PVT performance. We hypothesized that (1) NSI items endorsed with the lowest frequency will be consistent with those in the LOW6; (2) the percentage of those in the SVT-fail group will be comparable to the rate of overreporting identified in Vanderploeg et al.'s [23] clinical samples; and (3) the Validity-10 score will exhibit significant relationships with self-reported depression and PTSD symptoms, VA service connection, and PVT performance, consistent with previous studies of military servicemembers with a history of TBI [46]. The second aim was to elucidate the relationship of SVT failure to PVT failure by investigating whether Validity-10 would significantly predict PVT failure over and above depression, PTSD, and VA service connection, which have been associated with PVT failure in previous studies. Consistent with the rationale employed by Vanderploeg and colleagues [23] supporting the use of both atypical and infrequent reporting to detect response bias, we hypothesized that the Validity-10 would significantly contribute to the model of PVT failure over and above psychiatric comorbidities and VA service connection. A final exploratory aim was to replicate all analyses using a lower cut score on the Validity-10 identified by Lange and colleagues [27] to determine how this lower cutoff would relate to demographics, injury variables, psychiatric symptoms, VA service connection, and PVT failure.

METHODS
Participants

Between February 2003 and August 2013, a total of 586 Veterans within the VA San Diego Healthcare System were consecutively referred for clinical comprehensive neuropsychological testing to assess their cognitive functioning because of persistent cognitive complaints and a history of TBI. Records were included if (1) all items were answered on the NSI questionnaire, (2) Veteran was OIF/OEF era, and (3) Veteran had a history of mTBI as defined according to VA/Department of Defense (DOD) criteria described previously, except for Glasgow Coma Scale scores because they were not available [3]. TBI diagnoses were based on clinical diagnoses, and information about the presence and duration of LOC and PTA was collected by clinician neuropsychologists, medical doctors, or advanced doctoral trainees through a semistructured interview. This information was then coded by experienced research assistants who were trained to determine TBI severity based on the VA/DOD guidelines. In the cases where exact LOC or PTA minutes were not available but range estimates were (e.g., LOC 5–15 min), the cases were included if the range fell within LOC less than or equal to 30 min and PTA less than or equal to 24 h. Presence of clinical diagnoses of mental health disorders such as PTSD and depression were also included when available. Exclusion criteria included history of previous moderate to severe brain injury, history of learning disability, Wide Range Achievement Test-4 (WRAT-4) Reading standard score [47] less than 75, other neurological disorder, or current substance dependence or psychotic disorder.

Procedure

Retrospective medical chart reviews were completed by experienced raters who were trained to use a specific protocol to retrieve and enter data. Psychiatric and neurocognitive diagnoses, psychiatric self-report questionnaires, and neuropsychological test results were compiled for all participants using a standard data collection form developed for the present study. Specifically, past or current psychiatric diagnoses were coded as "present" if there was any indication of that diagnosis in the neuropsychological report, problems list in the medical chart, or general psychiatry clinical notes. VA service connection was extracted from participants' medical charts and used as a metric for compensation-seeking as was done in a previous study [29]. VA service connection is a percentage from 0 to 100 in which higher ratings represent greater interference of functioning from a disorder(s) that was incurred or aggravated during Active Duty; service connection is used to establish any monetary benefit paid to Veterans for this interference of functioning. Veterans in the present study were service connected for many different disorders, including but not limited to TBI. All Veterans underwent a comprehensive clinical assessment that included tests in the domains of attention, memory, executive function, and processing speed.

Measures
Premorbid Intelligence

The WRAT-4 Reading test [47] is a 70-item test of oral reading that is used to measure premorbid intelligence. Higher standard scores indicate higher academic achievement in reading.

Postconcussive Symptoms and Mental Health Variables

The NSI [24] is a 22-item self-report measure that rates difficulty with cognitive-, physical-, and emotion-related postconcussive symptoms on a Likert-type scale. It takes approximately 10 min to complete. A total score of 34 or greater in a clinical sample is considered clinically elevated [44]. The Beck Depression Inventory–Second Edition (BDI-II) is a 21-item self-report measure, with greater scores on a Likert-type scale indicating greater depression severity [48]. The VA requires the PTSD Checklist–Civilian Version (PCL-C) to be administered to every Veteran receiving PTSD treatment. It comprises 17 items on a Likert-type scale; greater scores indicate greater PTSD severity [49].

Validity Indicators

The Validity-10 is composed of 10 items embedded in the NSI. These 10 items are bolded in Table 1. Two scores were calculated for the NSI: the sum of all 22 items from the full NSI and the sum of Validity-10 items. Those scoring above 22 on the Validity-10 were identified as the SVT-fail group while those scoring 22 or below were in the SVT-pass group. Lange and colleagues identified a cutoff score of 13 or above for "symptom exaggeration"? in servicemembers with a history of mTBI [27]. Thus, a cutoff score of 13 was used for the exploratory aim. PVTs administered included the California Verbal Learning Test-Second Edition (CVLT-II) [50] Forced-Choice Trial and the TOMM [38] Trial 2 and Retention Trial. The CVLT-II Forced-Choice test is a 16-item embedded forced-choice memory test within a larger verbal list learning and memory test, and the TOMM is a 50-item picture memory test composed of three trials. Both have been successfully used to predict invalid performance in individuals with a history of TBI, and failure of even one of these PVTs warrants consideration of poor performance validity, particularly in individuals with a history of mTBI [51–52]. Aligned with standard scoring for the CVLT-II and the TOMM, participants were classified as failing PVTs if their performance fell below 45 on TOMM Trial 2 or the Retention Trial or below 15 on CVLT-II Forced Choice.


Table 1. 
Data Analyses

Data available varied for each participant depending on the clinical assessment administered. However, the majority of the participants received all of the measures described (Table 2). To address the first aim of conducting a descriptive analysis of the findings from the initial development and validation study [23] in a homogenous mTBI sample, the means and frequencies of the NSI items and the percentage of Veterans in the SVT-fail group were evaluated in relation to the findings from Vanderploeg and colleagues [23]. This was accomplished by determining if the low-frequency items in the current study matched the LOW6 and if the percentage of Veterans in the SVT-fail group was within the range of 7 to 15 percent, as observed in Vanderploeg and colleagues' clinical samples [23]. Additionally, chi-square tests or one-way analyses of variance (ANOVAs) were conducted between the SVT-pass and SVT-fail group to determine group differences on demographic variables, injury characteristics, VA service connection, psychiatric questionnaires, and PVT failure. All variables were checked for normality and examined for outliers. Nonparametric Mann-Whitney U tests were used when nonnormal distributions were unable to be corrected. Type I error was controlled at the Bonferroni corrected level of p < 0.01 to account for the analyses including demographic variables, injury characteristics, psychiatric questionnaires, VA service connection, and PVT failure. To address potential issues related to unequal sample sizes of the two groups, analyses conducted with the Validity-10 groups were replicated using one-way ANOVAs comparing a randomly selected 15 percent of the SVT-pass group with the SVT-fail group to lend further confidence to the results observed in the initial analyses. Hierarchical logistic regressions were then employed to address the second aim of identifying whether the Validity-10 significantly predicts PVT failure over and above psychiatric variables and VA service connection. To address the exploratory aim of determining whether the cutoff score found by Lange et al. [27] changed the results, all analyses were directly replicated using the cutoff score of greater than or equal to 13 on the Validity-10.


Table 2. 
Sample characteristics; N = 331 except as otherwise noted.
Race (n = 321)
No. Lifetime TBIs* (n = 330)
*Denotes median and interquartile range (IQR) statistics reported.
Service connection is percentage from 0–100; higher ratings represent greater interference of functioning from a disorder(s) that was incurred or aggravated during active military service.
BDI-II = Beck Depression Inventory-Second Edition, CVLT-II = California Verbal Learning Test-Second Edition, LOC = loss of consciousness, NSI = Neurobehavioral Symptom Inventory, PCL-C = PTSD Checklist–Civilian Version, PTA = posttraumatic amnesia, PTSD = posttraumatic stress disorder, PVT = performance validity test, SD = standard deviation, SVT = symptom validity test, TBI = traumatic brain injury, TOMM = Test of Memory Malingering, VA = Department of Veterans Affairs, WRAT-4 = Wide Range Achievement Test-4.
RESULTS

A total of 255 Veterans were excluded from the database based on inclusion and exclusion criteria, and the remaining 331 Veterans were included in the present study (Table 2 displays sample characteristics). The majority of the sample was male (90%), Caucasian (62%), had comorbid PTSD (74%), with a median age of 33 yr and interquartile range (IQR) of 8.7, median education of 13 yr (IQR = 1.8), median of 2 lifetime TBIs (IQR = 1.9), median of 0.05 min of LOC (IQR = 1.5), and median of 0 min of PTA (IQR = 1.0). A total of 93 Veterans (28%) were classified as failing PVTs (Table 2). All Veterans were assessed at least 3 mo after their TBI(s). Of the present sample, 8.5 percent were in the SVT-fail group, which was based on the original Validity-10 cutoff of 22 identified by Vanderploeg et al. [23]. Consistent with previous findings, analysis of the individual NSI item frequencies and severities revealed the items assessing dizziness, balance, coordination, nausea, vision, and taste/smell had the lowest frequencies (Table 1).

The SVT-fail and SVT-pass groups did not significantly differ on age, years of education, WRAT-4 Reading standard score, number of lifetime mTBIs, sex, ethnicity, minutes of LOC or PTA, or mechanism of the worst injury (all p > 0.05). Rates of PTSD and depression diagnoses were not significantly different between the two groups. However, the SVT-fail group had significantly higher scores on the BDI-II and PCL-C and higher VA service connection (all p < 0.001; Table 3). The Validity-10 was significantly associated with PVT failure. Specifically, 57 percent of the SVT-fail group failed PVTs compared with only 25 percent of SVT-pass group (χ2 = 12.8, p = 0.001). The odds of PVT failure were 4.8 times higher for the SVT-fail group. The effect size was small to medium (Table 3).


Table 3.
U/F or χ2
p-Value
d or φ
Note: For percentage data, χ2 and φ statistics reported.
*Those scoring above 22 on the Validity-10.
Effect size magnitude: 0.2 = small, 0.5 = medium, 0.8 = large [53].
Denotes significance at the family-wise Bonferroni-corrected p-value; p < 0.01.
Denotes that a one-way analysis of variance was conducted because the data were normally distributed.
BDI-II = Beck Depression Inventory-Second Edition, F = ratio of variance in one-way analysis of variance, IQR = interquartile range, LOC = loss of consciousness, mTBI = mild traumatic brain injury (a percentage from 0–100 in which higher ratings represent greater interference of functioning from a disorder[s] that was incurred or aggravated during active military service), PCL-C = PTSD Checklist–Civilian Version, PTA = posttraumatic amnesia, PTSD = posttraumatic stress disorder, SD = standard deviation, SVT = symptom validity test, U = Mann-Whitney U test score, VA = Department of Veterans Affairs, WRAT-4 = Wide Range Achievement Test-4.

These results were successfully replicated using a random sample of 15 percent of the SVT-pass group, which further supported that the large difference in sample size between the groups was not responsible for the outcome of the analyses.

In the first hierarchical logistic regression predicting PVT failure, depression diagnosis, PTSD diagnosis, and VA service connection were entered into the first block and the Validity-10 cutoff score was entered into the second block. Depression and PTSD diagnosis variables were used in the logistic regression instead of BDI-II and PCL-C because the sample would have been significantly reduced as a result of missing data. The first block in the model was significant, (χ2(3) = 30.41, p < 0.001) and predicted group membership with 72.5 percent correct classification. The presence of a depression diagnosis and having higher VA service connection emerged as significant predictors of failing one or more PVTs. Adding the Validity-10 in the second block was also significant (χ2(1) = 6.15, p = 0.01) and increased the overall model to a 74.0 percent correct classification rate. The same logistic regression was run without VA service connection in the first block and the overall results did not change. Table 4 displays the predictor variables that were entered in each block for the hierarchical regressions and the corresponding test statistics.


Table 4. 
p-Value
*p < 0.05.
VA service connection is a percentage from 0 to 100 in which higher ratings represent greater interference of functioning from a disorder(s) that was incurred or aggravated during active military service.

As an exploratory aim, all analyses were replicated using the Validity-10 cutoff of 13 found by Lange and colleagues [27] using military servicemembers with a history of mTBI. Using this cutoff, 43.8 percent of the sample was categorized as failing the Validity-10. The remaining results were largely unchanged by using the lower cutoff, with the exception of the percentage of Veterans with a depression or PTSD diagnosis. Using the lower cutoff of 13, diagnoses of depression (χ2 = 6.5, p = 0.007) and PTSD (χ2 = 22.5, p < 0.001) were significantly higher in the groups classified as failing the Validity-10. The results of the logistic regression did not change.

DISCUSSION

The present study extends prior work with the NSI Validity-10 [23,27] to a novel, treatment-seeking Veteran population with a history of mTBI. Analysis of the frequency and level of endorsement of individual items on the NSI in the current sample revealed exactly the same six items from the initial development and validation study to be the least frequently reported, indicating that the LOW6 scale has identical items regardless of the sample from which it was drawn. The percentage of Veterans in the SVT-fail group in the current sample was 8.5 percent, which fell into the range of that observed in the clinical samples from the original development and validation study. The replication of the LOW6 scale and the percentage of Veterans scoring above the cutoff of 22 on the Validity-10 lend support to the Validity-10's appropriateness for measuring symptom validity in this population. There were no significant group differences in age, education, ethnicity, or injury characteristics between the SVT-pass and SVT-fail groups; however, the SVT-fail group had higher VA service connection percentages, greater psychiatric symptom complaints, and higher rates of PVT failure. In a multivariate context, the Validity-10 predicted PVT failure after accounting for depression, PTSD, and VA service connection, providing support for the utility of the Validity-10 in predicting performance validity over and above psychiatric comorbidities and VA service connection. Using the cutoff score of 13 identified by Lange and colleagues [27], a much larger percentage of the sample (43.8%) was considered failing the Validity-10; however, the other analyses remained largely unchanged, with the exception of those scoring above the cutoff being more likely to have PTSD and depression diagnoses.

The large discrepancy between the percentages of Veterans failing the Validity-10 using the cut score of 13 (43.8%) versus 22 (8.5%) is notable. Although the percentage of Veterans failing the Validity-10 using the higher Vanderploeg et al. [23] cutoff is consistent with what was observed in the samples from the initial development and validation study, there is some evidence to suggest that the cutoff identified by Lange et al. [27] is more appropriate. First, the only other study that reported the percentage of Veterans with a history of mTBI who failed an SVT indicated a failure rate of 58 percent [29]. This suggests that the true base rate of symptom validity failure in this particular population may be closer to the 43.8 percent SVT failure rate found using a cutoff of 13 versus the 8.5 percent SVT failure rate found using a cutoff of 22. Additionally, PVT failure in the present sample was 28 percent. PVT and SVT failure are not completely overlapping but are correlated, which provides support for using the lower Validity-10 cut score. Finally, because the Validity-10 is intended to be used as a screening measure, identifying a higher percentage of individuals for further assessment and cautious interpretation of the current assessment is likely preferred over missing Veterans. However, the presence of significantly more Veterans with PTSD and depression diagnoses in the Validity-10 fail group using the lower cutoff may indicate a greater influence of these mental health disorders. Further research is warranted using a gold-standard measure of symptom validity in Veterans with a history of mTBI to determine which cutoff is more appropriate.

Veterans in both the SVT-pass and SVT-fail groups scored in the clinically elevated range on the BDI-II and the PCL-C on average. Yet the large effect sizes and significant and positive relationships between the Validity-10 and psychiatric questionnaires, in the absence of SVT group differences in demographic or injury variables, suggest that the degree of psychiatric symptoms still contributes significantly to response bias. Consistent with prior studies [29,54], VA service connection may also be contributing to response bias given the small to medium effect size of the relationship between VA service connection and the Validity-10. Those with elevated measures of response bias may warrant alternative treatment considerations [25,55].

The significant relationship between the Validity-10 and PVT failure also had a small to medium effect size and is consistent with what has been shown in the literature using well-validated measures of symptom validity, such as the MMPI-2, in civilian and Veteran samples with a history of TBI and other neurological problems [34,56–57]. However, there was not complete overlap between Veterans who scored above the cutoff on the Validity-10 and those who failed the PVT variable, which supports the use of both PVTs and SVTs when determining validity of assessment in this population because SVT failure and PVT failure are not completely overlapping constructs [21,36].

The Validity-10 also predicted PVT failure over and above the contributions of PTSD, depression, and VA service connection. Depression was a significant predictor in the model, increasing the likelihood of failing the PVT variable by twice as much. VA service connection was also a significant predictor in the model; however, the second model that did not include this variable was also significant. The Validity-10 continued to contribute additional, albeit modest, predictive utility to both models after psychiatric comorbidities and VA service connection were accounted for, providing support for the strong relationship between SVTs and PVTs as well as the incremental predictive utility that accompanies the use of an SVT [36]. This finding is only partially congruent with a study by Armistead-Jehle that found SVT performance was clinically elevated in Veterans with a history of mTBI who failed PVTs but did not significantly predict PVT failure [29]. It is possible that the use of the Validity-10 may be particularly relevant in this population given its use of two different types of response bias techniques and its direct targeting of postconcussive symptoms. Predicting PVT failure using a quick, embedded measure such as the Validity-10 could alert providers to refer Veterans directly to mental health services rather than to a long neuropsychological test battery that will likely be uninterpretable.

Some limitations of the present study should be noted. Not all Veterans had data for every measure of interest because of the clinical nature of the data acquisition; however, all of the variables included in the hierarchical logistic regression were present for all Veterans. Similarly, many of the Veterans had comorbid psychiatric, sleep, and pain conditions that further complicate postconcussive symptoms. This level of comorbidity is characteristic of this population, however, and the use of a clinical sample seeking treatment for postconcussive symptoms in the chronic phase after mTBI does allow better generalizability to this clinical population of interest. Additionally, several of the postconcussive symptoms on the Validity-10 are nonspecific to mTBI, such as slowed thinking, slowed decision making, and nausea. TBI injury information was based on retrospective self-report from the Veterans, and no exact information regarding time since injury was available. While relying on self-report for injury details is not ideal, it is virtually unavoidable in this population because corroborating information and/or documentation of concussive events (particularly sustained in theater) is rarely available. Although VA service connection was used as a metric of compensation-seeking, no direct measure was obtained. Additionally, the PVTs used in this study may be too insensitive to detect invalid performance. Thus, replication of the present study using more sensitive PVTs is warranted [58–59]. Dizziness, balance, coordination, nausea, vision, and taste/smell symptoms are not unexpected following an mTBI [17] but are the most infrequently endorsed symptoms in the postacute phase in the current data as well as in other studies [23,46,60]. Taken together, there is support for the interpretation of symptom overreporting when these symptoms are endorsed in the postacute phase. Nonetheless, approximately one-third of our sample, as well as that of Vanderploeg et al. [23], continues to endorse these symptoms, and characterizing these symptoms as infrequent and/or as indicators of overreporting could be a mischaracterization of the symptoms. Finally, there was no gold-standard measure of symptom validity, which did not allow us to replicate the analyses that created the NIM5 subscale of the Validity-10 and limits the degree of certainty that high scores on the Validity-10 are, in fact, measuring overreporting. Although the Validity-10 was previously cross-validated with the PAI Negative Impression Management scale, additional research comparing the Validity-10 to symptom validity indices on other traditional/gold-standard SVTs (e.g., MMPI-2 validity scales) in Veterans with a history of mTBI would instill additional confidence in the Validity-10's ability to validly detect symptom overreporting. The present study had multiple strengths. The NSI is widely administered within the VA; therefore, assessing the utility of an embedded measure within the NSI in a Veteran population allows for ease of translation into clinical care. Additionally, there were multiple other variables collected for the participants in this study, including PTSD symptoms, depression, years of education, VA service connection, and multiple measures of performance validity, which allowed for understanding of the Validity-10 in relation to variables commonly of interest in this population that may be related to a response bias.

Overall, the present study partially replicated and extended the initial development and validation study of the Validity-10 by Vanderploeg et al. [23] in a treatment-seeking sample of OIF/OEF Veterans with mTBI and further supports the use of this measure as an SVT in this population as well as its potential as a PVT predictor. The Validity-10 is a simple and quick embedded measure within the NSI that is administered to all Veterans in the VA with a history of mTBI, takes minimal time to administer and score, can provide information on symptom validity that is comparable to much longer measures, and can provide significant clinical value in developing a case conceptualization and in triaging a patient for the most appropriate clinical services. Although indications of response bias may arise from a multitude of sources, including purposeful exaggeration and somatization, the distress that contributes to any cause of symptom validity failure strongly suggests mental health as opposed to neurological symptom treatment, and this finding may help to better route the Veterans as efficiently as possible to the most appropriate services within the VA.

CONCLUSIONS

Postconcussive symptom validity as measured by the Validity-10 is significantly related to reporting of PTSD, depression, and VA service connection in OIF/OEF Veterans with a history of mTBI. Furthermore, the Validity-10 modestly predicts PVT failure over PTSD, depression, and VA service connection. The Validity-10 is quick to score and interpret, is embedded within a widely used symptom inventory, and appears to have clinical utility in informing appropriate triaging within the VA for Veterans presenting with neurobehavioral complaints.

ACKNOWLEDGMENTS
Author Contributions:
Study concept and design: S. M. Jurick, H. J. Orff, A. J. Jak.
Acquisition of data: S. M. Jurick, C. C. Hays.
Analysis and interpretation of data: S. M. Jurick, C. C. Hays, H. J. Orff.
Drafting of manuscript: S. M. Jurick.
Critical revision of manuscript for important intellectual content: E. W. Twamley, L. D. Crocker, A. J. Jak, H. J. Orff.
Statistical analysis: S. M. Jurick, S. Golshan.
Study supervision: A. J. Jak.
Financial Disclosures: The authors have declared that no competing interests exist.
Funding/Support: This material was unfunded at the time of manuscript preparation. Laura D. Crocker was supported by the VA Office of Academic Affiliations Interprofessional Polytrauma and Traumatic Brain Injury Rehabilitation Fellowship.
Institutional Review: The present study was approved by the University of California San Diego Institutional Review Board and the VA San Diego Healthcare System Human Research Protection Program.
Participant Follow-Up: The authors do not plan to notify participants of the publication of this study because there were no plans for participant contact. Thus, contact was not approved by the VA San Diego Healthcare System Human Research Protection Program.
REFERENCES
1.
Tanielian TL, Jaycox LH, editors. Invisible wounds of war: Psychological and cognitive injuries, their consequences, and services to assist recovery [Internet]. Santa Monica: Rand Corporation; 2008. Available from:
http://www.rand.org/content/dam/rand/pubs/monographs/2008/RAND_MG720.pdf
2.
McCrea M. Mild traumatic brain injury and postconcussion syndrome: The new evidence base for diagnosis and treatment. 1st ed. New York: Oxford University Press; 2008.
3.
Management of Concussion/mTBI Working Group. VA/DoD Clinical Practice Guideline for Management of Concussion/Mild Traumatic Brain Injury. J Rehabil Res Dev. 2009;46(6):CP168. [PMID:20108447]
4.
Belanger HG, Vanderploeg, RD. The neuropsychological impact of sports-related concussion: A meta-analysis. J Int Neuropsychol Soc. 2005;11(4):34557. [PMID:16209414]
5.
Belanger HG, Curtiss G, Demery JA, Lebowitz BK, Vanderploeg RD. Factors moderating neuropsychological outcomes following mild traumatic brain injury: A meta-analysis. J Int Neuropsychol Soc. 2005;11(3):21527. [PMID:15892898]
http://dx.doi.org/10.1017/S1355617705050277
6.
Lannsjö M, af Geijerstam JL, Johansson U, Bring J, Borg J. Prevalence and structure of symptoms at 3 months after mild traumatic brain injury in a national cohort. Brain Inj. 2009;23(3):21319. [PMID:19205957]
http://dx.doi.org/10.1080/02699050902748356
7.
Ryan LM, Warden DL. Post concussion syndrome. Int Rev Psychiatry. 2003;15(4):31016. [PMID:15276952]
http://dx.doi.org/10.1080/09540260310001606692
8.
Laborey M, Masson F, Ribéreau-Gayon R, Zongo D, Salmi LR, Lagarde E. Specificity of postconcussion symptoms at 3 months after mild traumatic brain injury: Results from a comparative cohort study. J Head Trauma Rehabil. 2014; 29(1):E2836. [PMID:23474878]
http://dx.doi.org/10.1097/HTR.0b013e318280f896
9.
Meares S, Shores EA, Taylor AJ, Batchelor J, Bryant RA, Baguley IJ, Chapman J, Gurka J, Dawson K, Capon L, Marosszeky JE. Mild traumatic brain injury does not predict acute postconcussion syndrome. J Neurol Neurosurg Psychiatry. 2008;79(3):3006. [PMID:17702772]
http://dx.doi.org/10.1136/jnnp.2007.126565
10.
Mooney G, Speed J, Sheppard S. Factors related to recovery after mild traumatic brain injury. Brain Inj. 2005; 19(12):97587. [PMID:16263640]
http://dx.doi.org/10.1080/02699050500110264
11.
Lew HL, Otis JD, Tun C, Kerns RD, Clark ME, Cifu DX. Prevalence of chronic pain, posttraumatic stress disorder, and persistent postconcussive symptoms in OIF/OEF veterans: Polytrauma clinical triad. J Rehabil Res Dev. 2009; 46(6):697702. [PMID:20104399]
http://dx.doi.org/10.1682/JRRD.2009.01.0006
12.
Carroll LJ, Cassidy JD, Peloso PM, Borg J, von Holst H, Holm L, Paniak C, Pépin M; WHO Collaborating Centre Task Force on Mild Traumatic Brain Injury. Prognosis for mild traumatic brain injury: Results of the WHO Collaborating Centre Task Force on Mild Traumatic Brain Injury. J Rehabil Med. 2004;36(43 Suppl):84105.
[PMID:15083873]
http://dx.doi.org/10.1080/16501960410023859
13.
Belanger HG, Kretzmer T, Vanderploeg RD, French LM. Symptom complaints following combat-related traumatic brain injury: Relationship to traumatic brain injury severity and posttraumatic stress disorder. J Int Neuropsychol Soc. 2010;16(1):19499. [PMID:19758488]
http://dx.doi.org/10.1017/S1355617709990841
14.
Brenner LA, Ivins BJ, Schwab K, Warden D, Nelson LA, Jaffee M, Terrio H. Traumatic brain injury, posttraumatic stress disorder, and postconcussive symptom reporting among troops returning from Iraq. J Head Trauma Rehabil. 2010;25(5):30712. [PMID:20042982]
http://dx.doi.org/10.1097/HTR.0b013e3181cada03
15.
Bryant RA, Harvey AG. Relationship between acute stress disorder and posttraumatic stress disorder following mild traumatic brain injury. Am J Psychiatry. 1998;155(5):6259.
16.
Schneiderman AI, Braver ER, Kang HK. Understanding sequelae of injury mechanisms and mild traumatic brain injury incurred during the conflicts in Iraq and Afghanistan: Persistent postconcussive symptoms and posttraumatic stress disorder. Am J Epidemiol. 2008;167(12): 144652. [PMID:18424429]
http://dx.doi.org/10.1093/aje/kwn068
17.
Hoge CW, McGurk D, Thomas JL, Cox AL, Engel CC, Castro CA. Mild traumatic brain injury in U.S. Soldiers returning from Iraq. N Engl J Med. 2008;358(5):45363.
[PMID:18234750]
http://dx.doi.org/10.1056/NEJMoa072972
18.
Howe LL. Giving context to post-deployment post–concussive-like symptoms: Blast-related potential mild traumatic brain injury and comorbidities. Clin Neuropsychol. 2009;23(8):131537. [PMID:19882474]
http://dx.doi.org/10.1080/13854040903266928
19.
Logan BW, Goldman S, Zola M, Mackey A. Concussive brain injury in the military: September 2001 to the present. Behav Sci Law. 2013;31(6):80313. [PMID:24130079]
http://dx.doi.org/10.1002/bsl.2092
20.
McCrea M, Pliskin N, Barth J, Cox D, Fink J, French L, Hammeke T, Hess D, Hopewell A, Orme D, Powell M, Ruff R, Schrock B, Terryberry-Spohr L, Vanderploeg R, Yoash-Gantz R. Official position of the military TBI task force on the role of neuropsychology and rehabilitation psychology in the evaluation, management, and research of military veterans with traumatic brain injury. Clin Neuropsychol. 2008;22(1):1026. [PMID:18247218]
http://dx.doi.org/10.1080/13854040701760981
21.
Larrabee GJ. Performance validity and symptom validity in neuropsychological assessment. J Int Neuropsychol Soc. 2012;18(4):62530. [PMID:23057079]
http://dx.doi.org/10.1017/S1355617712000240
22.
Butcher J, Dahlstrom WG, Graham JR, Tellegen A, Kaemmer B. Manual for administration and scoring, MMPI-2, Minnesota Multiphasic Personality Inventory-2. Minneapolis: University of Minnesota Press; 1989.
23.
Vanderploeg RD, Cooper DB, Belanger HG, Donnell AJ, Kennedy JE, Hopewell CA, Scott SG. Screening for postdeployment conditions: Development and cross-validation of an embedded validity scale in the neurobehavioral symptom inventory. J Head Trauma Rehabil. 2014;29(1): 110. [PMID:23474880]
http://dx.doi.org/10.1097/HTR.0b013e318281966e
24.
Cicerone KD, Kalmar K. Persistent postconcussion syndrome: The structure of subjective complaints after mild traumatic brain injury. J Head Trauma Rehabil. 1995;10(3):117.
http://dx.doi.org/10.1097/00001199-199510030-00002
25.
Cooper DB, Nelson L, Armistead-Jehle P, Bowles AO. Utility of the Mild Brain Injury Atypical Symptoms Scale as a screening measure for symptom over-reporting in Operation Enduring Freedom/Operation Iraqi Freedom service members with post-concussive complaints. Arch Clin Neuropsychol. 2011;26(8):71827. [PMID:21873326]
http://dx.doi.org/10.1093/arclin/acr070
26.
Morey LC. Personality Assessment Inventory (PAI). The encyclopedia of clinical psychology; 2014.
http://dx.doi.org/10.1002/9781118625392.wbecp284
27.
Lange RT, Brickell TA, French LM. Examination of the Mild Brain Injury Atypical Symptom Scale and the Validity-10 Scale to detect symptom exaggeration in US military service members. J Clin Exp Neuropsychol. 2015;37(3): 32537. [PMID:25849968]
http://dx.doi.org/10.1080/13803395.2015.1013021
28.
Lange RT, Brickell TA, Lippa SM, French LM. Clinical utility of the Neurobehavioral Symptom Inventory validity scales to screen for symptom exaggeration following traumatic brain injury. J Clin Exp Neuropsychol. 2015;37(8): 85362. [PMID:26245293]
http://dx.doi.org/10.1080/13803395.2015.1064864
29.
Armistead-Jehle P. Symptom validity test performance in U.S. veterans referred for evaluation of mild TBI. Appl Neuropsychol. 2010;17(1):5259. [PMID:20146122]
http://dx.doi.org/10.1080/09084280903526182
30.
Whitney KA, Shepard PH, Williams AL, Davis JJ, Adams KM. The medical symptom validity test in the evaluation of Operation Iraqi Freedom/Operation Enduring Freedom soldiers: A preliminary study. Arch Clin Neuropsychol. 2009;24(2):14552. [PMID:19395348]
http://dx.doi.org/10.1093/arclin/acp020
31.
Miskey HM, Shura RD, Yoash-Gantz RE, Rowland JA. Personality Assessment Inventory profiles of veterans: Differential effects of mild traumatic brain injury and psychopathology. Brain Imaging Behav. 2015;9(3):46171.
[PMID:25913646]
http://dx.doi.org/10.1007/s11682-015-9391-7
32.
Whiteside DM, Galbreath J, Brown M, Turnbull J. Differential response patterns on the Personality Assessment Inventory (PAI) in compensation-seeking and non-compensation-seeking mild traumatic brain injury patients. J Clin Exp Neuropsychol. 2012;34(2):17282. [PMID:22136511]
http://dx.doi.org/10.1080/13803395.2011.630648
33.
Jak AJ, Gregory A, Orff HJ, Colón C, Steele N, Schiehser DM, Delano-Wood L, Jurick SM, Twamley EW. Neuropsychological performance in treatment-seeking Operation Enduring Freedom/Operation Iraqi Freedom veterans with a history of mild traumatic brain injury. J Clin Exp Neuropsychol. 2015;37(4):37988. [PMID:25850338]
http://dx.doi.org/10.1080/13803395.2015.1020769
34.
Whitney KA, Davis JJ, Shepard PH, Herman SM. Utility of the Response Bias Scale (RBS) and other MMPI-2 validity scales in predicting TOMM performance. Arch Clin Neuropsychol. 2008;23(7–8):77786. [PMID:18930375]
http://dx.doi.org/10.1016/j.acn.2008.09.001
35.
Ross SR, Millis SR, Krukowski RA, Putnam SH, Adams KM. Detecting incomplete effort on the MMPI-2: An examination of the Fake-Bad Scale in mild head injury. J Clin Exp Neuropsychol. 2004;26(1):11524. [PMID:14972699]
http://dx.doi.org/10.1076/jcen.26.1.115.23933
36.
Van Dyke SA, Millis SR, Axelrod BN, Hanks RA. Assessing effort: Differentiating performance and symptom validity. Clin Neuropsychol. 2013;27(8):123446. [PMID:24028487]
http://dx.doi.org/10.1080/13854046.2013.835447
37.
Green P. Medical Symptom Validity Test (MSVT) for Microsoft Windows: User's manual. Edmonton, Canada: Paul Green Publishing Inc; 2004.
38.
Tombaugh TN. Test of Memory Malingering. North Tonawanda (NY): Multi-Health Systems; 1996.
39.
Cernich AN, Chandler L, Scherdell T, Kurtz S. Assessment of co-occurring disorders in veterans diagnosed with traumatic brain injury. J Head Trauma Rehabil. 2012;27(4): 25360. [PMID:22767073]
http://dx.doi.org/10.1097/HTR.0b013e3182585cd5
40.
Kashluba S, Paniak C, Casey JE. Persistent symptoms associated with factors identified by the WHO Task Force on Mild Traumatic Brain Injury. Clin Neuropsychol. 2008; 22(2):195208. [PMID:17853135]
http://dx.doi.org/10.1080/13854040701263655
41.
Lange RT, Iverson GL, Rose A. Depression strongly influences postconcussion symptom reporting following mild traumatic brain injury. J Head Trauma Rehabil. 2011;26(2): 12737. [PMID:20631632]
http://dx.doi.org/10.1097/HTR.0b013e3181e4622a
42.
Lange RT, Brickell TA, Kennedy JE, Bailie JM, Sills C, Asmussen S, Amador R, Dilay A, Ivins B, French LM. Factors influencing postconcussion and posttraumatic stress symptom reporting following military-related concurrent polytrauma and traumatic brain injury. Arch Clin Neuropsychol. 2014;29(4):32947. [PMID:24723461]
http://dx.doi.org/10.1093/arclin/acu013
43.
Ponsford J, Willmott C, Rothwell A, Cameron P, Kelly AM, Nelms R, Curran C, Ng K. Factors influencing outcome following mild traumatic brain injury in adults. J Int Neuropsychol Soc. 2000;6(5):56879. [PMID:10932476]
http://dx.doi.org/10.1017/S1355617700655066
44.
Soble JR, Silva MA, Vanderploeg RD, Curtiss G, Belanger HG, Donnell AJ, Scott SG. Normative data for the Neurobehavioral Symptom Inventory (NSI) and post-concussion symptom profiles among TBI, PTSD, and nonclinical samples. Clin Neuropsychol. 2014;28(4):61432.
[PMID:24625213]
http://dx.doi.org/10.1080/13854046.2014.894576
45.
Heilbronner RL, Sweet JJ, Morgan JE, Larrabee GJ, Millis SR; Conference Participants. American Academy of Clinical Neuropsychology Consensus Conference Statement on the neuropsychological assessment of effort, response bias, and malingering. Clin Neuropsychol. 2009;23(7):10931129.
[PMID:19735055]
http://dx.doi.org/10.1080/13854040903155063
46.
Lange RT, Brickell TA, Ivins B, Vanderploeg RD, French LM. Variable, not always persistent, postconcussion symptoms after mild TBI in U.S. military service members: A five-year cross-sectional outcome study. J Neurotrauma. 2013;30(11):95869. [PMID:23205671]
http://dx.doi.org/10.1089/neu.2012.2743
47.
Wilkinson GS, Robertson GJ. Wide Range Achievement Test 4 (WRAT4): Professional manual. Wilmington (MA): Harcourt Assessment; 2006.
48.
Beck AT, Steer RA, Brown GK. Beck Depression Inventory-II. San Antonio (TX): The Psychological Corporation; 1996.
49.
Weathers FW, Litz BT, Herman DS, Huska JA, Keane TM. The PTSD Checklist (PCL): Reliability, validity, and diagnostic utility. In: Proceedings of the International Society for Traumatic Stress Studies Annual Meeting; 1993 Oct 25; San Antonio, Texas.
50.
Delis DC, Kramer JH, Kaplan E, Ober BA. California Verbal Learning Test-Second Edition (CVLT-II) manual. San Antonio (TX): PsychCorp; 1999.
51.
Proto DA, Pastorek NJ, Miller BI, Romesser JM, Sim AH, Linck JF. The dangers of failing one or more performance validity tests in individuals claiming mild traumatic brain injury-related postconcussive symptoms. Arch Clin Neuropsychol. 2014;29(7):61424. [PMID:25252598]
http://dx.doi.org/10.1093/arclin/acu044
52.
Moore BA, Donders J. Predictors of invalid neuropsychological test performance after traumatic brain injury. Brain Inj. 2004;18(10):97584. [PMID:15370897]
http://dx.doi.org/10.1080/02699050410001672350
53.
Cohen J. Statistical power for the social sciences. Hillsdale (NJ): Laurence Erlbaum and Associates; 1988.
54.
Paniak C, Reynolds S, Toller-Lobe G, Melnyk A, Nagy J, Schmidt D. A longitudinal study of the relationship between financial compensation and symptoms after treated mild traumatic brain injury. J Clin Exp Neuropsychol. 2002;24(2):18793. [PMID:11992201]
http://dx.doi.org/10.1076/jcen.24.2.187.999
55.
Nelson NW, Hoelzle JB, McGuire KA, Sim AH, Goldman DJ, Ferrier-Auerbach AG, Charlesworth MJ, Arbisi PA, Sponheim SR. Self-report of psychological function among OEF/OIF personnel who also report combat-related concussion. Clin Neuropsychol. 2011;25(5):71640.
[PMID:21722045]
http://dx.doi.org/10.1080/13854046.2011.579174
56.
Jones A, Ingram MV. A comparison of selected MMPI-2 and MMPI-2-RF validity scales in assessing effort on cognitive tests in a military sample. Clin Neuropsychol. 2011; 25(7):120727. [PMID:21902565]
http://dx.doi.org/10.1080/13854046.2011.600726
57.
Thomas ML, Youngjohn JR. Let's not get hysterical: Comparing the MMPI-2 validity, clinical, and RC scales in TBI litigants tested for effort. Clin Neuropsychol. 2009;23(6): 106784. [PMID:19343593]
http://dx.doi.org/10.1080/13854040902795000
58.
Armistead-Jehle P, Gervais RO. Sensitivity of the Test of Memory Malingering and the Nonverbal Medical Symptom Validity Test: A replication study. Appl Neuropsychol. 2011;18(4):28490. [PMID:22074067]
http://dx.doi.org/10.1080/09084282.2011.595455
59.
Baker R, Donders J, Thompson E. Assessment of incomplete effort with the California Verbal Learning Test. Appl Neuropsychol. 2000;7(2):11114. [PMID:10863607]
http://dx.doi.org/10.1207/S15324826AN0702_8
60.
Kashluba S, Paniak C, Blake T, Reynolds S, Toller-Lobe G, Nagy J. A longitudinal, controlled study of patient complaints following treated mild traumatic brain injury. Arch Clin Neuropsychol. 2004;19(6):80516. [PMID:15288333]
http://dx.doi.org/10.1016/j.acn.2003.09.005
This article and any supplementary material should be cited as follows:
Jurick SM, Twamley EW, Crocker LD, Hays CC, Orff HJ, Golshan S, Jak AJ. Postconcussive symptom overreporting in Iraq/Afghanistan Veterans with mild traumatic brain injury. J Rehabil Res Dev. 2016;53(5):571–84.
http://dx.doi.org/10.1682/JRRD.2015.05.0094
iThenticateCrossref

Go to TOP

VHA

Last Reviewed or Updated Thursday, November 3, 2016 12:56 PM

Valid HTML 4.01 Transitional