Journal of Rehabilitation Research & Development (JRRD)

Quick Links

  • Health Programs
  • Protect your health
  • Learn more: A-Z Health
Veterans Crisis Line Badge
 

Volume 51 Number 8, 2014
   Pages 1203 — 1216

Diagnostic accuracy of Posttraumatic Stress Disorder Checklist in ­blast-exposed military personnel

William C. Walker, MD;1–3* Scott D. McDonald, PhD;1,3–4 Laura Manning Franke, PhD1–3

1Department of Physical Medicine and Rehabilitation, School of Medicine, Virginia Commonwealth University, Richmond, VA; 2Defense and Veterans Brain Injury Center, Richmond, VA; 3Hunter Holmes McGuire Department of Veterans Affairs Medical Center, Richmond, VA; 4Department of Psychology, Virginia Commonwealth University, Richmond, VA

Abstract — Researchers often extrapolate posttraumatic stress disorder (PTSD) status from PTSD Checklist (PCL) data. When doing so, cut points should be based on samples with similar characteristics. This study assessed PCL diagnostic accuracy and postconcussive symptom levels within 106 Iraq/Afghanistan war Veterans and servicemembers with recent blast exposure. Two definitions of PTSD were applied: (1) “strict” Diagnostic and Statistical Manual of Mental Health Disorders (DSM), 4th edition (DSM-IV) criteria and (2) “relaxed” DSM-IV criteria dropping the A2 criterion as per the DSM, 5th edition. Using a structured interview for PTSD, we found moderate agreement with the PCL. Under strict criteria, PTSD prevalence was 16%, PCL cut point was 66 at peak kappa, and mean Rivermead Postconcussion Questionnaire (RPQ) score trended higher for those with PTSD than for those without PTSD (35.5 +/– 11.2 vs 30.5 +/– 10.7, respectively; p = 0.08). Under relaxed criteria, PTSD prevalence was 26.4%, PCL cut point was 58 at peak kappa, and those with PTSD had higher RPQ scores than those without PTSD (36.4 +/– 11.2 vs 29.5 +/– 10.2, respectively; p = 0.003). Participants diagnosed with blast-related mild traumatic brain injury (n = 90) did not differ from those without mild traumatic brain injury (n = 16) in symptom scores. In conclusion, persons with combat-related blast exposure need higher than conventional PCL cut points and those with PTSD have more severe postconcussive-type symptoms than those without PTSD.

Key words: brain injury, concussion, diagnostic criteria, diagnostic errors, dual diagnosis, explosive blast, investigative techniques, medical history taking, military injury, posttraumatic stress disorder, sensitivity, specificity.

Abbreviations: CI = confidence interval; DoD = Department of Defense; DSM = Diagnostic and Statistical Manual of Mental Health Disorders; DSM-IV = DSM, 4th edition; DSM-5 = DSM, 5th edition; ECMP = Events Checklist for Military Personnel; IQR = interquartile range; MINI = Mini-International Neuropsychiatric Interview; mTBI = mild traumatic brain injury; NPP = negative predictive power; OEF = Operation Enduring Freedom; OIF = Operation Iraqi Freedom; OND = Operation New Dawn; PCL = Posttraumatic Stress Disorder Checklist; PCS = postconcussion syndrome; PPP = positive predictive power; PTSD = posttraumatic stress disorder; ROC = receiver operating characteristic; RPQ = Rivermead Postconcussion Questionnaire; SCID = Structured Clinical Interview for DSM Disorders; SCM = symptom cluster method; SD = standard deviation; SM = servicemember; TBI = traumatic brain injury; VA = Department of Veterans Affairs; VHA = Veterans Health Administration.
*Address all correspondence to William C. Walker, MD; Virginia Commonwealth University, Department of Physical Medicine and Rehabilitation, MCV Station, Box 677, Richmond, VA 23298; 804-828-0861; fax: 804-828-5074.
Email: wwalker@mcvh-vcu.edu
http://dx.doi.org/10.1682/JRRD.2013.12.0271
INTRODUCTION

Exposure to psychologically traumatic events is an inherent aspect of military combat deployment and often may lead to posttraumatic stress disorder (PTSD). In Operation Iraqi Freedom (OIF), Operation Enduring Freedom (OEF), and Operation New Dawn (OND), U.S. servicemembers (SMs) have experienced an especially high rate of exposure to blast-induced traumatic events. Heavily used by the insurgents, explosive munitions have accounted for about 78 percent of wounded-in-action cases, the highest proportion for any large-scale conflict [1]. Accordingly, OIF/OEF/OND combatants are typically screened for PTSD after returning from deployment, usually via the easy-to-administer and widely used PTSD Checklist (PCL) [2–3]. The PCL is favored by both the Department of Defense (DoD) and the Veterans Health Administration (VHA) as a PTSD screening tool and is mandated in certain clinical settings.

Although not intended as a diagnostic tool, the PCL has been used in numerous published clinical research studies to categorize individuals into PTSD positive versus negative groups, typically using a total score ≥50 to define PTSD [4]. But most studies, particularly those focusing on mild traumatic brain injury (mTBI) cohorts, have used the PCL in this manner without fully considering its diagnostic accuracy and optimal cut point within their sampled population [5]. The accuracy of any diagnostic tool, such as sensitivity and specificity values, is heavily influenced by the true prevalence of the index condition within the population under study. Regarding PTSD, the entire postdeployment population does not have uniformity of traumatic exposures and risk level. Exposure rates to blast and other traumatic combat events vary widely depending on one’s military role and deployment-specific geographic location and missions [6]. Therefore, the true prevalence of PTSD will vary with different sample selection methods, which in turn affects the diagnostic accuracy of the PCL. Therefore, it is unlikely that commonly accepted Veteran population cut points (e.g., 50) [2,7] would be equally appropriate across different population types or study samples. The population of blast-exposed SMs and Veterans is a key target population because they are most at risk for the signature wounds of OIF/OEF/OND, traumatic brain injury (TBI) and PTSD, and are the focal point of Department of Veterans Affairs (VA) and DoD clinical care and research efforts. But the literature lacks guidance on how to use the PCL to categorize PTSD in this very high risk population.

Current warfighters and the situations they face are different than those faced by the original validation samples for the PCL (Vietnam and first Gulf war combat Veterans). Now, new stressors may be involved in the development of PTSD—“exponentially” more and much longer deployments, improvised explosive devices and suicide bombers, exposure of peacekeeping forces to combative situations, and higher survivability of wounds [8]. The potential for TBI is also a critical consideration that may confound PTSD determinations within the military and Veteran population. More than 266,000 TBI casualties have been reported by the DoD between 2000 and 2013 [9], and concussion, also known as mTBI, accounts for well over 80 percent of these [10]. Importantly, up to 20 percent of persons sustaining mTBI will develop postconcussion syndrome (PCS), a condition of chronic symptoms and potential psychosocial dysfunction [11–12] that has significant symptom overlap with PTSD. Because of this symptom overlap, the presence of PCS after mTBI may inflate the PCL score and generate higher rates of false positive PTSD screens if standard cut points are used. While PTSD symptoms are reported to occur acutely in up to 40 percent of U.S. military personnel following an mTBI [13] and persistently in 42 percent of recent OIF/OEF Veterans with a history of mTBI [14], the actual PTSD risk and prevalence among those who sustained mTBI during OIF/OEF/OND is unknown because these and other investigations have determined PTSD status from PCL cut points that were derived from much different samples of combat Veterans, notably without high mTBI prevalence. For example, Kontos et al. reported that military personnel with a self-reported blast-related mTBI diagnosis were at risk (odds ratio 4.2 vs no mTBI diagnosis) for reporting “clinical levels” of PTSD symptoms, but this was defined as a cut point ≥28 on the PCL, which was not crossvalidated for clinical PTSD [15].

The specific PCL scoring method used will also influence its diagnostic accuracy. Some investigators have proposed using a symptom categorization method rather than total score to categorize study participants. The symptom cluster method (SCM) requires the endorsement of one re-experiencing, three numbing/avoidance, and two hyperarousal symptoms as per Diagnostic and Statistical Manual of Mental Health Disorders (DSM), 4th edition (DSM-IV), Text Revision diagnostic criteria [16]. A combination of the total score and SCM has also been proposed and used ostensibly to enhance specificity and positive predictive power (PPP) by raising the symptom severity threshold of SCM alone [17].

Because of these sampling and scoring issues, published PTSD prevalence rates among SMs and Veterans with OIF/OEF/OND combat deployment histories have shown widely varying rates from 1.4 to 31 percent [4]. Moreover, almost all studies in the past 2 decades have relied on DSM-IV criteria to base their gold standard PTSD definition, which may not correlate well with the recently released DSM, 5th edition (DSM-5) criteria. It will be important for future studies using legacy data to impute their findings into this emerging standard.

In summary, the present study was undertaken because much remains to be learned about the influence of mTBI on PTSD and PCS symptom reporting and on the diagnostic accuracy of the PCL among persons with military blast exposure. The current study is part of an overarching research project aiming to comprehensively assess individuals with one or more combat-related blast experience, defined as a blast event that they were proximate to and felt some immediate physical effect. Such individuals are believed to be at very high risk for not only TBI but also PTSD and are typical of former OIF/OEF/OND combatants seeking care or evaluation for TBI within the DoD or VA. We first sought to determine the true prevalence for the diagnosis of PTSD within this high risk study sample by using structured interview as the reference standard for PTSD. Second, because the diagnostic accuracy of the PCL is not well established for such a population, we also sought to assess its accuracy across cut points and determine the optimal method for dichotomizing the PCL into PTSD diagnoses. Third, we aimed to assess what effect the exclusion of the DSM-IV A2 criterion (emotional reaction to stressor) might have on these findings because this method has been advocated for combat experiences and is more consistent with DSM-5. Lastly, we sought to determine the relationship between the diagnoses of PTSD and blast-related mTBI to current PTSD and PCS symptom levels.

METHODS

The study sample was 106 consecutive participants who consented and completed baseline evaluations after structured interviews were added to the parent epidemiologic study of military blast exposure in 2010. All appropriate institutional review board and governmental approvals were obtained. SMs and Veterans were eligible if they had a blast experience within the past 2 yr while deployed in OIF/OEF/OND. Participants were recruited via letters and advertisements and from ambulatory health care clinics at the Hunter Holmes McGuire VA Medical Center in Richmond, Virginia; Fort Lee Army Base in Prince George County, Virginia; Quantico Marine Corps Base in Prince William County, Virginia; and Camp Lejeune Marine Corps Base in North Carolina. Blast experience was defined as having any of the following symptoms or experiences occurring during or shortly after exposure to blast or explosion: dazed, confused, saw stars, headache, dizziness, irritability, memory gap (not remembering injury or injury period), hearing loss, abdominal pain, shortness of breath, struck by debris, knocked over or down, knocked into or against something, helmet damaged, or medically evacuated. Individuals with severe or moderate TBI were excluded, so participants either had no TBI or sustained an mTBI during their blast experience. Severe or moderate TBI was defined as more than 30 min of lost consciousness, brain bleeding or blood clot (abnormal brain computed tomography scan), or amnesia for the first 24 or more hours after event.

As part of a comprehensive baseline assessment battery, all participants completed the PCL and the Rivermead Postconcussion Questionnaire (RPQ). We used the civilian version of the PCL, in which items are identical to the military version, to avoid assuming only military-related traumatic life events had occurred. The RPQ is a 16-item self-report measure of the presence and severity of the 16 most commonly reported postconcussion symptoms found in the literature as compared to pretraumatic event [18–19]. Subsequently and separately, trained research assistants who were blinded to the PCL results administered to each participant the structured interview battery consisting of the Events Checklist for Military Personnel (ECMP), the Mini-International Neuropsychiatric Interview (MINI) version 6.0, and a diagnostic TBI interview.

The ECMP is a questionnaire developed specifically for this study that was used to identify distressing combat and noncombat events that met the DSM-IV Criterion A for PTSD (qualifying stressor). The ECMP differs from other traumatic events questionnaires [20–21] in that items pertaining to combat events are listed separately from noncombat events. For each list, respondents are first asked to mark whether each event occurred (e.g., “Witnessed the serious injury or death of enemy troops”). Next, they are asked to identify which event “was the MOST distressing or traumatic”; the date; their age; and a series of questions regarding their response to the event (e.g., fear, helplessness, or horror; anger), the intensity of distress at the time of the event and at the time of the rating (8-point scale from “not at all” to “extremely”), and the outcome (e.g., “Were you physically injured during the event?”). Finally, respondents are asked to identify whether a combat or noncombat event was the most traumatic event ever experienced.

The MINI is a validated, short, structured diagnostic interview based on DSM and International Classification of Diseases criteria that was developed by psychiatrists and clinicians jointly in the United States and Europe [22]. The MINI has been validated against the Structured Clinical Interview for DSM Disorders (SCID) and the Composite International Diagnostic Interview with good concordance [22]. In comparisons of the MINI and the SCID for the diagnosis for PTSD, Sheehan et al. reported a sensitivity of 0.85, specificity of 0.96, positive predictive value of 0.82, negative predictive value of 0.97, and kappa of 0.78 [22]. More recently, Jones et al. found high concordance between the MINI and the SCID and recommended the MINI as a shorter, standardized interview for Axis I diagnoses [23]. When this study was initiated, the DSM-IV was the current gold standard upon which the MINI was based. In order to better inform whether or not the A2 criterion was met, we added structured follow-up questioning if the participant denied that the most distressing event was followed by “an emotional reaction characterized by intense fear, helplessness, or horror.” Specifically, if the initial query response was “no,” then the participant was also asked (1) to “describe the emotion,” if any, and (2) “were you stunned or shocked in a way that you didn’t feel anything at all?” followed, if “yes,” by querying again whether the A2 emotional response occurred “after the event had passed.” Interviewers received workshop training given by a member of the MINI development staff followed by in vivo practice vignettes, rating of three practice video tapes, and fidelity evaluation by a licensed clinical psychologist (S. M.).

From the MINI information, each participant’s PTSD diagnosis was determined using both a “strict” DSM-IV algorithm and a “relaxed” DSM-IV algorithm. The algorithms were identical except the strict required the DSM-IV A2 criterion to be met, while the relaxed ignored A2 so as to simulate DSM-5. For those participants initially negative for the A2 structured question but who either had another strong emotion or a delay in A2 after an immediate “numb” period, the study investigators made the A2 determination after reviewing the written descriptions.

The diagnostic interview for TBI was developed by the study investigators and was administered by a trained research assistant and consisted of both structured and unstructured components. For those with multiple blast-related experiences, the self-identified “worst” potential concussive event was selected for interview. The structured component focused on recalled immediate postevent symptoms that suggest alteration of consciousness occurred (e.g., amnesia, loss of consciousness, dazed, confused, saw stars) and queried for other postevent symptoms (e.g., headache, dizziness, irritability, fatigue, or poor concentration). Responses were independently reviewed by several experienced TBI physicians who individually rated each participant’s worst (or only) blast exposure as “yes” versus “no” in reference to the DoD/VA common definition for mTBI (http://www.cdc.gov/nchs/data/icd/Sep08TBI.pdf). Under these guidelines, determining that an immediate period of altered consciousness occurred is essential to diagnosing mTBI pursuant to an injury force; other postinjury symptoms can be used to support, but cannot be used to make, a diagnosis of mTBI in adults. A consensus diagnosis was obtained for each participant based on a simple majority.

History of mTBI prior to military service, prior deployments, and number of blast experiences were also collected using a modified version of the Walter Reed Army Medical Center Blast Injury Questionnaire, described by Scherer et al. [24].

Using the MINI to categorize the PTSD positive and negative status groups under both criteria, we performed between-group analyses with respect to their PCS symptom severity (RPQ) and PTSD symptom severity (PCL). We also compared PCL scores and RPQ scores for mTBI versus not TBI groups. All statistical analyses were conducted using SPSS Statistics version 21.0 (IBM Corporation; Armonk, New York). If variables were normally distributed (i.e., Shapiro-Wilk p-value > 0.05), then independent-sample, unpaired, two-tailed t-tests were conducted to assess for differences between groups. The Levene test for the equality of variances was calculated, and if the significance was found to be less than 0.05, equal variances were not assumed. For variables not normally distributed, we used the nonparametric Mann-Whitney U test for comparing independent samples.

Sensitivity, specificity, and area under receiver operating characteristic (ROC) curves were calculated using SPSS Statistics version 21.0. Other diagnostic accuracy indices (e.g., kappa) and Wald 95 percent confidence intervals (CIs) [25] were calculated using commonly available spreadsheet software. Using STAndards for the Reporting of Diagnostic accuracy studies guidelines [26], we next analyzed the diagnostic accuracy of the PCL for both MINI algorithms and the two PCL scoring methods. Classification metrics computed were sensitivity, specificity, PPP, negative predictive power (NPP), percentage correctly classified, and kappa statistic of the PCL using different PCL scoring methods and for both strict and relaxed DSM-IV criteria. To determine the optimal cut point for each condition, the diagnostic accuracy parameters were inspected at every cut point. Classification rate and kappa were both considered when considering the cut point with optimal diagnostic accuracy. The PCL SCM under DSM-IV PTSD criteria is positive if the person endorses at a level of moderately or higher (3, 4, or 5) at least one intrusion symptom (questions 1–5), at least three avoidance symptoms (questions 6–12), and at least two hyperarousal symptoms (questions 13–17) [5]. For the PCL total score method, we analyzed overall diagnostic accuracy from the ROC curve and inspected diagnostic accuracy parameters at each coordinate (cut point). CIs (95%) were computed for all diagnostic accuracy fractions using the Wald method [25].

RESULTS
Participant Characteristics

The demographic characteristics of the study sample (n = 106) are displayed in Table 1. To summarize, participants were evaluated at a median of 15.1 mo (interquartile range [IQR] = 10.1–24.4) after their worst blast experience and 12.9 mo (IQR = 8.0–19.9) after the most recent of their three reported worst blast experiences. The vast majority (n = 90, 84.9%) were determined to have sustained an mTBI during their worst blast experience.


Table 1. 
Psychologically Traumatic Events Experienced

All participants reported at least one psychologically traumatic event during combat tour and most (85.7%) also reported at least one traumatic event outside of ­combat tour. Regarding their “most” psychologically traumatic event, 91 participants reported that it was a combat event that occurred at a median age of 23 yr (IQR = 20.5–26.0). The remaining 15 participants (13.2% of total sample) reported that a noncombat event was their most traumatic; these occurred at a median age of 19 yr (IQR = 15.0–23.0). The specific types of events and frequencies are displayed in Table 2.

Prevalence of Posttraumatic Stress Disorder in Sample

The implementation versus nonimplementation of DSM-IV Criteria A2 had significant bearing on the true PTSD prevalence rate derived from the MINI. The prevalence rate was 16.0 percent (17/106) under the strict algorithm and was significantly higher at 26.4 percent (28/106) under the relaxed algorithm (McNemar’s test of symmetry, p = 0.001). Among the 11 participants meeting relaxed but not strict criteria, 3 participants reported having no strong emotional response and 8 participants reported some “other” strong emotional responses: “anger” by 4 and “adrenaline,” “training kicked in,” “respect,” and “disgust” by 1 each. The PTSD prevalence was not influenced by mTBI status (PTSD relaxed criteria prevalence if yes TBI = 26.7% vs if no TBI = 25%, Pearson chi square, p = 0.89; PTSD strict criteria prevalence if yes TBI = 26.7% vs if no TBI = 26.7%, Pearson chi square, p > 0.99).

Posttraumatic Stress Disorder Checklist Total Score and Rivermead Postconcussion Questionnaire Score Subgroup Analyses

The results of analyses of PTSD and PCS symptom severity between PTSD positive and negative groups on the MINI under both criteria are displayed in tabular form in Table 3 and Table 4. When applying the strict (A2 inclusive) DSM-IV criteria, those with PTSD (n = 17) had a higher mean ± standard deviation (SD) PCL total score than those without PTSD (n = 89), 62.4 ± 11.0 versus 45.9 ± 14.03, t = 5.86, p < 0.001, d = 1.22; and trended toward higher RPQ scores, 35.5 ± 11.2 versus 30.5 ± 10.7, t = 1.77, p = 0.08, d = 0.46. When dropping the A2 criterion, those with PTSD (n = 28) again had significantly higher PCL scores than those without PTSD (n = 78), 61.0 ± 11.6 versus 44.0 ± 12.0, t = 6.50, p < 0.001, d = 1.43; and had significantly higher RPQ scores, 36.4 ± 11.2 versus 29.5 ± 10.2, t = 2.98, p = 0.003, d = 0.66.

The additional between-group analyses are not shown in tabular form. When comparing the two PTSD positive groups under strict DSM-IV criteria (n = 17) versus relaxed DSM-IV criteria (n = 11), there was no difference in mean PCL scores, 62.4 ± 11.0 versus 58.9 ± 12.6, p = 0.52, d = 0.30; as well as no difference in RPQ scores, 35.5 ± 11.2 versus 37.6 ± 11.6, t = −0.479, p = 0.64, d = 0.18.

Comparative PTSD and PCS symptom severity between mTBI positive versus negative groups was also analyzed. Mean ± SD PCL total scores did not differ between participants with historical blast-related mTBI versus without it; 48.9 ± 13.9 versus 46.4 ± 15.3, respectively, p = 0.49, d = 0.18. Likewise Mean ± SD RPQ scores did not differ between the mTBI positive and ­negative groups; 31.3 ± 10.5 versus 31.3 ± 12.9, p >0.99, d < 0.01.

Given our findings that PCS symptoms were influenced by having PTSD, we removed PTSD positive participants (relaxed criteria) and performed further explorative symptom severity analyses within the exclusively PTSD negative subgroup (n = 78). In this group, mean ± SD PCL scores again did not differ between the mTBI positive (n = 12) and negative (n = 66) participants; 42.3 ± 13.9 versus 44.3 ± 11.7, p = 0.95, d = 0.17. Within this subgroup, mean ± SD RPQ scores also did not differ between the mTBI positive and negative participants; 29.5 ± 9.7 versus 29.3 ± 13.1, p = 0.95, d = 0.02. However, we did find within this PTSD negative subgroup that mean ± SD RPQ scores trended toward higher scores among those with multiple blast exposures (n = 61) versus a single exposure (n = 17); 30.6 ± 9.6 versus 25.5 ± 11.6, p = 0.07, d = 0.51. Those with multiple blast exposures also had higher PCL scores; 45.6 ± 12.0 versus 38.3 ± 10.5, p = 0.03, d = 0.63.

Diagnostic Accuracy of Posttraumatic Stress Disorder Checklist

For the PCL total score method, the diagnostic accuracy was analyzed for both the strict (A2 inclusive) DSM-IV criteria and the relaxed (not requiring A2) DSM-IV criteria. Overall, the ROC curve’s area under the curve was similar for each of the analyses (0.83 for each).

The accuracy values and their 95 percent CI for the full range of cut points under strict criteria are shown in Table 5. At 16.0 percent PTSD prevalence and ignoring CIs, a cut point of 66 provided the peak kappa value (0.49) as well as the peak correct classification rate (87%). This cut point offered high specificity (0.93) and NPP (0.91), but lower sensitivity (0.53) and PPP (0.60). The calculated prevalence of PTSD at this cut point was near the true prevalence (14% vs 16%). The cut points that generated the most accurate prevalence rates were 64.5 (15%) and 63.5 (17%).


Table 5. 
Diagnostic accuracy of Posttraumatic Stress Disorder Checklist (PCL) using strict posttraumatic stress disorder (PTSD) criteria at PTSD prevalence of 16.0 percent.

Accuracy data with CIs across cut points using the relaxed criteria are shown in Table 6. At a true 26.4 percent PTSD prevalence, a cut point of 58 provided the peak kappa value (0.54), a fairly high classification percent (0.81), and a slight overestimate of PTSD prevalence (30.2%). At this cut point and at a 26 percent PTSD prevalence, the PCL tended to favor the minimizing of false positives (37%; 1 - PPP) over false negatives (11%; 1 - NPP). Ignoring CIs, a cut point of 66 provided the peak classification percent (0.82), but kappa (0.46) was below peak (0.54). It also generated a significant underestimate of PTSD prevalence (14.2%), such that its entire 95 percent CI (9%–22%) was below true prevalence (26.4%). Not considering CI, a cut point of 60.5 generated the most accurate prevalence rate at 25 percent.

Next, the diagnostic accuracy of the PCL using the SCM was examined. Across the three analyses (strict criteria, relaxed criteria, and strict criteria with 26.42% prevalence), classification percentage and kappa were considerably lower than had been produced by the total score method at the optimal cut points (Tables 5 and 6).


Table 6. 
Diagnostic accuracy of Posttraumatic Stress Disorder Checklist (PCL) using relaxed posttraumatic stress disorder (PTSD) criteria at PTSD prevalence of 26.4 percent.

We had intended to also analyze a combined SCM and total score scoring method, but all participants with a PCL total > 48.5 were SCM test positive except for one SCM negative subject with a total of 56 who was MINI negative. Requiring positivity on both methods would not enhance specificity because the SCM failed to correctly reclassify any of the 10 total score false positive misclassifications under relaxed criteria at its best-performing cut point. Hence, we abandoned further analysis of combining the two scoring methods.

DISCUSSION

In examining the diagnostic accuracy of the PCL, we began by calculating the true prevalence of PTSD in this high risk sample. Among these blast-exposed and predominantly mTBI positive Veterans and SMs, the prevalence rate for PTSD was 16.0 percent under strict adherence of DSM-IV criteria, but jumped to 26.4 percent with removal of the A2 criterion that stipulates having a subjective immediate response of “intense fear, helplessness, or horror.” The A2 criterion has been criticized for weak association with developing clinical PTSD and, hence, was removed in the recently released DSM-5 criteria [27]. Calhoun et al. has previously predicted that the PTSD prevalence rate would rise under DSM-5 criteria for samples that have rates below 50 percent under DSM-IV criteria [28]. This prediction was primarily based on the changes to the symptom cluster criteria rather than the A2 criteria, given that 97 percent of Calhoun et al.’s sample meeting A1 also met A2. Importantly, Calhoun et al.’s study sample was predominantly civilian, with only 15 percent reporting combat-related stressful events. Thus, our study findings of a large increase in prevalence when removing the DSM-IV A2 criterion suggest that migration to DSM-5 will have a greater effect on military and Veteran samples relative to civilian samples.

Other investigators have noted that by limiting the subjective response to immediate “intense fear, helplessness, or horror,” the A2 criterion does not adequately describe the experience of military personnel in combat who develop clinical PTSD. For example, in a sample of U.S. soldiers returning from combat deployment in Iraq, Adler et al. reported that 16 percent of those with combat-related A1 events did not also endorse the A2 criterion but nonetheless had significant PTSD symptoms warranting further clinical evaluation [29]. Our finding that removal of A2 did not change mean PCL scores for those otherwise DSM-IV positive adds to the mounting evidence that the A2 criterion lacks clinical utility in military and Veteran populations. It appears that for many military combatants, the immediate strong emotional response is suppressed and/or the total combat experience accounts for persisting PTSD symptoms rather than a specific event. Bliese et al. chose to remove A2 from their structured interview because they noted that many soldiers did not endorse A2 reactions but instead reported reactions such as “my training kicked in” or “I was angry” when asked how they reacted to combat experiences [30]. We found similar responses among our participants when queried for “other strong emotions” if negative for “intense fear, helplessness, or horror.” Thus, our findings provide further evidence that removal of the DSM-IV A2 criterion is preferred for combat experiences and that published PTSD prevalence rates are likely to climb after the adoption of DSM-5, at least as compared with studies having used strict DSM-IV criteria in similar high risk postdeployment samples.

Moreover, if military samples are affected by the symptom cluster changes to DSM-5 in a fashion similar to Calhoun et al.’s study [28], our study findings may be underestimating the projected rise in formal prevalence rates for military samples. This is because we used a DSM-IV version of structured interview and so did not assess for the influence of the DSM-5 changes to the symptom cluster criteria. In addition to changing the clusters from three to four, DSM-5 adds three new items not assessed for in the present study [31].

Interestingly, we found that a history of sustaining a blast-related mTBI did not alter the prevalence of PTSD, the severity of PTSD-like symptoms, or the severity of PCS symptoms in this blast-exposed sample. These findings should be interpreted with caution because the power of these analyses was limited by a small (n = 16) group of non-mTBI participants. The very small effect size for the PCL (d = 0.18) does suggest that even if significance were achieved with a larger sample size, any relationship of mTBI diagnosis to PTSD symptom severity appears weak. When we analyzed only participants without PTSD (n = 88), power was limited even further, but there was again no significant difference in PCL scores between mTBI diagnosis groups (d = 0.17). Regarding PCS symptoms, the small non-TBI group size has less bearing on results interpretation because persons diagnosed with mTBI had virtually identical mean RPQ scores as those without mTBI diagnosis with effect sizes that were either unmeasurable or less than 0.02.

Much has been speculated about comorbid PTSD and mTBI regarding whether one increases risk or impedes recovery of the other. Our findings do not support the hypothesis that a recent history of blast-related mTBI is a significant risk factor for PTSD or that mTBI significantly exacerbates or ameliorates PTSD symptom severity. In contrast, our data do support the hypothesis that PTSD influences PCS symptom severity because those with PTSD had higher RPQ scores than those without PTSD. Despite the small sample size, this finding was a trend (p = 0.09) under strict DSM-IV criteria and reached significance (p = 0.003) under the relaxed criteria. Because similar findings have been reported by others, including Hoge et al. [13], this underscores the importance of considering the influence of PTSD in studies of mTBI outcome in blast-exposed samples.

The lack of association found between blast-related mTBI and residual PTSD or PCS symptoms is perhaps unsurprising in view of the divisive existing literature. Findings similar to ours have been reported in several investigations [32–34], while others have shown the opposite, that historical blast-related mTBI is associated with both PCS and PTSD symptom severity [15,35]. Differences in methodology among these studies, including differing operational definitions of mTBI, make the conflicting evidence difficult to reconcile. In exploratory analyses of the PTSD negative subgroup, we did find a trend toward higher PCS scores among those with more than one blast exposure. This may reflect subclinical, or so-called subconcussive, insults to the brain from multiple blast exposures, or it may simply indicate a higher degree of postdeployment psychologically induced stress short of clinical PTSD as suggested by their higher PCL scores. Regardless, it appears further research is still needed to better elucidate what, if any, link exists between blast-related mTBI and chronic PCS or PTSD symptoms.

Regarding the diagnostic accuracy of the PCL, our results indicate that the total score method performed better than the SCM across a wide range of cut points for both relaxed and strict criteria. Overall, the diagnostic accuracy of the PCL under either set of criteria was in the range of “moderate” agreement [36] across the cut points with higher kappa values and was similar to previous studies of the PCL [5]. Which criteria were applied to the MINI affected the specific accuracy parameter values at given cut points, and therefore cut point selection was criteria dependent. The choice of the “best” PCL cut point in any population is complex and depends on which specific accuracy index(es) a researcher desires to maximize. A balanced approach values nearness of calculated prevalence to true prevalence, higher correct classification rate, and higher kappa statistic. However, the most accurate calculated prevalence, the peak classification rate, and the peak kappa typically do not occur at the same cut point. Moreover, because of sampling error, there is usually a range of cut points for which the best values (true prevalence, peak kappa, and peak classification rate) are all contained within their respective 95 percent CI. In our data, under strict criteria (requiring A2), the true prevalence was 16 percent, the peak kappa value was 0.49, the peak classification rate was 87 percent, and each occurred at different cut points. The range of cut points was 61.5–67.5, for which all three values (16%, 0.49, 87%) resided within their respective parameter’s 95 percent CI. But for practical implementation, a single cut point is desired, so we made the qualitative choice to give precedence to the kappa statistic. Within the 61.5–67.5 range, the peak kappa value of 0.49 was found at the cut point of 66. This specific cut point and range are much higher than convention and higher than the cut point of 60 found in the recent, carefully done study on the psychometric properties of the PCL by Keen et al. on a sample of 100 percent male Veterans [37]. Our higher cut point compared with Keen et al. may reflect a greater prevalence of combat experience (100% vs 65%, respectively) and/or a shorter time from exposure (mean age 26.0 ± 7.2 vs 47.4 ± 7.1, respectively). The prevalence of mTBI also may have differed but was not reported by Keen et al.

Similarly, under relaxed criteria (aligning with DSM-5 by ignoring A2), the true prevalence was 26.4 percent, the peak kappa was 0.54, and the peak classification rate was 81 percent. The range of cut points was 54.5–62.5, where these values fell within their respective 95 percent CI. Within this range, the peak kappa value occurred at a cut point of 58. Since our relaxed criteria approximate DSM-5 by also removing the A2 criterion, it is reasonable to opine that this cut point of 58 is better for extrapolating from legacy DSM-IV PCL data into DSM-5. For future prospective PCL research, PTSD symptom data will likely be collected with the just-developed DSM-5 version of the PCL, coined the PCL-5 (http://www.ptsd.va.gov/professional/assessment/adult-sr/ptsd-checklist.asp). The PCL-5 not only adds the three additional DSM-5 symptom items but also rescales the Likert ranges for each item from 1–5 to 0–4. If one were to assume that the mean score of the three additional items is equal to the mean score of the other 17 items, then the PCL-5 equivalent cut point could easily be estimated from the legacy PCL ([legacy PCL total ÷ 17 – 1] × 20 = PCL-5 total). Under this assumption, our found cut point of 58 for the legacy PCL would translate into a PCL-5 rounded cut point of 48.

As noted earlier, the decision regarding what PCL cut point is “best” depends on the context of the evaluation and which accuracy parameter(s) are desired to be maximized. As a clinical screening to indicate potential need for future evaluation and treatment, more weight should be placed on maximizing sensitivity, so a lower cut point is preferred. In this scenario, the identified cut point of 58 (employing the relaxed PTSD criteria) will have poor utility, as 29 percent of those with PTSD in our sample will be missed. A lower cut point will reduce false negatives but will also lead to increased false positives and reduced efficiency. For example, all participants with a MINI diagnosis of PTSD had a PCL score of >38.5. Thus, 100 percent of those scoring above that cut point would be detected. However, 67 percent of those screening positive would be false positives, potentially overwhelming clinical resources available for second-level PTSD assessments. The appropriate cut point threshold employed as part of a clinical screening program [38] is a decision that must weigh resources, available treatments, ethical considerations, stakeholder acceptability, and other important factors.

When the PCL is used to detect only those with a high likelihood of PTSD, a clinical researcher may prefer a higher cut point to minimize false positives (i.e., maximize PPP) and reduce the costs of clinical interviews needed to confirm diagnosis. The cut point one chooses will depend on the relative cost of screening to the second-level interview evaluations, the size of the population available for screening, and the desired number of study enrollees. Similarly, when using existent PCL data to group persons into probable versus unlikely PTSD diagnosis on strict DSM-IV criteria for research purposes, the threshold for misdiagnosis and uncertainty will depend on a study’s objective. Investigators are encouraged to consult the literature and consider the potential benefits and pitfalls of using the PCL as an indicator of probable PTSD when developing new research. We have provided data on the diagnostic accuracy parameters across the entire spectrum of cut points to enable choosing the parameter weightings that best fit the specific purpose. The specific qualitatively “best” cut points that we identified may be useful as clinical research thresholds because we balanced parameters such that the vast majority of those classified as positive and negative will be true positives and true negatives, respectively.

Although this study’s diagnostic accuracy results support the utility of the PCL as a screening tool for probable PTSD for blast-exposed individuals, they may not generalize to the postdeployment population at large in which there are widely varying deployment time periods, geographic locations, and duty assignments. Our findings are more likely to generalize to populations that have a similarly elevated risk of TBI and PCS, such as Veterans and SMs with positive initial TBI screens, and are a similar time frame postdeployment. Our sample had demographic characteristics similar to other reports on PTSD from the blast exposed OIF/OEF/OND population (Kontos et al. [15]: 96% male and mean age of 29.5 ± 6.8 yr; Adler et al. [29]: 98% male; 48% married, 45% single, 7% divorced). Like these studies, ours may not generalize to females. Also noteworthy is that we did not include individuals with moderate-severe TBI, who are generally accepted to be more protected from PTSD, given longer periods of retrograde and anterograde amnesia.

One weakness of this study is sample size, which resulted in fairly large 95 percent CIs for diagnostic accuracy parameters. CIs and effect sizes where relevant are provided to assist readers in making their own interpretation. The nature of the sample offers both strengths and weaknesses. It fills an important research gap in that there are sparse empirical data on diagnostic accuracy of the PCL in military blast-exposed populations at high risk for PTSD and TBI. Conversely, the results may not generalize to noncombat, stressful event exposures or to postdeployment SMs and Veterans without blast exposure. Additionally, we enrolled participants from a narrow geographical region and the characteristics of blast exposure may differ from those who returned postdeployment to other regions. The sample was almost entirely male, so results may not generalize to females. Another potential weakness is that by using the DSM-IV-based MINI and PCL available at the time of study implementation, we could only approximate the future adoption of the DSM-5 criteria by removing the A2 criteria. But this study’s findings should remain useful to researchers and others, given the plethora of research and administrative data both historical and still being collected using the existing DSM-IV version of the PCL.

CONCLUSIONS

Among these blast-exposed SMs and Veterans, the prevalence rate for PTSD using DSM-IV criteria depended on whether the A2 criterion (i.e., the immediate subjective response to a traumatic event consisting of “intense fear, helplessness, or horror”) was included. The prevalence was higher when using the DSM-5-like relaxed criteria than when strictly adhering to DSM-IV and enforcing A2. PTSD prevalence rates did not differ between participants with and without blast-related mTBI under either set of criteria. Regarding symptoms, participants with PTSD had not only higher PCL scores, but also higher RPQ scores than those without PTSD. Participants with mTBI were indistinguishable from those without TBI on symptom scores, although power was limited by the small non-TBI group size. Regarding diagnostic accuracy of the PCL, the total score method performed better than the SCM. Using a balanced approach among the accuracy parameters and their 95 percent CI, the best cut point range was 61.5–67.5 under strict criteria and 54.5–62.5 under relaxed criteria. Within this range, the peak kappa value was at cut points of 66 and 58, respectively. These finding should be useful for studies in this population that rely on the legacy DSM-IV-derived PCL instrument to categorize persons by way of PTSD diagnosis.

ACKNOWLEDGMENTS
Author Contributions:
Study concept and design: W. C. Walker, S. D. McDonald.
Analysis and interpretation of data: W. C. Walker, S. D. McDonald, L. M. Franke.
Drafting of manuscript: W. C. Walker, S. D. McDonald, L. M. Franke.
Financial Disclosures: The authors have declared that no competing interests exist.
Funding/Support: This material was based on work supported by the Congressionally Directed Medical Research Programs (award W91ZSQ8118N620) with some support from the Defense and Veterans Brain Injury Center. This material was also based on work supported in part by a VA, VHA, Office of Research and Development, and Rehabilitation Research and Development Career Development Award-2 (1IK2RX000703–01 to S. M.).
Institutional Review: All appropriate institutional review board and governmental approvals were obtained.
Participant Follow-Up: The authors do not plan to inform participants of the publication of this study.
Disclaimer: The views expressed are those of the authors and do not necessarily represent or reflect the policy or position of the Department of the Army, DoD, U.S. Government, VA, or any of the institutions with which the authors are affiliated.
REFERENCES
This article and any supplementary material should be cited as follows:
Walker WC, McDonald SD, Franke LM. Diagnostic accuracy of Posttraumatic Stress Disorder Checklist in blast-exposed military personnel. J Rehabil Res Dev. 2014;51(8):1203–16.
http://dx.doi.org/10.1682/JRRD.2013.12.0271
ResearcherID/ORCID: William C. Walker, MD: N-3162-2014; Scott D. McDonald, PhD: N-3799-2014; Laura Manning Franke, PhD: F-2670-2012
iThenticateCrossref

Go to TOP

Last Reviewed or Updated  Wednesday, January 14, 2015 2:29 PM

Valid HTML 4.01 Transitional