Journal of Rehabilitation Research & Development (JRRD)

Quick Links

  • Health Programs
  • Protect your health
  • Learn more: A-Z Health
Veterans Crisis Line Badge
 

Volume 48 Number 2, 2011
   Pages 89 — 102

Measurement of community reintegration in sample of severely wounded servicemembers

Linda Resnik, PT, PhD;1-2* Melissa Gray, MOT, OTR/L;3 Matthew Borgia, BS2

1Providence Department of Veterans Affairs Medical Center, Providence, RI; 2Department of Community Health, Brown University, Providence, RI; 3Brooke Army Medical Center, San Antonio, TX

Abstract — The Community Reintegration of Servicemembers (CRIS) is a new measure of community reintegration. The purpose of this study was to test the CRIS with seriously injured combat veterans. Subjects were 68 patients at the Center for the Intrepid. Each patient completed three CRIS subscales, the 36-Item Short Form Health Survey for Veterans (SF-36V), the Quality of Life Scale (QOLS), and two Craig Handicap Assessment and Reporting Technique subscales at visit 1 and the 3-month follow-up. Of the patients, 11 also completed the measures within 2 weeks of visit 1. We abstracted diagnoses and activities of daily living from the medical record. We evaluated test-retest reliability using intraclass correlation coefficients (ICCs). We evaluated concurrent validity with Pearson product moment correlations. We used multivariate analyses of variance to compare scores for subjects with and without posttraumatic stress disorder (PTSD), traumatic brain injury (TBI), and depression. Responsiveness analyses evaluated floor and ceiling effects, percent achieving minimal detectable change (MDC), effect size (ES), and the standardized response mean (SRM). CRIS subscale ICCs were 0.90 to 0.91. All subscales were moderately or strongly correlated with QOLS and SF-36V subscales. CRIS subscale scores were lower in PTSD and TBI groups (p < 0.05). CRIS Extent of Participation and Satisfaction with Participation subscales were lower for subjects with depression (p < 0.05). Of the sample, 17.4% to 23.2% had change greater than MDC. The ES ranged from 0.227 to 0.273 (SRM = 0.277-0.370), showing a small effect between visit 1 and the 3-month follow-up. Results suggest that the CRIS is a psychometrically sound choice for community reintegration measurement in severely wounded servicemembers.

Key words: community reintegration, disability, measurement, military healthcare, outcomes assessment, participation, psychometric testing, reliability, traumatic brain injury, veterans.

Abbreviations: ADL = activity of daily living, ANOVA = analysis of variance, BAMC = Brooke Army Medical Center, CFI = Center for the Intrepid, CHART = Craig Handicap Assessment and Reporting Technique, CRIS = Community Reintegration of Servicemembers, ES = effect size, ICC = intraclass correlation coefficient, ICF = International Classification of Function, IED = improvised explosive device, MANOVA = multivariate analysis of variance, MDC = minimal detectable change, OEF = Operation Enduring Freedom, OIF = Operation Iraqi Freedom, PF-10 = 10-Item Physical Functioning Subscale, PTSD = posttraumatic stress disorder, QOLS = Quality of Life Scale, SD = standard deviation, SF-36V = 36-Item Short Form Health Survey for Veterans, SRM = standardized response mean, TBI = traumatic brain injury, VA = Department of Veterans Affairs.
*Address all correspondence to Linda Resnik, PT, PhD; Providence VA Medical Center, 830 Chalkstone Ave, Providence, RI 02908; 401-273-7100, ext 2368; fax: 401-863-3489. Email: Linda_Resnik@brown.edu
DOI:10.1682/JRRD.2010.04.0070
INTRODUCTION

Evidence to date suggests that demobilization and return home after combat can be challenging for military servicemembers. Numerous reintegration problems have been reported among veterans from the gulf war and more recent conflicts in Iraq and Afghanistan, including marital difficulties, financial difficulties, problems with alcohol or substance abuse, medical problems, behavioral problems such as depression or anxiety [1], homelessness [2], and motor vehicle accidents [3]. Readjustment to community living is likely to be especially challenging for servicemembers who are injured, as readjustment may be complicated by the co-occurrence of physical injuries and postwar adjustment difficulties such as posttraumatic stress disorder (PTSD), depression, substance abuse, and severe mental illness [1,4].

Although the survival rate for servicemembers injured in recent conflicts is far greater than that of previous conflicts, the improved survivability is associated with an increased rate of servicemembers with severe injuries that include head injuries, burns, and extensive injuries to the limbs. Improvised explosive devices (IEDs) are the cause of a majority of these injuries [5]. As of May 2010, over 31,800 U.S. servicemembers have been wounded in Operation Iraqi Freedom (OIF) and Operation Enduring Freedom (OEF) [6]. Injuries caused by IEDs are associated with the unusually high prevalence of traumatic brain injury (TBI) [7] and PTSD among the injured [8-9], conditions that are likely to present substantial challenges in reintegrating into community roles. Data suggests that OIF/OEF service will negatively affect a far greater number of persons beyond those counted in the combat casualty statistics, with upwards of 790,000 veterans expected to seek disability benefits for service-related health problems [10]. Society's understanding of the effects of poor postdeployment reintegration stems largely from the experience of Vietnam war veterans, a disproportionate number of whom suffer from chronic PTSD and pervasive difficulties in their everyday lives, including marital and work difficulties, poor parenting skills, violent behavior, alcohol and drug abuse, involvement with the criminal justice system, suicide attempts, and homelessness [11-16]. More than one-third of homeless men in the United States are veterans [17], with an estimated 250,000 veterans homeless on a given night and more than 500,000 homeless over the course of a year [16]. Given what is known about the experiences of veterans from previous wars, it is imperative that we find ways to assess the community reintegration of today's combat veterans and that we intervene early to prevent long-term consequences for returning servicemembers, their families, and society.

To date, no systematic efforts have estimated the scope of these problems. At present, neither Department of Veterans Affairs (VA) nor Department of Defense electronic medical records contain standardized data elements related to community reintegration. Enhanced clinical information systems are a key component of improving care delivery for patients with chronic and complex conditions. Routine assessment of community reintegration would enhance patient assessment and referral targeting to mental health, social services, and benefit programs as well as drive interventions that address underlying factors related to poor community reintegration [18].

The Community Reintegration of Servicemembers (CRIS) is a new measure of community reintegration developed to be a veteran-specific outcome measure. Initial content of the CRIS was identified through formative research on OIF/OEF veterans, caregivers, and clinical experts and a comprehensive review of concepts and content of existing measures [18]. CRIS development was based on the conceptual framework of the World Health Organization's International Classification of Function (ICF). Its three subscales measure nine domains of participation and three dimensions: objective and subjective aspects of participation as well as satisfaction with participation.

After initial development, cognitive-based testing, and refinement of the instrument, pilot studies with 126 veterans seeking care at the Providence VA Medical Center (Providence, Rhode Island) were conducted to examine unidimensionality, internal consistency, reliability, and construct validity of the CRIS subscales. The three CRIS subscales demonstrated strong reliability, conceptual integrity, and construct validity in convenience samples of veterans from the Providence VA Medical Center [18]. Our earlier preliminary analysis showed that the subscales were unidimensional and Rasch models predicted the majority of variance in the data for each subscale (Extent of Participation = 0.53, Perceived Limitations = 85.2, Satisfaction with Participation = 73.3) [18]. A subset of items was selected from the larger CRIS item sets to form the CRIS Fixed Form Scales (henceforth called the CRIS). Alphas for the scales were Extent of Participation = 0.91, Perceived Limitations = 0.93, and Satisfaction with Participation = 0.97. Tests revealed that these subscales predicted between 0.97 and 0.98 of the variance of the larger item sets. These findings suggested that the CRIS possessed the psychometric properties that would enable it to be used as a standardized assessment measure for the monitoring of community reintegration outcomes of injured servicemembers from recent conflicts and that it may have usefulness in monitoring outcomes at the individual patient level as a means of evaluating outcomes of therapeutic services.

Outcome measures used at the individual patient level must meet several essential psychometric standards. First, they must be highly reproducible and have a small standard error of measurement; in other words, they must have excellent test-retest reliability. Second, outcome measures must be valid indicators of the constructs they are hypothesized to represent. Third, they must show sensitivity to clinical change and have adequate scale range so that they can be used to detect changes in scores when they occur; in other words, they must exhibit minimal floor and ceiling effects. Fourth, outcome measures used to assess changes resulting from therapeutic interventions must also be responsive to change; in other words, they must be able to detect change when it happens.

Because psychometric properties like reliability and validity are application- and population-specific, rather than inherent attributes of a measure [19], additional studies are needed to examine psychometric properties of the CRIS in a younger injured combat veteran sample. Prior to the current study, the psychometric properties of the CRIS had not been examined in younger samples or those with more severe injuries. The majority of subjects in our initial pilot studies were 35 years old, and few, if any, had sustained severe combat-related injuries. Thus, the purpose of this study was to conduct psychometric testing of the CRIS using a sample of injured combat veterans (severely injured servicemembers) who are undergoing rehabilitation at the Center for the Intrepid (CFI) at Fort Sam Houston, San Antonio, Texas.

METHODS
Setting

We conducted this research study at the CFI, a service under the Department of Orthopedics and Rehabilitation at Brooke Army Medical Center (BAMC), San Antonio, Texas. The CFI provides interdisciplinary rehabilitation for servicemembers with a wide range of polytraumatic injuries ranging from upper- and lower-limb amputation to limb reconstruction, burns, TBI, and mental health issues [20]. The resources and services provided by the CFI have been previously described by Yancosek et al. [20]. The CFI is staffed by uniformed medical providers from the U.S. Army, Department of Army civilians, employees of the Veterans Health Administration and Veterans Benefits Administration, and contract providers. The clinical team includes physicians, occupational therapists, physical therapists, prosthetists, behavioral medicine providers, social workers, case managers, a recreation therapist, and vocational rehabilitation personnel. The rehabilitation team coordinates care to maximize the injured servicemember's return to duty and to community living.

Patients come to the CFI for rehabilitation after upper- and/or lower-limb amputation and severe limb trauma as well as serious burns. Other individuals who need specific training in one of the above areas can be referred for specialty training on an individual basis. The proximity of the CFI to BAMC allows individuals who have sustained an amputation or other injury to start their care at the CFI as soon as it is appropriate while they are still inpatients at BAMC. Once patients are discharged from BAMC, they continue their rehabilitative care at the CFI on an outpatient basis. Individuals with burns are referred to the CFI after they are able to tolerate advanced activities of daily living (ADLs) training.

Sample

We used a convenience sample of servicemembers who were patients at the CFI. All patients treated at the CFI who were able to provide informed consent were eligible to participate.

Data Collection

We administered study questionnaires to all subjects on two occasions: visit 1 and at least 3 months after visit 1. We chose a follow-up time period of at least 3 months because we were uncertain how long a time period would be necessary to realize improvement in community reintegration during treatment at the CFI. A trained research assistant who read each question aloud and recorded the subjects' responses administered all measures. We included 11 of these subjects in the pilot reliability portion of this study and administered the CRIS to them on an additional occasion within 2 weeks of visit 1. We collected demographic and diagnostic data at visit 1 by interview and from abstraction from the medical record.

Outcome Measures

At visit 1 and the 3-month follow-up, we administered the three CRIS subscales (Extent of Participation, Perceived Limitations, and Satisfaction with Participation) and four other measures of social-role function to all subjects: the 36-Item Short Form Health Survey for Veterans (SF-36V) Role Function subscales [21] and the Craig Handicap Assessment and Reporting Technique (CHART) Social Integration and Occupational Functioning subscales.

Community Reintegration of Servicemembers Subscales

The CRIS used in this study was comprised of three separate subscales (Appendix (pdf), available online only). Questions on these subscales pertain to each of the nine domains of activities and participation as defined by the ICF [18], with specific content areas identified. In summary, CRIS items relate to acquiring complex skills; focusing attention; solving problems; reading; undertaking multiple tasks; carrying out a daily routine; handling stress; communicating and conversing; moving around in different locations; driving and using transportation; initiating self-care activities and health maintenance; preparing meals; doing housework and caring for household objects and children; maintaining basic and complex interpersonal relationships, relationships with family members, and intimate relationships; acquiring, keeping, and terminating a job; making complex economic transactions; maintaining economic self-sufficiency; undertaking recreation and leisure; socializing; and maintaining citizenship and a political life.

The Extent of Participation subscale is a 50-item scale that asks how often an individual experiences or participates in specific activities. Items use 7-point scales that indicate number of times per week or other frequency of occurrence (not at all, very often, etc.). The 54-item Perceived Limitations subscale uses two different 7-point response scales; the first indicates the magnitude of perceived limitations and the second asks the respondent to agree or disagree with specific statements about the amount of limitation that they have. The 47-item Satisfaction with Participation subscale asks about satisfaction with different aspects of participation and uses a 7-point response scale that ranges from "terrible" to "very happy."

Craig Handicap Assessment and Reporting Technique

The CHART Social Integration subscale consists of six questions about extent of participation in and maintenance of customary social relationships [22]. The CHART Occupational Functioning subscale consists of seven questions about extent of participation in occupational activities customary to a person's sex, age, and culture [22].

36-Item Short Form Health Survey for Veterans

We used four subscales of the SF-36V [23]. The Role Physical subscale uses 4-items that measure difficulty with role function in work or ADLs attributable to physical health problems [18,24]. The 3-item Role Emotional subscale measures difficulty with role function in work or ADLs attributable to mental health problems. The 2-item Social Functioning subscale measures interference with social activities related to physical and emotional problems [18,24]. The 10-Item Physical Functioning subscale (PF-10) measures difficulty with performance of physical activities [18,24].

Quality of Life Scale

The Quality of Life Scale (QOLS) consists of 16 questions that assess satisfaction with independent living and self-care activities [23].

Activities of Daily Living

We abstracted data on difficulty performing ADLs (walking, bathing, dressing, eating, transferring, and toileting) from therapy notes in the medical records. Therapists observe ADL performance and note whether or not the patient has difficulty with each particular activity. We included a count of the number of ADLs that the therapist observed that the patient had difficulty performing. We used ADL difficulty count as a measure of discriminant validity.

Statistical Analyses

We generated descriptive statistics for those subjects who completed both visit 1 and the 3-month follow-up and those subjects lost to follow-up and reported them in the appropriate metric for continuous and categorical variables.

Reliability Analysis

We included 11 subjects in the pilot reliability portion of this study (reliability group). We needed this pilot data because the CRIS has not previously been administered in a severely injured population and analyses are needed to ensure that all scales are reliable in this sample. We administered the CRIS on three occasions to subjects in the reliability portion of the study: visit 1, within 2 weeks of visit 1, and at the 3-month follow-up.

We used test-retest data (on visit 1 and the 2-week follow-up) from the reliability group to examine test-retest reliability using the Shrout and Fleiss (type 2,1) intraclass correlation coefficient (ICC), which is generally denoted by ICC (2,1) [18,25]. ICC (2,1) is a two-way mixed effects, single-measure reliability, where the target is a random effect, the number of measurements on each target is a fixed effect, and the unit of analysis is the individual measurement instead of the mean of measurements [18].

We used the coefficients from the ICC to calculate the minimal detectable change (MDC) at 90 and 95 percent confidence levels using the following formula: MDC 95% = [z score for 95% confidence level)] × [SD at visit 1] × [square root of (2 [1 - [r (i.e., ICC)], where SD = standard deviation, r = correlation, and z = a measure of distance in SDs of a sample of the mean. MDC is a statistical measure of meaningful change, defined as the minimum amount of change that exceeds measurement error. The resulting MDC 95% value, for example, is a change score (MDC) in the units of the scale that only 5 percent of stable patients are expected to exceed (either positively or negatively). A score larger than the MDC indicates that a true change has occurred, because there is a less than 10 percent chance that the patient is from a distribution of stable patients.

Concurrent and Discriminant Validity

We evaluated concurrent validity of the CRIS by exploring the Pearson product moment correlations of the CRIS with existing measures that assess specific community reintegration dimensions. We evaluated discriminant validity of the CRIS by exploring correlations of the CRIS with measures assessing differing constructs, including the SF-36V (PF-10) and the count of ADL difficulties. We used Cohen's values of correlation to interpret the strength of correlation coefficients as weak (<0.3), moderate (0.3-0.5), and strong (>0.5) [26].

Known-Group Validity

We examined differences in scale scores for several subgroups of patients that we expected to have differing scores: those with PTSD compared with those without, those with a diagnosis of depression and those without, those undergoing treatment for TBI and those both not undergoing TBI treatment and those without TBI. Prior research has reported that these conditions negatively affect interpersonal relationships, concentration, and social function [27-33]. Because the CRIS assesses these domains, we expected to see lower CRIS scores for those with PTSD, with depression, and undergoing treatment for TBI. We performed separate multivariate analyses of variance (MANOVAs) for each of the conditions using the three separate dependent variables of the CRIS subscales and examined Wilks Lamda multivariate test of overall differences among subjects and univariate between-subjects statistics.

Responsiveness

We examined responsiveness of the CRIS several ways. First, we assessed the extent of floor and ceiling effects by examining the distribution of scores for each CRIS subscale, observing the shape and presence of score clustering. We calculated the percentage of the sample achieving scores that range within the MDC of the lowest (floor effect) and highest (ceiling effect) score for each subscale. We considered floor and ceiling effects lower than 15 percent acceptable [19]. Next, we calculated the percentage of subjects who had change scores greater than the MDC.

We examined differences between visit 1 and the 3-month follow-up scores by conducting separate paired t-tests for each of the CRIS scores. We used Bonferroni post hoc analyses to account for multiple t-tests. Lastly, we calculated effect sizes (ESs) and standardized response means (SRMs) for each of the three outcomes measurements [34]. Both of these change coefficients provide a standardized measurement of change and aid interpretation [35]. While ES may be more commonly used, its estimate may be affected by the number of subjects in the sample. We decided to examine both ES and SRM because SRM is not influenced by sample size, and thus may be preferred [34]. We obtained ES by dividing the average change between initial and follow-up measurements by the SD of the initial measurement instrument. The following interpretations are commonly used for ES: small (0.2-0.4), moderate (0.5-0.7), and large (Š0.8) [35]. We calculated SRM by dividing the average change by the SD of the change scores. We compared the ES and SRM for the CRIS with those of the measures used in our concurrent validity examination.

RESULTS
Descriptives

We recruited 74 eligible subjects into the study and tested them at visit 1. Of the subjects, 68 (92%) completed the study and 6 (8%) were lost to follow-up. Table 1 shows the characteristics of the subjects who completed the study as well as those who were lost to follow-up. Generally, those lost to follow-up had a slightly higher mean age, a longer mean time since deployment, and fewer ADL difficulties than those who completed the study. Of those who completed the study, 94.1 percent were male, 42.6 percent were married, and 76.5 percent identified themselves as white. CRIS subscale scores at the 3-month follow-up for those who completed the study were higher than scores at visit 1 for all subscales.


Table 1.
Characteristics of subjects completing study (n = 68) and those lost to 3-month follow-up (n = 6).

Characteristic
Complete
Lost

Age, yr (mean ± SD)
27.1 ± 5.6
27.8 ± 7.6
Return from Deployment, mo (mean ± SD)
15.8 ± 15.0
26.3 ± 19.8
Onset of Injury, d (mean ± SD)
397.6 ± 270.6
1,487.5 ± 1,489.5
From Start at CFI to Visit 1, d (mean ± SD)
213.2 ± 203.5
233.5 ± 245.0
From Visit 1 to Follow-Up, d (mean ± SD)
142.0 ± 61.0
-
No. of ADL Difficulties (mean ± SD)
1.1 ± 2.1
0.33 ± 0.52
CRIS Visit 1 (mean ± SD)
   
 
Extent of Participation
54 ± 6
56 ± 7
 
Perceived Limitations
56 ± 8
60 ± 10
 
Satisfaction with Participation
58 ± 7
60 ± 8
CRIS Follow-Up
   
 
Extent of Participation
56 ± 7
-
 
Perceived Limitations
58 ± 8
-
 
Satisfaction with Participation
59 ± 7
-
Race, n (%)
   
 
White
52 (76.5)
6 (100.0)
 
Black
3 (4.41)
0 (0.0)
 
Other
14 (20.6)
0 (0.0)
Hispanic, n (%)
15 (22.1)
2 (33.3)
Male, n (%)
64 (94.1)
6 (100.0)
Marital Status, n (%)
   
 
Married
29 (42.6)
3 (50.0)
 
Unmarried
29 (42.6)
3 (50.0)
 
Separated/Divorced
10 (14.7)
0 (0.0)
Has Children, n (%)
26 (38.2)
2 (33.3)
Employment, n (%)
   
 
Full-Time
21 (30.9)
2 (33.3)
 
Part-Time
10 (14.7)
3 (50.0)
 
Not Working Due to Disability
36 (52.9)
0 (0.0)
Receives Disability Benefits, n (%)
47 (69.1)
3 (50.0)
Has a Nonmedical Assistant, n (%)
16 (23.5)
0 (0.0)
ADL Difficulty, n (%)
   
 
Bathing
12 (17.6)
0 (0.0)
 
Dressing
12 (17.6)
0 (0.0)
 
Eating
6 (8.8)
0 (0.0)
 
Getting out of Bed
4 (5.9)
0 (0.0)
 
Walking
12 (17.6)
2 (33.3)
 
Toileting
12 (17.7)
0 (0.0)
 
Getting Outside
4 (5.9)
0 (0.0)
 
Grooming
10 (14.7)
0 (0.0)
Education, n (%)
   
 
High School or GED
30 (44.1)
3 (50.0)
 
Some College/College
36 (52.9)
3 (50.0)
 
Postgraduate
2 (2.9)
0 (0.0)
Medical Condition, n (%)
   
 
TBI
27 (39.7)
1 (16.7)
 
Burn
30 (44.1)
2 (33.3)
 
Infection
13 (19.1)
1 (16.7)
 
Nerve Problem
52 (76.5)
3 (50.0)
 
Sensory Impairment
36 (52.9)
2 (33.3)
 
UL Amputation
6 (8.8)
1 (16.7)
 
LL Amputation
27 (39.7)
3 (50.0)
Mental Health Condition, n (%)
   
 
PTSD
28 (41.2)
2 (33.3)
 
Depression
12 (17.6)
0 (0.0)
 
Other
47 (69.1)
3 (50.0)

ADL = activity of daily living, CFI = Center for the Intrepid, CRIS = Community Reintegration of Servicemembers, GED = general equivalency diploma, LL = lower limb, PTSD = posttraumatic stress disorder, SD = standard deviation, TBI = traumatic brain injury, UL = upper limb.


Reliability Analysis

We calculated ICCs using data from visit 1 and the 3-month follow-up for each of the CRIS subscale scores. The ICCs were 0.91, 0.90, and 0.90 for the Extent of Participation, Perceived Limitations, and Satisfaction with Participation CRIS subscales, respectively. Table 2 shows the MDC calculated for both the 90 and 95 percent confidence levels.


Table 2.
Calculated intraclass correlation coefficients (ICCs) and minimal detectable changes (MDCs) for each Community Reintegration of Servicemembers (CRIS) subscale score.

CRIS Subscale
ICC
MDC
90%
MDC
95%

Extent of Participation
0.91
4.74
5.68
Perceived Limitations
0.90
5.79
6.93
Satisfaction with Participation
0.90
4.85
5.81


Concurrent and Discriminant Validity

Table 3 shows the Pearson product moment correlation for the CRIS subscales and the QOLS, CHART, SF-36V, and ADL measures. We correlated all CRIS subscales with QOLS, with the Satisfaction with Participation subscale the most strongly correlated (r = 0.79). CRIS Perceived Limitations and Satisfaction with Participation subscale scores had a negative correlation with number of ADL difficulties (r = -0.24 and -0.25, respectively). No CRIS subscale was correlated with the CHART Occupational Function subscale; however, the CRIS Satisfaction with Participation subscale was correlated with the CHART Social Integration subscale (r = 0.26).


Table 3.
Concurrent and discriminant validity of Community Reintegration of Servicemembers subscales: Pearson product moment correlations.

Measure
Extent of Participation
 
Perceived Limitations
 
Satisfaction with Participation
r
p-Value
 
r
p-Value
 
r
p-Value

Quality of Life Scale
0.57
<0.001
 
0.67
<0.001
 
0.79
<0.001
CHART
               
 
Occupational Function
-0.04
0.721
 
-0.12
0.314
 
-0.12
0.310
 
Social Integration
0.17
0.150
 
0.22
0.064
 
0.26
0.025
SF-36V
               
 
Role Physical
0.33
0.005
 
0.26
0.028
 
0.36
0.001
 
Role Emotional
0.54
<0.001
 
0.36
0.002
 
0.45
0.001
 
Social Function
0.48
<0.001
 
0.50
<0.001
 
0.54
<0.001
 
Physical Function
0.33
0.004
 
0.40
0.001
 
0.37
0.001
Activities of Daily Living
-0.18
0.118
 
-0.24
0.042
 
-0.25
0.034

CHART = Craig Handicap Assessment and Reporting Technique, SF-36V = 36-Item Short Form Health Survey for Veterans.


Known-Group Validity

Table 4 provides the mean ± SD and results of MANOVA tests on the CRIS for the groups of subjects with or without PTSD, with or without depression, or undergoing or not undergoing treatment for TBI on admission. The Wilks Lambda multivariate test of overall differences among groups with or without a PTSD diagnosis was statistically significant (p = 0.008). Further, univariate between-subjects tests showed that PTSD diagnosis was significantly related to the CRIS Extent of Participation (p = 0.004), Perceived Limitations (p = 0.01), and Satisfaction with Participation (p = 0.001) subscales. The Wilks Lambda multivariate test of overall differences among groups with or without a depression diagnosis was statistically significant (p = 0.02). Univariate between-subjects tests showed that diagnosis of depression was significantly related to the CRIS Extent of Participation (p = 0.045) and Satisfaction with Participation (p = 0.046) subscales, but not related to the Perceived Limitations subscale (p = 0.11). Lastly, the Wilks Lambda multivariate test of overall differences among groups with or without treatment for TBI was statistically significant (p = 0.03). Univariate between-subjects tests showed that TBI treatment was significantly related to the CRIS Extent of Participation (p = 0.02), Perceived Limitations (p = 0.02), and Satisfaction with Participation (p = 0.03) subscales.


Table 4.
Descriptive statistics and multivariate analysis of variance results by known groups of Community Reintegration of Servicemembers subscale scores.

Result
Extent of Participation
 
Perceived Limitations
 
Satisfaction with Participation
Mean ± SD
p-Value
 
Mean ± SD
p-Value
 
Mean ± SD
p-Value

PTSD Test
 
0.004
   
0.01
   
0.001
 
Positive
52 ± 6
   
53 ± 7
   
54 ± 6
 
 
Negative
55 ± 5
   
58 ± 8
   
60 ± 6
 
TBI Treatment
 
0.02
   
0.02
   
0.03
 
Yes
49 ± 7
   
49 ± 7
   
52 ± 8
 
 
No
55 ± 5
   
57 ± 8
   
58 ± 6
 
Depression
 
0.045
   
0.11
   
0.046
 
Positive
51 ± 5
   
53 ± 7
   
54 ± 5
 
 
Negative
55 ± 6
   
57 ± 8
   
58 ± 7
 

PTSD = posttraumatic stress disorder, SD = standard deviation, TBI = traumatic brain injury.


Responsiveness

Figure 1 shows the histograms for the CRIS Extent of Participation, Perceived Limitations, and Satisfaction with Participation subscale scores at visit 1. Table 5 shows the percentages of scores within the 95 and 90 percent confidence level MDCs of the floor and ceiling effects. The ceiling effect using the MDC 90% was acceptable (<15%) for Extent of Participation and Perceived Limitations subscales, but was 16.2 percent for the Satisfaction with Participation subscale. The ceiling effect using MDC 95% was 22.1 percent for the Perceived Limitations subscale and 16.2 percent for the Satisfaction with Participation subscale. The percentage of subjects whose scores improved more than the MDC 95% (Figure 2) ranged from 17.4 to 23.2 percent for the three CRIS subscales. We noted no floor effects for any subscale using either MDC 90% or MDC 95%.


Figure 1. Histograms of Community Reintegration of Servicemembers subscale scores at visit 1.

Figure 1. Click Image to Enlarge

View as PowerPoint Slide


Table 5.
Prevalence of floor and ceiling effects.

MDC
Extent of Participation
 
Perceived Limitations
 
Satisfaction with Participation
MDC
95%
MDC
90%
 
MDC
95%
MDC
90%
 
MDC
95%
MDC
90%

Floor
0
0
 
0
0
 
0
0
Ceiling
2.94
1.47
 
22.1
14.7
 
16.2
16.2

MDC = minimal detectable change.



Figure 2. Percent of sample with changes greater than minimal detectable change (MDC) of Community Reintegration of Servicemembers(CRIS) measure.

Figure 2. Click Image to Enlarge

View as PowerPoint Slide

Results of the paired t-tests and Bonferroni post hoc analyses showed that scores on all three CRIS subscales increased significantly between visit 1 and the 3-month follow-up (Extent of Participation: p = 0.002, Perceived Limitations: p = 0.002, and Satisfaction with Participation: p = 0.01). For a Bonferroni correction on three tests with a desired significance level of 0.05, significant p-values should be below 0.05/3 = 0.01.

Evaluation of ES and SRM (Table 6) showed a small effect between visit 1 and the 3-month follow-up. The magnitude of ES and SRM of the CRIS subscales were comparable in size to those of the other measures used for concurrent validation (Table 7).


Table 6.
Effect size (ES) and standardized response mean (SRM) of Community Reintegration of Servicemembers (CRIS) subscales.

CRIS Subscales
ES
SRM

Extent of Participation
0.264
0.356
Perceived Limitations
0.273
0.370
Satisfaction with Participation
0.227
0.277



Table 7.
Effect size (ES) and standardized response mean (SRM) of measures used in concurrent validation.

Measure
ES
SRM

QOLS
0.33
0.39
CHART
   
 
Social Integration
0.06
0.07
 
Occupational Function
0.09
0.08
SF-36V
   
 
Role Physical
0.36
0.37
 
Role Emotional
0.10
0.11
 
Social Functioning
0.03
0.03

CHART = Craig Handicap Assessment and Recording Technique, QOLS = Quality of Life Scale, SF-36V = 36-Item Short Form Health Survey for Veterans.


DISCUSSION

Our results confirm that the CRIS subscales are reliable for this sample. The distribution of scores shows that there were minimal issues with floor or ceiling effects in this severely injured cohort (as gauged by the MDC 90% confidence levels). There was a small ceiling effect apparent in the Satisfaction with Participation subscale, which may, in future revision, benefit from the addition of more difficult items to decrease the clustering of scores near the upper end of the scale range.

The correlations that we observed between the CRIS and other measures were, for the most part, similar to those found in our pilot analyses. Results of concurrent validity analyses demonstrated that the CRIS was moderately to strongly associated with QOLS and SF-36V measuring social, emotional, and physical functioning and weakly associated with number of ADL difficulties. Our findings contrast with our earlier analyses that show a weak correlation between the CRIS subscales and the CHART Occupational Functioning subscale (the current study showed no relationship). Prior studies showed no relationship between the CHART Social Integration subscale and CRIS subscales, while we observed a weak relationship in the current study. These findings also differ slightly from our earlier analyses that showed only a weak association

between the Satisfaction with Participation subscale and the PF-10. It is our hypothesis that the stronger association shown in this sample may be attributable to the younger age and more severe injuries of this cohort. It is possible that younger, previously highly fit persons (such as those military servicemembers in our sample) are less satisfied with participating in role functions with concomitant physical disabilities as compared with their elder counterparts who may have lower expectations for their physical functioning. We used Cohen's interpretation of magnitude of correlation size [26] in comparing the results of current and prior analyses. However, we recognize that other authors interpret the magnitude of correlation coefficients differently than Cohen.

We observed that the CRIS was responsive to change over the course of this study, but that the magnitude of the observed effect was small. We found that SRM scores were slightly higher than ES scores. This finding indicates that the change scores and SD of change scores (used in calculating the SRM) were more homogenous than the change scores and SD of the visit 1 scores (used in calculating the ES).

The ESs and SRMs that we observed were comparable to those of other measures used for concurrent validation, indicating that overall the increase in social role function in this sample was present but small.

We were surprised at the relatively high visit 1 scores for this group of subjects as compared with the scores found in our earlier pilot work. This phenomenon may be attributable to the unique environment provided by the CFI and surrounding area of Fort Sam Houston and BAMC. This hypothesis is based on our data, but further study with a different study design would need to be conducted to test this hypothesis and reaffirm this observation. We believe that the presence of a "San Antonio effect" is plausible for several reasons related to services at the CFI and the environment at BAMC.

CFI rehabilitation services are comprehensive and focus on social reintegration. Services include fitting and training with upper- and lower-limb prosthetics, firearm training systems, functional capacity evaluations, swimming and wave pools, a driving simulator, community reintegration, and an ADL apartment for life skills training. The CFI community reintegration program was developed to provide patients with exposure to the community within a supportive, therapeutic, structured program. Individuals who participate in the community reintegration program may attend a variety of community outings, which are "graded" to provide progressive amounts of exposure to and interaction with the larger community outside of the CFI and BAMC. The first level outings provide socialization and exposure to the community through group trips to a restaurant and a movie. More complex outings are typically longer day or multiple-day excursions and involve more physical demanding activities such as adaptive sports, paintball and laser tag, and a firing range.

The Fort Sam Houston and BAMC environment provides many supportive resources to soldiers and their families, which may facilitate their participation in society. For example, the Soldier Family Assistance Center offers 14 different services ranging from financial counseling, Army continuing education systems, and traumatic injury protection under servicemember group life insurance services to referral to VA services. At Fort Sam Houston, Army Community Support offers several classes on topics such as anger management, communication and leadership skills, and healthy relationships. The Warrior Family Support Center is available for families of servicemembers from OIF and OEF. This center provides emotional support, assists families with answering questions, and provides up to 48 monthly events in the military and civilian community [36].

Our estimates of ES and SRM should be interpreted cautiously for several reasons. First, our subject pool was very heterogeneous. The length of time from injury to beginning of treatment at the CFI varied considerably among our subjects, and the length of time from beginning CFI treatment to participation in visit 1 assessment also varied, with few subjects assessed within 1 month of beginning treatment at the CFI. Furthermore, we made no attempts to control for the types of interventions delivered or to assess the content of the intervention program. Nor did we have a criterion measure to use to assess "improved" community reintegration status. Thus, it is not possible to use this data to assess the responsiveness of the CRIS to the specific treatment rendered at the CFI.

Another limitation of the current study is that we calculated summary CRIS scores for each of the subscales by summing the scores for all completed items and dividing by the number of items completed. Some CRIS items were answered only by those respondents who were working or parenting young children. Respondents who were not working or who did not have children marked such questions as not applicable. Of the respondents, 32 percent (n = 22) answered at least one of the parenting questions and 47 percent (n = 32) answered at least one of the work questions. Thus, the summary scores used varying numbers of items, with those who were not working or parenting answering fewer questions. We recognize that this may have influenced our findings. Therefore, we undertook a sensitivity analysis to explore the effect of removing the work and parenting questions. We recalculated the summary score for the core items in each subscale, which were completed by all respondents. We recalculated the ICC values of each of the revised CRIS subscales, examined correlations between the scores of the revised CRIS subscales and other measures, and examined construct validity by performing analyses of variance (ANOVAs) on known groups. Overall, the means ± SDs of the complete CRIS subscales and the revised CRIS subscales (constructed from core items only) were comparable with mean ± SD values of 54 ± 6, 56 ± 8, and 57 ± 7 for the CRIS Extent of Participation, Perceived Limitations, and Satisfaction with Participation subscales, respectively. Furthermore, ICCs of the revised CRIS subscales were comparable with those of the original full scales (0.92, 0.89, and 0.91 for the CRIS Extent of Participation, Perceived Limitations, and Satisfaction with Participation subscales, respectively). The significance and magnitude of correlations between the core item subscales and other measures (concurrent validation) was largely unchanged. Lastly, the findings from the ANOVAs were also similar. Because the inclusion of the working and parenting items did not bias the results of the current study, we scored the CRIS as originally described. We are unsure whether the results that we observed would generalize to other samples, and thus we recommend that those who use the CRIS in the future explore the effect of skipped questions on overall measure scoring.

CONCLUSIONS

The CRIS exhibited excellent test-retest reliability as well as strong concurrent and known-group validity. We found that the CRIS was equally or more responsive to change as other measures of quality of life and role function. Together, these results demonstrate that the CRIS is a psychometrically sound choice for measurement of community reintegration in severely wounded combat veterans. Measurement of community integration is important in promoting the development of treatments that target enhanced community integration, assessing such treatments, documenting program effectiveness, and tracking population health in terms of involvement with (vs disengagement from) adult life roles. Further analyses are needed to examine responsiveness of the measure over an episode of rehabilitative care.

ACKNOWLEDGMENTS
Author Contributions:
Study concept and design: L. Resnik, M. Gray.
Acquisition of data: L. Resnik, M. Gray.
Analysis and interpretation of data: L. Resnik, M. Borgia.
Drafting of manuscript: L. Resnik, M. Gray, M. Borgia.
Critical revision of manuscript for important intellectual content: L. Resnik.
Statistical analysis: L. Resnik, M. Borgia.
Obtained funding: L. Resnik.
Study supervision: L. Resnik, M. Gray.
Financial Disclosures: The authors have declared that no competing interests exist.
Funding/Support: This material was based on work supported by the VA, Veterans Health Administration, Office of Research and Development, Health Services Research and Development (grant SDR-07-327).
Additional Contributions: The authors wish to thank Sandi Jarzombek, Major Jay Clasing, Major Lisa Smurr, Debra Kelty, and Kimberly McConnell for their effort in assisting with data collection and study management. Captain Gray is now at the Department of Occupational Therapy, Tripler Army Medical Center, Honolulu, Hawaii. The view(s) expressed herein are those of the author(s) and do not reflect the official policy or position of BAMC, the U.S. Army Medical Department, the U.S. Army Office of the Surgeon General, the Department of the Army, the Department of Defense, or the U.S. Government.
Institutional Review: The research protocol was approved by the institutional review board of the Providence VA Medical Center, Providence, Rhode Island, and BAMC, San Antonio, Texas.
Participant Follow-Up: The authors do not plan to inform participants of the publication of this study.
REFERENCES
1. Doyle ME, Peterson KA. Re-entry and reintegration: Returning home after combat. Psychiatr Q. 2005;76(4):361-70.
[PMID: 16217631]
DOI:10.1007/s11126-005-4972-z
2. Brustein J. Back from Iraq. Gotham Gazette. 2006 Mar. p. 4.
3. Hooper TI, Debakey SF, Bellis KS, Kang HK, Cowan DN, Lincoln AE, Gackstetter GD. Understanding the effect of deployment on the risk of fatal motor vehicle crashes: A nested case-control study of fatalities in Gulf war era veterans, 1991-1995. Accid Anal Prev. 2006;38(3):518-25.
[PMID: 16405857]
DOI:10.1016/j.aap.2005.11.009
4. National Center for Post-Traumatic Stress Disorder; Walter Reed Army Medical Center. Iraq war clinician guide. 2nd ed. Washington (DC): Department of Veterans Affairs; 2004 [cited 2004 Jan 26]. Litz B, Orsillo SM. Chapter 3. The returning veteran of the Iraq war: Background issues and assessment guidelines. Available from:
http://www.ptsd.va.gov/professional/pages/vets-iraq-war-guidelines.asp.
5. Fischer H. United States military casualty statistics: Operation Iraqi Freedom and Operation Enduring Freedom. Washington (DC): Congressional Research Service; 2009.
6. Iraq Index. Washington (DC): Brookings; 2010 [updated 2010 Sep 30; cited 2010 May 25]. Available from: http://www.brookings.edu/saban/iraq-index.aspx.
7. Lew HL, Poole JH, Alvarez S, Moore W. Soldiers with occult traumatic brain injury. Am J Phys Med Rehabil. 2005; 84(6):393-98. [PMID: 15905652]
DOI:10.1097/01.phm.0000163703.91647.a7
8. Hoge CW, Auchterlonie JL, Milliken CS. Mental health problems, use of mental health services, and attrition from military service after returning from deployment to Iraq or Afghanistan. JAMA. 2006;295(9):1023-32. [PMID: 16507803]
9. Hoge CW, Castro CA, Messer SC, McGurk D, Cotting DI, Koffman RL. Combat duty in Iraq and Afghanistan, mental health problems, and barriers to care. N Engl J Med. 2004; 351(1):13-22. [PMID: 15229303]
DOI:10.1056/NEJMoa040603
10. Committee on the Initial Assessment of Readjustment Needs of Military Personnel, Veterans, and Their Families. Returning home from Iraq and Afghanistan: Preliminary assessment of readjustment needs of veterans, service members, and their families. Washington (DC): The National Academies Press; 2010.
11. Smith MW, Schnurr PP, Rosenheck RA. Employment outcomes and PTSD symptom severity. Ment Health Serv Res. 2005;7(2):89-101. [PMID: 15974155]
DOI:10.1007/s11020-005-3780-2
12. Johnson DR, Fontana A, Lubin H, Corn B, Rosenheck R. Long-term course of treatment-seeking Vietnam veterans with posttraumatic stress disorder: Mortality, clinical condition, and life satisfaction. J Nerv Ment Dis. 2004;192(1): 35-41. [PMID: 14718774]
DOI:10.1097/01.nmd.0000105998.90425.6a
13. Rosenheck R, Fontana A. A model of homelessness among male veterans of the Vietnam war generation. Am J Psychiatry. 1994;151(3):421-27. [PMID: 8109652]
14. Bullman TA, Kang HK. Posttraumatic stress disorder and the risk of traumatic deaths among Vietnam veterans. J Nerv Ment Dis. 1994;182(11):604-10. [PMID: 7964667]
DOI:10.1097/00005053-199411000-00002
15. Freeman TW, Roca V. Gun use, attitudes toward violence, and aggression among combat veterans with chronic posttraumatic stress disorder. J Nerv Ment Dis. 2001;189(5): 317-20. [PMID: 11379976]
DOI:10.1097/00005053-200105000-00008
16. Resnik LJ, Allen SM. Using International Classification of Functioning, Disability and Health to understand challenges in community reintegration of injured veterans. J Rehabil Res Dev. 2007;44(7):991-1006. [PMID: 18075956]
DOI:10.1682/JRRD.2007.05.0071
17. Rosenheck R, Frisman L, Chung AM. The proportion of veterans among homeless men. Am J Public Health. 1994; 84(3):466-69. [PMID: 8129068]
DOI:10.2105/AJPH.84.3.466
18. Resnik L, Plow M, Jette A. Development of CRIS: Measure of community reintegration of injured service members. J Rehabil Res Dev. 2009;46(4):469-80. [PMID: 19882482]
DOI:10.1682/JRRD.2008.07.0082
19. McHorney CA, Tarlov AR. Individual-patient monitoring in clinical practice: Are available health status surveys adequate? Qual Life Res. 1995;4(4):293-307. [PMID: 7550178]
D
OI:10.1007/BF01593882
20. Yancosek K, Daugherty SE, Cancio L. Treatment for the service member: A description of innovative interventions. J Hand Ther. 2008;21(2):189-95. [PMID: 18436140]
DOI:10.1197/j.jht.2007.12.002
21. Kazis LE, Ren XS, Lee A, Skinner K, Rogers W, Clark J, Miller DR. Health status in VA patients: Results from the Veterans Health Study. Am J Med Qual. 1999;14(1):28-38.
[PMID: 10446661]
DOI:10.1177/106286069901400105
22. Whiteneck GG , Charlifue SW, Gerhart KA, Overholser JD, Richardson GN. Quantifying handicap: A new measure of long-term rehabilitation outcomes. Arch Phys Med Rehabil. 1992;73(6):519-26. [PMID: 1622299]
23. Burckhardt CS, Anderson KL. The Quality of Life Scale (QOLS): Reliability, validity, and utilization. Health Qual Life Outcomes. 2003;1(1):60. [PMID: 14613562]
DOI:10.1186/1477-7525-1-60
24. Ware JE, Snow KK, Kosinski M, Gandek B; New England Medical Center Hospital. The Health Institute. SF-36 Health Survey: Manual and interpretation guide. Boston (MA): The Health Institute, New England Medical Center; 1993.
25. Shrout PE, Fleiss JL. Intraclass correlations: Uses in assessing rater reliability. Psychol Bull. 1979;86(2):420-28.
[PMID: 18839484]
DOI:10.1037/0033-2909.86.2.420
26. Cohen J. Statistical power analysis for the behavioral sciences. Revised ed. New York (NY): Academic Press; 1977.
27. Dekel R, Solomon Z, Bleich A. The contribution of social disability to the evaluation of mental disability among PTSD veterans. Isr J Psychiatry Relat Sci. 2004;41(4):237-47.
[PMID: 15941019]
28. Neal LA, Green G , Turner MA. Post-traumatic stress and disability. Br J Psychiatry. 2004;184:247-50.
[PMID: 14990523]
DOI:10.1192/bjp.184.3.247
29. Wagner AK, Hammond FM, Sasser HC, Wiercisiewski D. Return to productive activity after traumatic brain injury: Relationship with measures of disability, handicap, and community integration. Arch Phys Med Rehabil. 2002; 83(1):107-14. [PMID: 11782840]
DOI:10.1053/apmr.2002.27470
30. Luty SE, Joyce PR, Mulder RT, McKenzie JM. Social adjustment in depression: The impact of depression severity, personality, and clinic versus community sampling. J Affect Disord. 2002;70(2):143-54. [PMID: 12117626]
DOI:10.1016/S0165-0327(01)00331-7
31. Bellack AS, Green MF, Cook JA, Fenton W, Harvey PD, Heaton RK, Laughren T, Leon AC, Mayo DJ, Patrick DL, Patterson TL, Rose A, Stover E, Wykes T. Assessment of community functioning in people with schizophrenia and other severe mental illnesses: A white paper based on an NIMH-sponsored workshop. Schizophr Bull. 2007;33(3): 805-22. [PMID: 16931542]
DOI:10.1093/schbul/sbl035
32. Linden MA, Crothers IR, O'Neill SB, McCann JP. Reduced community integration in persons following traumatic brain injury, as measured on the Community Integration Measure: An exploratory analysis. Disabil Rehabil. 2005;27(22):1353-56. [PMID: 16321919]
DOI:10.1080/09638280500164180
33. Hoofien D, Gilboa A, Vakil E, Donovick PJ. Traumatic brain injury (TBI) 10-20 years later: A comprehensive outcome study of psychiatric symptomatology, cognitive abilities and psychosocial functioning. Brain Inj. 2001;15(3): 189-209. [PMID: 11260769]
DOI:10.1080/026990501300005659
34. Stratford PW, Binkley FM, Riddle DL. Health status measures: Strategies and analytic methods for assessing change scores. Phys Ther. 1996;76(10):1109-23. [PMID: 8863764]
35. Jette DU, Jette AM. Physical therapy and health outcomes in patients with spinal impairments. Phys Ther. 1996;76(9): 930-45. [PMID: 8790272]
36. Soldier and family services. San Antonio (TX): U.S. Army MWR; 2010 [cited 2010 April 1]. Available from: http://www.fortsamhoustonmwr.com/default2.asp.
Submitted for publication April 19, 2010. Accepted in revised form September 14, 2010.
This article and any supplementary material should be cited as follows:
Resnik L, Gray M, Borgia M. Measurement of community reintegration in sample of severely wounded servicemembers. J Rehabil Res Dev. 2011;48(2):89-102.
DOI:10.1682/JRRD.2010.04.0070
Crossref

Go to TOP

Go to the Table of Contents of Vol. 48 No. 2

Last Reviewed or Updated  Friday, February 11, 2011 9:38 AM

Valid HTML 4.01 Transitional