JRRD Logo


Volume 44 Number 3 2007
Pages 363 — 372


Development and validation of Patient Reported Impact of Spasticity Measure (PRISM)

Karon F. Cook, PhD;1-3* Cayla R. Teal, PhD;1,4-5 Joan C. Engebretson, DrPH, RN, AHN-BC;6 Karen A. Hart, PhD;7 Jane S. Mahoney, DSN, APRN, BC;6 Susan Robinson-Whelen, PhD;7 Arthur M. Sherwood, PhD7

1Department of Veterans Affairs (VA) Measurement Excellence and Training Resource Information Center, Houston, TX; 2Department of Rehabilitation Medicine, University of Washington, Seattle, WA; 3Evanston Northwestern Healthcare Center on Outcomes Research and Education, Chicago, IL; 4Houston Center for Quality of Care and Utilization Studies, VA Medical Center, Houston, TX; 5Department of Medicine, Baylor College of Medicine, Houston, TX; 6University of Texas Health Science Center at Houston, School of Nursing, Houston, TX; 7Department of Physical Medicine and Rehabilitation, Baylor College of Medicine, Houston, TX

Abstract — Persons with spinal cord injury (SCI) may experience a range of symptoms typically labeled "spasticity." Previous efforts to develop assessment tools that measure spasticity have failed to represent the experiences of persons who live with the condition. The purpose of this multicenter study was to develop an instrument that measures the impact of spasticity on quality of life. Based on 24 semistructured interviews, a developmental form of the Patient Reported Impact of Spasticity Measure (PRISM) was constructed. The developmental PRISM was administered to 180 persons at five sites. Subscales were developed based on factor analytic results. Evidence for the reliability and validity of the scores was evaluated. Seven subscales were developed, including one that measures the positive effects of spasticity. Results of reliability and validity assessments indicate that the PRISM subscale scores effectively measure the impact of spasticity in the population of veterans with SCI.

Key words: abnormal muscle control, factor analysis, involuntary muscle movement, motor control, outcome assessment, psychometrics, quality of life, rehabilitation, spasticity, spinal cord injury.


Abbreviations: AMC = abnormal muscle control, ASIA = American Spinal Injury Association, DA = Daily Activities, ICC = intraclass correlation coefficient, IMM = involuntary muscle movement, IRT = item response theory (model), NAP = Need for Assistance/Positioning, NI = Need for Intervention, PA = Psychological Agitation, PI = Positive Impact, PRISM = Patient Reported Impact of Spasticity Measure, SAA = Social Avoidance/Anxiety, SCI = spinal cord injury, SE = Social Embarrassment, TIRR = The Institute of Rehabilitation and Research, VAMC = Department of Veterans Affairs medical center.
*Address all correspondence to Karon F. Cook, PhD; 801 Cortland St, Houston, TX 77007; 713-291-3918.
Email: karonc2@u.washington.edu
DOI: 10.1682/JRRD.2006.04.0036
INTRODUCTION

Spinal cord injury (SCI) can suddenly change a person's life physically, emotionally, and socially. Persons with spinal cord lesions frequently experience involuntary movements or spasms, altered motor control, weakness or paralysis, and altered sensation or pain. Motor control problems are typically labeled "spasticity." These problems can substantially impact the quality of life of those affected. In surveys of patients' perceptions of problems associated with their SCI, spasticity has been consistently identified in the top three to five life concerns [1]. Because the experience of spasticity or altered motor control is multidimensional in nature and broad in scope, a comprehensive means of assessing its impact is needed [2]. Previous efforts to develop instruments for measuring spasticity focused on explicit mechanical properties but failed to address and, consequently, failed to represent the experiences of persons who live with spasticity [3].

This multicenter study developed, from the perspective of persons with SCI, an instrument that measures the impact of abnormal muscle control (AMC) or involuntary muscle movement (IMM) on quality of life.

METHODS
Study Sample

A total of 212 persons participated in the study. A group of 24 participated in semistructured interviews, a focus group of 8 responded to and evaluated a first draft of the Patient Reported Impact of Spasticity Measure (PRISM), and 180 completed a developmental form of the PRISM. So that we could evaluate test/retest reliability, 36 of these 180 participants were asked to complete the instrument a second time and return it within 1 week. Participants were recruited from the Department of Veterans Affairs medical centers (VAMCs) in Cleveland, Ohio; Dallas and Houston, Texas; and Palo Alto, California. In addition, participants were recruited from The Institute for Rehabilitation and Research (TIRR), Houston, Texas. The institutional review boards at all sites approved the study protocol. Persons were included if they had sustained an SCI, were over age 18, spoke and read English, and stated that they currently or in the past had experienced spasticity.

Procedures
Development of Initial Item Pool

Twenty-four participant interviews were conducted by one of the authors (KAH), who has extensive experience in research and education programs with and for persons with SCI. Consistent with established qualitative methods [4-5], the interviews were semistructured and included queries such as, "I am interested in learning about your experience with having involuntary movements. Can you tell me about it?" Interviews were audiotaped, transcribed, and then evaluated by two team members experienced in qualitative research (JCE and JSM). The data were reviewed and summarized with particular attention paid to identifying recurrent themes and recording the language that participants used to describe their experiences. A domain analysis of reported experiences ensured that the instrument was grounded in the phenomenon under investigation. Seven specific domains were identified that described participants' experiences: physical characteristics, impact on activities (positive and negative), psychological sequelae, financial costs, impact on interpersonal relations, functional self-management, and attributions. Descriptive subcategories within each domain were identified. Details of these results are reported elsewhere [6].

Based on the qualitative findings, the psychometrician on the study team (KFC) drafted an initial pool of items, paying particular attention to the participants' natural language statements. Items were written to reflect the themes and categories in all the identified domains. In the development of an instrument, beginning with more items than are likely to be included in the final measure is important. This method allows developers to be "choosy" and select the items that prove to have the strongest psychometric properties. In the current study, we developed 65 candidate items. Because no agreed-upon clinical definition of spasticity exists and persons with SCI may classify a range of experiences as spasticity, we were careful to specify the kind of experiences about which we wished participants to report. The introductory text for the questionnaire stated, "The following questions are about your experience of AMC or IMM. Different people have different terms they use for AMC and IMM. Some of these are (1) spasticity, (2) muscle stiffness (tone), (3) spasms, (4) clonus (bouncing), (5) when muscles don't cooperate together like they're supposed to, and (6) when trying to move one part of my body causes another part to move also."

The stem for the items stated, "Over the PAST WEEK, my abnormal muscle control or involuntary muscle movement . . . ." The items that followed queried respondents about both the positive and negative impacts of their AMC or IMM. A panel of researchers and clinicians with experience in SCI reviewed the initial items. Items were modified based on their input, and a 5-point Likert-type response scale was chosen in which 0 = "never true for me," 1 = "rarely true for me," 2 = "sometimes true for me," 3 = "often true for me," and 4 = "very often true for me."

Refinement of Initial Item Pool

Using cognitive testing methods [7], we evaluated the initial item pool with a focus group of eight participants with SCI and self-reported spasticity. Participants commented in particular on whether the instructions and item wording were easy to understand and whether the item content comprehensively captured their experiences of AMC/IMM. Items were revised based on the participants' input. We created a survey that included the 65 items plus demographic and clinical questions. In addition, one item asked respondents to compare the positive and negative effects of their involuntary muscle control. Response options for this item were 1 = "positive strongly outweighs negative," 2 = "positive slightly outweighs negative," 3 = "positive and negative are equal," 4 = "negative slightly outweighs positive," and 5 = "negative strongly outweighs positive."

Administration and Evaluation of Developmental PRISM

A survey containing the PRISM developmental items was administered to a sample of 180 persons with SCI who reported experiencing AMC/IMM. In addition to completing the questionnaire at enrollment, a subsample of 36 persons recruited from the two Houston sites took home a copy of the instrument along with a posted envelope in which to return the completed instrument. These participants were instructed to return the second copy of the instrument within 1 week. However, as we report subsequently, most were returned after a longer time period.

A subsample (n = 33) recruited from the Houston sites (Houston VAMC = 25, TIRR = 8) underwent a clinical examination and had their injuries classified based on the American Spinal Injury Association (ASIA) scale [8]. We refer to this subsample as the "clinical subsample." All examinations were conducted by a physical therapist with clinical and research experience in SCI. This subsample reported the global severity (i.e., "mild," "moderate," or "severe") and frequency ("no spasms" to "spasms occurring more than 10 times per hour") of their spasms. They also reported the degree to which spasms interfered with their function (i.e., "did not interfere," "made function difficult," or "prevented function"). All study data were double-entered into an electronic database.

Analyses
Identification of Factor Structure and Content Validity

Mplus software (Muthén & Muthén, Los Angeles, California) [9] was used to examine the factor structure and content validity of the item responses. Mplus was chosen because it accounts for the categorical nature of the data. An exploratory, first-order factor analysis was conducted with the polychoric correlation matrix. The optimal number of factors representing the associations among the variables was determined by several methods [10], including parallel analysis [11-12], scree test [13], examination of eigenvalues, and examination of the residual correlation matrix. The factors were estimated with an Unweighted Least Squares method, extracted with a principal axes (or "common factor") solution, and then rotated with a Varimax orthogonal rotation followed by a Promax oblique rotation. The pattern matrix was used to interpret the factor solutions.

Reliability Assessment

The reliability of the PRISM was assessed by calculation of the interitem consistency of subscale items (Cronbach a). The reproducibility of scores was evaluated by calculation of the test/retest correlation (intra-class correlation coefficient [ICC]) [14].

Validity Assessment

We share the view that "content validity is built into a test from the outset through the choice of appropriate items" [15]. The content-related validity of the PRISM is supported by the domain analysis of participant interviews and review of the candidate items by an expert review panel. In addition to content validity, we evaluated the capability of the PRISM to discriminate "known groups." That is, we formed a priori hypotheses regarding which groups of participants were more likely to experience the impact of AMC/IMM. Specifically, we developed the following hypotheses:

1. Persons reporting more severe "problematic spasms" will score higher on PRISM subscales that measure negative impacts than will persons reporting less severe problematic spasms.
2. Persons reporting more interference from problematic spasms will score higher on PRISM subscales that measure negative impacts than will persons reporting less severe problematic spasms.
3. Persons indicating that, for them, the positive effects of AMC/IMM either slightly or strongly outweigh the negative effects will score higher on the PRISM subscale that assesses positive impact than will persons indicating the opposite.

Our clinical subsample, the subsample of persons who were clinically examined, was small (n = 33). For the statistical comparisons, therefore, we collapsed interference and severity ratings into two categories each. Participants who reported that their problematic spasms were "mild" were compared with participants who reported that they were "moderate" or "severe." Those who reported that they experienced no interference with function were compared with those who reported that their problematic spasms interfered or prevented function. Because group sizes were unequal and data were categorical, all statistical comparisons were made with the less powerful but more appropriate Mann-Whitney U statistical test of mean ranks.

RESULTS
Study Sample

Demographic and clinical variables are reported in Table 1. A total of 49 participants were recruited from the Houston VAMC, 14 from Houston TIRR, 64 from Dallas, 21 from Cleveland, and 32 from Palo Alto.


Table 1.
Demographic and clinical variables of 180 participants responding to Patient Reported Impact of Spasticity Measure development questionnaire.*
Variable
All Participants
Clinical Subsample
Sex, n (%)
 
 
Male
162 (90)
25 (78)
Female
18 (10)
7 (22)
Injury, n (%)
 
 
Complete
55 (30)
11 (36)
Incomplete
121 (70)
20 (65
Voluntary Movement Below Injury Level, n (%)
 
 
Yes
95 (54)
18 (58)
No
82 (46)
13 (42)
Sensation Below Injury Level, n (%)
 
 
Yes
129 (73)
25 (81)
No
49 (28)
6 (9)
Age (mean ± SD)
52 ± 12
51 ± 10
Years Since Injury (mean ± SD)
15 ± 11
15 ± 11
Race/Ethnicity, n (%)
 
 
Caucasian
129 (71)
18 (55)
African American
37 (21)
10 (30)
Hispanic
8 (4.5)
4 (12)
Asian
1 (0.6)
0 (0)
Other
3 (1.7)
0 (0)
Severity of Spasms, n (%)
 
 
Mild
-
9 (28)
Moderate
-
16 (50)
Severe
-
7 (22)
Level of Interference with Function, n (%)
 
 
Spasms Do Not Interfere
-
8 (24)
Spasms Make Function Difficult
-
19 (58)
Spasms Prevent Function
-
6 (18)
Frequency of Spasms, n (%)
 
 
No Spasms
-
1 (3)
Spasms Induced by Stimulation
-
4 (3)
Infrequent Spasms <1 Per Hour
-
15 (46)
Spasms >1 Per Hour
-
11 (33)
Spasms >10 Per Hour
-
2 (6)
Note: Participants were 2 months to 56 years postinjury and mean  ±  SD 52 ± 12 years of age. Racial/ethnic breakdown was 71% Caucasian, 21% African American, 4.5% Hispanic, 0.6% Asian, and 1.7% other.
*Because of missing responses, n varies by variable. Percentages represent valid percentage (not including missing). Because of rounding, not all percentages sum to 100.
Clinical subsample refers to 33 participants who underwent clinical examination.
SD = standard deviation.

The data in Table 1 show that the demographic and clinical characteristics of the clinical subsample were similar to those of the rest of the study population, though, in the clinical subsample, the proportion of women was higher. ASIA classifications [8] were obtained for 32 of the 33 participants for whom clinical data were gathered; 21 were classified as ASIA A, 1 as ASIA B, 3 as ASIA C, and 7 as ASIA D. The clinical subsample also reported on the severity, frequency, and interference of their problematic spasms, and these results are included in Table 1.

Factor Analytic Structure

Of the 65 PRISM items, only 55 were included in the exploratory factor analysis. Ten items were judged not to pertain strictly to the impact of AMC/IMM (Appendix 1). Some of the 10 items described the experience of AMC/IMM rather than the intensity of their impact (e.g., ". . . my abnormal muscle control . . . was unpredictable"), and others queried respondents about AMC/IMM "triggers" (e.g., ". . . my abnormal muscle control . . . was triggered by my emotions").

An exploratory first-order factor analysis was conducted based on the responses of the 147 persons who answered all 55 remaining items. The purpose of this analysis was twofold. First, we wanted to associate items with distinct subscales. Second, because we began with a larger item pool than we expected to use, we wanted to identify items that could be deleted from the item pool. The multiple approaches used to determine the number of factors consistently suggested a 12-factor solution. The factors were rotated with a Varimax orthogonal rotation followed by a Promax oblique rotation. The 12 extracted factors accounted for 65 percent of the data variance.

Our goal was to include subscales measuring as many of the 12 factors as possible without constructing subscales that were either poorly defined or underpopulated with items. Based on this goal, we elected not to construct subscales to represent factors 8 through 12. Factor 8 had four items associated with it: sleep, fatigue, and AMC/IMM as "a warning that something was going on" (e.g., urinary infection). Because these four items did not define a clearly interpretable dimension, we elected not to include a subscale assessing this factor. Factors 9 through 12 were defined by only one or two items each. These factors and the items most highly associated with them were dropped from further consideration. Factors 1 through 7 accounted for 50 percent of the total variance. Factor loadings for all items are presented in Appendix 2. Shaded items in this appendix were not retained in the final PRISM. For all retained items (bolded), we have included item numbers to indicate our suggested order of administration. We designed this ordering to limit potential acquiescence bias in participant responses and to standardize the order of administration in future studies that use the PRISM.

Seven subscales were constructed to represent the first seven factors. Items were retained that loaded at Š 0.400 on a retained factor. One item, "Caused me to worry," was factorially complex, loading almost equally on the first (0.491) and second factors (0.496). Because this item did not help distinguish among impact factors, it was deleted from consideration for the PRISM measure.

We labeled the factors based on input from the researchers who conducted and analyzed the participant interviews and from clinicians and researchers experienced in SCI. The seven factors were labeled Social Avoidance/Anxiety (SAA), Psychological Agitation (PA), Daily Activities (DA), Need for Assistance/Positioning (NAP), Positive Impact (PI), Need for Intervention (NI), and Social Embarrassment (SE) (Appendix 2).

PRISM Subscale Descriptive Statistics

As described earlier, items were scored from 0 to 4, with higher scores indicating greater impact. To account for missing responses, we obtained subscale scores by averaging item scores and multiplying by the number of items in the subscale rather than by totaling item scores. For each subscale, scores for participants who responded to less than four items or less than 80 percent of a subscale's items were recorded as "missing." Descriptive statistics for the subscale scores are reported in Table 2. For almost every subscale, observed scores extended over the full range of the subscale; that is, there were persons who obtained the lowest possible score and persons who obtained the highest possible score. Responses were not uniformly distributed across response options, however. Scores were somewhat skewed toward higher scores; that is, relative to the range of the scale, more persons scored in lower ranges (indicating less impact).


Table 2.
Descriptive statistics and reliability for Patient Reported Impact of Spasticity Measure (PRISM) subscales.
PRISM Subscale
n
No. of Items
 
Observed Range
(Potential Range)
Mean ± SD
Cronbach
a
ICC
(n = 33)
Social Avoidance/Anxiety
180
11
 
0-42 (0-44)
9.7 ± 12.1
0.96
0.82
Psychological Agitation
180
5
 
0-20 (0-20)
7.0 ± 6.1
0.91
0.84
Daily Activities
180
6
 
0-22 (0-24)
6.5 ± 5.4
0.87
0.86
Need for Assistance/Positioning
179
5
 
0-18 (0-20)
4.8 ± 4.4
0.76
0.86
Positive Impact
174
4
 
0-16 (0-16)
4.9 ± 4.0
0.76
0.87
Need for Intervention
180
5
 
0-18 (0-20)
3.6 ± 4.2
0.74
0.91
Social Embarrassment
177
5
 
0-19 (0-20)
4.7 ± 4.9
0.86
0.87
ICC = intraclass correlation coefficient, SD = standard deviation.

The correlations among pairs of subscales are reported in Table 3. As expected, subscale scores correlated positively with one other. That these correlations were moderate and not high suggests that the domains measured are distinct. The PI subscale was the least highly correlated with the other subscales (0.07-0.27), suggesting that whether persons experience positive impacts of AMC/IMM is relatively independent of whether they experience negative impacts. Some persons may simultaneously experience positive and negative impacts of spasticity. Others may experience high negative and low positive impact; still others may experience the reverse.


Table 3.
Correlations among pairs of Patient Reported Impact of Spasticity Measure (PRISM) subscales (n = 180).
PRISM Subscale
SAA
PA
DA
NAP
PI
NI
SE
Social Avoidance/Anxiety (SAA)
-
0.73
0.61
0.45
0.07
0.65
0.72
Psychological Agitation (PA)
-
-
0.59
0.54
0.12
0.63
0.70
Daily Activities (DA)
-
-
-
0.63
0.29
0.52
0.62
Need for Assistance/Positioning (NAP)
-
-
-
-
0.27
0.48
0.59
Positive Impact (PI)
-
-
-
-
-
0.15
0.26
Need for Intervention (NI)
-
-
-
-
-
-
0.59
Social Embarrassment (SE)
-
-
-
-
-
-
-
Reliability of PRISM Subscales

In addition to presenting the descriptive statistics just described, Table 2 reports the internal consistency (Cronbach a) and reproducibility (ICC) for each subscale. Internal consistency was lowest for the NI subscale (0.74) and highest for the SAA subscale (0.96). ICC values were high for all seven subscales (range = 0.82-0.91). Unfortunately, study staff failed to enter the return date for half (n = 17) the retest forms. Of participants for whom the date was available, one participant returned the retest form the next day and another waited 49 days. On average, 11 days separated the first and second administrations (standard deviation = 13 days).

Validity of PRISM Subscales

As mentioned above, we evaluated validity by developing and testing a priori hypotheses regarding the relationship between PRISM subscales scores and responses to the severity and interference questions. The resulting probabilities and mean ranks are reported in Table 4. All mean ranks were in the expected direction for both questions. All the comparisons based on severity responses were statistically significant (Mann-Whitney U, p < 0.05, one-tailed). However, only two of the comparisons based on interference ratings were statistically significant (DA and NI). The small size of the clinical subsample may have contributed to the failure to find statistical significance. This small sample size is a limitation of the current study.


Table 4.
Patient Reported Impact of Spasticity Measure (PRISM) subscale discrimination of groups based on levels of spasm severity and interference.
PRISM Subscale
Severity
 
Interference
p-Value
Mean Ranks
 
p-Value
Mean Ranks
Mild
Moderate or Severe
 
Does Not Interfere
Makes Difficult or Prevents Function
Social Avoidance/Anxiety
0.001
8.3
19.7
 
0.095
13.2
18.2
Psychological Agitation
0.001
8.0
19.8
 
0.3
15.4
17.5
Daily Activities
0.02
11.2
18.6
 
0.04
11.8
18.7
Need for Assistance/Positioning
0.02
11.1
18.6
 
0.15
14.0
18.0
Need for Intervention
0.008
10.3
18.9
 
0.03
11.5
18.8
Social Embarrassment
0.02
10.8
18.1
 
0.29
14.9
17.0

All 180 participants reported on the relative strength of positive and negative effects of AMC/IMM. The distribution of responses is reported in Table 5. We compared scores on the PRISM PI subscale with the participants'judgments regarding whether the positive effects of AMC/IMM outweighed the negative ones. The two groups compared were the participants reporting that the positive impacts either "slightly" or "strongly" outweighed the negative (n = 38) and the participants reporting that the negative impacts either "slightly" or "strongly" outweighed the positive (n   =   82). Those reporting that the impact of each was "about equal" were excluded from this analysis. For those reporting that positive impacts outweighed negative impacts, the mean rank PI subscale score was 76.3. For those with the opposite judgment, the mean rank was 51.3. This difference was statistically significant (Mann-Whitney U, p < 0.001, one-tailed). That the differences were in the expected direction and statistically significant support the validity of the PI subscale.


Table 5.
Reported relative strength of positive and negative impacts of abnormal muscle control and/or involuntary muscle movements (N = 180, with 16 missing).
Response
n (%)
Positive Strongly Outweighs Negative
20 (12)
Positive Slightly Outweighs Negative
18 (11)
Positive and Negative Equal
44 (27)
Negative Slightly Outweighs Positive
28 (17)
Negative Strongly Outweighs Positive
54 (33)
DISCUSSION AND CONCLUSIONS

The PRISM is a new instrument that standardizes the collection of self-report information relevant to the clinical assessment of AMC/IMM. Priebe et al. noted that spasticity is a multidimensional phenomenon in persons with SCI [2]. The PRISM subscales assess the effect of altered motor control with respect to SAA, PA, DA, NAP, NI, and SE, as well as the positive effect of altered motor control.

The study results have a number of clinical implications. Current measures focus on physical manifestations of spasticity, with the implicit assumption that all spasticity is bad and more is worse. Since "you get what you measure," studies based on previously developed measures are unlikely to register any beneficial aspects of spasticity. Based on input from team members who have clinical experience in spasticity following SCI, we considered the possibility that participants find spasticity to have positive as well as negative aspects. Clinical team members observed that individuals may seek control, rather than eradication, of spasticity, and so we offered participants in the structured interviews the opportunity to report any beneficial aspects of spasticity they experienced.

The legitimacy and importance of measuring the positive aspects of spasticity were validated by the qualitative analysis of the structured interviews. A detailed description of the qualitative results is reported elsewhere [6]. Many interviewees indicated that their spasticity benefited them in one or more ways. For example, spasticity was described as a strategy for preventing muscle atrophy, an indicator of a possible urinary tract infection or other problem, and an aid in transfers. Another level of validation of the importance of measuring the benefits of spasticity was obtained from participants' responses to auxiliary items of the developmental PRISM. As reported, a substantial portion of participants indicated that the positive effects of spasticity outweighed the negative effects. Because the PRISM offers the opportunity to explore the range of spasticity's impact, its use may encourage a more nuanced approach to the treatment of spasticity, one that focuses on appropriate management of AMC rather than suppression of all motor control.

The results of this study support the notion that interventions for AMC/IMM should focus on restoration of active control rather than on abolition of all motor activity associated with AMC/IMM. Many interventions (e.g., intrathecal baclofen) markedly suppress motor excitability at the spinal level, with a consequent suppression of residual voluntary motor control. Yet, in the current study, almost one-fourth of study participants reported that the positive impacts of their altered motor control outweighed the negative. This finding provides a compelling reason why practitioners should consider patients' subjective impressions when selecting treatment interventions. Future research should evaluate whether persons for whom the positive impacts of spasticity outweigh the negative have different patterns or physical features of AMC/IMM than do persons for whom this is not the case. Another research question worth investigating is the relationship between persons' evaluations of the relative positive and negative impacts of their altered motor control and the degree to which their treatment is considered "optimized."

The meaning of existing, physically based measures will be informed by comparisons with the PRISM subscales. This comparison, too, may help optimize treatment. For example, some measures may be found to be better correlated with self-reported impact of spasticity. If so, these measures should be favored under the assumption that patients are the best judges of how they are doing.

Ten items were dropped from the initial item pool because they were judged to be descriptive of the nature of AMC/IMM as opposed to indicators of its impact. Though exclusion of these items was consistent with our goal of developing an evaluative instrument, the loss of these items has content validity implications, since the items represent content that participants associated with their experience of AMC/IMM. In future research, the development of one or more checklists should be explored. For example, a checklist could be developed with which participants report on the nature of their AMC/IMM and the conditions and circumstances that trigger it. Checklists are qualitatively different from evaluative instruments (e.g., Likert-type scales). With checklists, because there is interest in the specific answers to each item of the checklist, items are not summed. A checklist could be used to answer the question, "What aspects of the environment trigger respondents' AMC/IMM?" Though development of a checklist was beyond the scope of the current study, creating such an instrument would be worthwhile in future research to augment the PRISM subscales and provide information that would be valuable for both clinical and research purposes.

A limitation of the PRISM scoring system is that item scores are averaged and multiplied. This approach fails to consider that the measurement level of the item scores is ordinal not equal interval. Though the categorical nature of Likert-type data is seldom considered in the scoring of health outcome scales, we recognize that it is a limitation. An alternative is to score the subscales by calculating the median of the item scores. The net effect of scoring the subscales in this way, however, would be having only five possible scores (1-5). We judged this disadvantage to be more problematic than applying mathematical operations that are not optimal for ordinal-level data. A more elegant solution would be to calibrate the items of the subscales with an item response theory (IRT) model [16]. One of the many advantages of IRT modeling is that, compared with summing or averaging categorical item scores, it produces scores that more closely approximate an equal interval scale. Larger sample sizes are required for IRT parameter estimation (e.g., >250). The next step in our development of the PRISM is to collect subscale responses from a large sample and calibrate these responses using an IRT model.

Though frequently done, describing any instrument as reliable or valid is inaccurate. It is the scores obtained with the instrument that will be reliable and valid for particular populations and specified purposes. For this reason, validation of an instrument continues as it is evaluated in different samples and used for various purposes. The work presented here is developmental, but our results indicate that the PRISM scores are valid and reliable in the veteran population with SCI. Evidence to this effect included the findings that (1) ranks of subscale scores were consistent with the rankings of self-reported levels of severity and interference, (2) content of the measure was grounded in persons' reported experiences, and (3) scores were internally consistent and reproducible across time.

The PRISM is the only instrument that has been developed to assess, from the perspective of persons with SCI, the wide range of AMC/IMM impact. We were unable to test concurrent validity because no previous scales have attempted to describe the experience of persons with disabilities. The only self-report scale currently in the literature is the Penn Spasm Frequency Scale [17] that focuses on only one narrow aspect of spasticity and does not probe other aspects of the experience. We expect future research to add to the body of evidence regarding the value and limitations of the PRISM.

ACKNOWLEDGMENTS

This material was based on work supported by the Department of Veterans Affairs, Veterans Health Administration, Rehabilitation Research and Development Service (grant B2212B). Additional salary support was provided by the National Institute of Arthritis and Musculoskeletal and Skin Diseases (grant 1U01AR52171-01).

The authors have declared that no competing interests exist.

REFERENCES
1. Rintala DH, Hart KA, Priebe MM, Ballinger DA, Davis CL. Future directions for spinal cord injury research: Consumer involvement in setting research priorities. Arch Phys Med Rehabil. 1998;79(10):1343.
2. Priebe MM, Sherwood AM, Graves DE, Mueller M, Olson WH. Effectiveness of gabapentin in controlling spasticity: A quantitative study. Spinal Cord. 1997;35(3):171-75.
[PMID: 9076868]
3. Priebe MM, Sherwood AM, Thornby JI, Kharas NF, Markowski J. Clinical assessment of spasticity in spinal cord injury: A multidimensional problem. Arch Phys Med Rehabil. 1996;77(7):713-16. [PMID: 8670001]
4. Fontana A, Frey JH. The interview: From neutral stance to political involvement. In: Denzin NK, Lincoln YS, editors. The Sage handbook of qualitative research. 3rd ed. Thousand Oaks (CA): Sage Publications; 2005. p. 695-727.
5. Holstein JA, Gubrium JF. The active interview. Thousand Oaks (CA): Sage Publications; 1995.
6. Mahoney JS, Engebretson JC, Cook KF, Hart KA, Robinson-Whelen S, Sherwood AM. Spasticity experience domains in persons with spinal cord injury. Arch Phys Med Rehabil. 2007;88(3):287-94. [PMID: 17321818]
7. Collins D. Pretesting survey instruments: An overview of cognitive methods. Qual Life Res. 2003;12(3):229-38.
[PMID: 12769135]
8. Ditunno JF Jr, Young W, Donovan WH, Creasey G. The international standards booklet for neurological and functional classification of spinal cord injury. American Spinal Injury Association. Paraplegia. 1994;32(2):70-80.
[PMID: 8015848]
9. Muthén BO, Muthén LK. Mplus user's guide. 2nd ed. Los Angeles (CA): Muthén & Muthén; 2001.
10. Thompson B. Exploratory and confirmatory factor analysis: Understanding concepts and applications. Washington (DC): American Psychological Association; 2004.
11. Horn JL. A rationale and test for the number of factors in factor analysis. Psychometrika. 1965;30:179-85.
[PMID: 14306381]
12. Zwick WR, Velicer WF. Comparison of 5 rules for determining the number of components to retain. Psychol Bull. 1986;99(3):432-42.
13. Cattell RB. The scientific use of factor analysis. New York (NY): Plenum Press; 1978.
14. Shrout PE. Measurement reliability and agreement in psychiatry. Stat Methods Med Res. 1998;7(3):301-17.
[PMID: 9803527]
15. Anastasi A. Psychological testing. New York (NY): Macmillan Publishing Company; 1988.
16. Embretson SE, Reise SP. Item response theory for psychologists. Mahwah (NJ): Lawrence Erlbaum Associates; 2000.
17. Penn RD. Intrathecal baclofen for severe spasticity. Ann N Y Acad Sci. 1988;531:157-66. [PMID: 3382141]
Submitted for publication April 24, 2006. Accepted in revised form January 26, 2007.

Go to TOP  

Go to the Contents of Vol. 43 No. 5

Last Reviewed or Updated  Friday, July 20, 2007 7:07 AM