Journal of Rehabilitation Research & Development (JRRD)

Quick Links

  • Health Programs
  • Protect your health
  • Learn more: A-Z Health
Veterans Crisis Line Badge
Guest Editorial

Volume 49 Number 1, 2012
   Pages 83 — 86

Rehabilitation Research and Development state-of-the-art conference on outcome measures in rehabilitation

Timothy R. Elliott, PhD, ABPP

Outcomes, by and large, remain the ultimate validators of the effectiveness and quality of medical care.– Avedis Donabedian, 1966 [1, p. 196]

In January 2010, the Veterans Health Administration (VHA) Office of Research and Development brought clinical, scientific, and policy experts from a variety of professions to Miami, Florida, for a 3-day State-of-the-Art (SOTA) Conference on outcome measures in rehabilitation. The need for such a SOTA is obvious in light of the clinical complexity and long-term nature of injuries incurred by service personnel returning from international conflicts, combined with the ongoing commitment to aging Veterans who receive their healthcare from the VHA. Less obvious, perhaps, are other pressing matters relevant to the provision of quality care to Veterans with chronic and disabling conditions.

First, the United States is experiencing an unprecedented need for measures of quality care in health service delivery. This movement may be best realized in the emergence of the Patient Reported Outcome Measurement Information System (PROMIS;, funded by the National Institutes of Health, for the development, refinement, and use of patient-reported outcomes to benefit consumers, clinicians, researchers, service delivery systems, and policymakers. This ambitious endeavor will promote standards and the implementation of qualitative and quantitative methods in outcome measurement. Across the country, several funded projects are investigating the feasibility and applicability of measures for use with persons who have chronic and disabling conditions. Ideally, measures that emanate from PROMIS will be reliable, valid, sensitive to change, and clinically useful for assessing outcomes across physical, mental, and social domains. Furthermore, these measures should be useful and informative to all health professions–regardless of specialty or discipline–invested in quality care and meaningful outcomes.

Second, the push to develop sensitive patient-oriented outcome measures for all health services is consonant with the ascending International Classification of Functioning, Disability, and Health (ICF) [2]. The ICF emphasizes a greater appreciation for the environmental context in which disability–secondary to any diagnostic condition–occurs, and it advocates assessment of specific body functions and structures, activities and participation (and the capacity for and performance of such), and environmental barriers and facilitators of activity and participation [3]. This formula goes beyond the assumed outcome of symptom reduction and anticipates the acquisition and resumption of desired activities in personal and social roles [4].

Although this framework has been described as a –workable compromise between medical and social models– [3, p. 5], its utility in clinical practice and research is contingent on interdisciplinary collaborations. Indeed, interdisciplinary collaboration is one of the major issues facing clinical service and informed research to benefit returning Veterans with acquired disabilities. The complex nature of polytrauma and its clinical spectrum of posttraumatic stress disorder (PTSD) with brain injury, burns, limb loss, neuromuscular disability (including spinal cord disorders), and sensory loss (vision and hearing impairments, specifically) challenges clinicians and scientists to work in a unified, collaborative fashion to identify best practices and deliver optimal service. Unfortunately, the relative lack of co-occurrence among these conditions in the clinic and in the literature before 2006 (particularly brain injury and PTSD) [5] complicates interdisciplinary endeavors. Traditionally, the predominating diagnostic condition dictates the health specialty that provides and coordinates subsequent service to the Veteran (particularly PTSD and psychiatry, brain injury and rehabilitation). The long-standing boundaries between specialty services contribute to an unfortunate scenario in which each discipline tends to feel superior, dismisses the work of others, and sees interdisciplinary science as –second rate– [6]. But optimal outcomes for our Veterans are contingent upon effective interdisciplinary collaborations in science and practice, including a shared set of meaningful outcome measures of value to the Veteran and to each specialty invested in providing quality service and care. In the quest for meaningful and patient-driven outcomes for Veterans with chronic, complex conditions, no single specialty has a corner market on truth.


The conference convened on the evening of ??January 26, 2010. Following opening remarks from Patricia A. Dorn, PhD (interim director of the Office of Rehabilitation Research and Development for the Department of Veterans Affairs [VA]) and Walter Penk, PhD (conference chair), Lynn Bufka, PhD (assistant executive director of the American Psychological Association) addressed the group on the –International Classification of Functioning.– The following morning, Michael E. Selzer, MD, PhD (director, Shriners Hospital Pediatric Research Center in Philadelphia, Pennsylvania) presented an overview and charge to the work groups. Groups were tasked with determining the best methods for identifying and recommending best measures and ??practices for conducting research and studying rehabilitation outcomes of importance to Veterans, families, policymakers, and clinical service. For the remainder of that day and for the opening hours of the following morning, participants were assigned to small work groups to identify and critique prominent measures of activity and participation outcomes for several clinical issues directly relevant to the VHA rehabilitation mission: traumatic brain injury, chronic mental illness, spinal cord injury, community reintegration, and vocational services. Another group was impaneled to identify contemporary approaches and develop recommendations for research methodologies for studies with small samples (and N = 1 designs) common in rehabilitation settings because of the complexities and often ??individualized needs imposed by various comorbidities, injuries, and polytrauma. Advanced graduate students from the accredited Counseling Psychology doctoral program at Texas A&M University attended the conference to take notes for each group. On the final day of the conference (January 28), a representative from each group gave a brief presentation of the issues and direction of the group.

In the months after the conference, the groups prepared and submitted manuscripts to the Journal of Rehabilitation Research and Development (JRRD) for review and possible publication. Experts from diverse backgrounds were invited to serve as ad hoc reviewers. Essentially all invited reviewers volunteered their time and energy to provide timely and insightful critiques. The final set of manuscripts were thus subjected to a rigorous peer review process, and in our opinion, they were burnished to ensure the quality expected by the JRRD readership. Unfortunately, little consensus resulted from the limb loss group; consequently, no manuscript from this group was submitted for review.

Ideally, the set of articles included in this issue–s single-topic section will address the needs of Veterans and their families, address the needs of clinicians and the VA system, inform policymakers, and advance our understanding of best practices in the rehabilitation of our Veterans.

We anticipate that a second SOTA will be convened in late 2012 or early 2013 to expand on the work of the first SOTA. In order to identify and implement best practices in rehabilitation, we must be able to compare results across studies. The task for policymakers within the VA of identifying best treatment practices from the extant research literature has been complicated by the fact that rehabilitation research studies, even within the same diagnostic category, often use different outcome measures. This has complicated efforts to identify best treatment and cost-effective practices in rehabilitation. An immediate goal of the Office of Rehabilitation Research and Development is to encourage VA rehabilitation researchers to use common outcome measures to facilitate the translation from research to practice. Therefore, one focus of a follow-up SOTA would be to help VA rehabilitation researchers identify and utilize common outcome measures. The SOTA would identify outcome measures that have been developed and are considered the –gold standard,– that are being developed, and that need further refinement.

Translation from research to clinical practice is often impeded by cost considerations. Policymakers are hesitant to implement innovative treatments if they are perceived to be too costly. Many research studies in rehabilitation have demonstrated effective treatments that have simply –withered on the vine– because decisionmakers deemed them too costly to implement in clinical practice. Researchers are obliged to demonstrate that proposed treatment innovations are cost-effective in the long term even when short-term costs go up (as is usually the case). To enable VA researchers to meet this obligation, a follow-up SOTA would include health economists with a focus on rehabilitation.

Finally, there is a pressing need in rehabilitation research to identify and understand qualitative factors that affect the outcome of rehabilitation interventions. Why patients –fail– or –succeed– in treatment often cannot be understood solely through the use of quantitative methods. A variety of qualitative approaches are currently in use by medical anthropologists and others working in rehabilitation research. Research proposals using qualitative approaches often do not receive funding because reviewers, who are sometimes not very knowledgeable about qualitative approaches, justly or unjustly deem them to lack a requisite rigor. Another goal of a follow-up SOTA, then, would be to achieve consensus about how qualitative approaches should be conducted and what criteria should be applied to evaluate qualitative research proposals.

Appreciation is extended to Sunil K. Sen-Gupta, PhD, MPH, the Scientific Program Manager for the Office of Research and Development in the VA, for initiating and coordinating the SOTA. We are indebted to all who participated in the conference, and we are particularly grateful to Stacieann Yuhasz, PhD, and Lloyd C. Tinker, for their guidance and assistance during this process. We are also grateful to the colleagues who served as ad hoc reviewers.

The author is indebted to Allen Heinemann for bringing the Donabedian quote to his attention.

Timothy R. Elliott, PhD, ABPP
Texas A&M University, Department of Educational Psychology, College Station, TX
ICF: International Classification of Functioning, Disability and Health. Geneva (Switzerland): World Health Organization; 2001.
Reed G–, Lux J, Bufka L, Trask C, Peterson D, Stark S, Threats TT, Jacobson J, Hawley J. Operationalizing the International Classification of Functioning, Disability and Health (ICF) in clinical settings. Rehabil Psychol. 2005;50:122–31.
Bombardier CH, Fann JR, Temkin N, Esselman PC, Pelzer E, Keough M, Dikmen S. Posttraumatic stress disorder symptoms during the first six months after traumatic brain injury. J Neuropsychiatry Clin Neurosci. 2006;18(4):501–8.
[PMID: 17135375]
Pellmar R, Eisenberg L. Bridging disciplines in the brain, behavioral, and clinical sciences. Washington (DC): National Academies Press; 2000.
Chwalisz K, Shah SR, Hand KM. Facilitating rigorous qualitative research in rehabilitation psychology. Rehabil Psychol. 2008;53(3):387–99.
This article and any supplementary material should be cited as follows:
Elliott TR. Guest Editorial: Rehabilitation Research and Development state-of-the-art conference on outcome measures in rehabilitation. J Rehabil Res Dev. 2012; 49(1):83–86.

Go to TOP

Last Reviewed or Updated  Wednesday, March 7, 2012 8:14 AM