JRRD Logo


Volume 44 Number 4 2007
Pages 537 — 546


Utility of Treatment Implementation methods in clinical trial with rehabilitation teams

Alan B. Stevens, PhD;1-2 Dale C. Strasser, MD;3-5* Jay Uomoto, PhD;6 Susan E. Bowen, PhD;7 Judith A. Falconer, PhD, MPH, OTR8

1Scott & White Memorial Hospital, Temple, TX; 2Texas A&M University Health Science Center, College Station, TX; 3Birmingham/Atlanta Geriatric Research, Education, and Clinical Center, Birmingham, AL; 4Atlanta Department of Veterans Affairs (VA) Medical Center, Decatur, GA; 5Emory University, Atlanta, GA; 6Center for Polytrauma Care, Neuropsychology and Rehabilitation Psychology, VA Puget Sound Health Care System (Seattle Division), Seattle, WA; 7Health Sciences Research and Development, Atlanta VA Medical Center, Decatur, GA; 8Northwestern University Feinberg School of Medicine, Chicago, IL

Abstract — Clinical trials of rehabilitation interventions pose unique challenges to researchers. Treatments can be technically complex, often requiring a multidisciplinary team of professions. This article demonstrates the application of Treatment Implementation (TI) methods in a rehabilitation team-training intervention conducted with 29 team leaders (12 medical doctors, 4 physical therapists, 3 speech-language pathologists, 2 occupational therapists, 3 kinesiotherapists, 2 registered nurses, 1 social worker, 1 program coordinator, and 1 administrator) from 15 Department of Veterans Affairs hospitals. We describe the intervention along with the influence of three TI categories (delivery, receipt, and enactment) on the design and implementation of the team-training intervention. Positive findings from the use of TI methods include (1) consistent and accurate presentation of intervention components and (2) evidence of study participants' receipt and enactment of intervention strategies.

Clinical Trial Registration: ClinicalTrials.gov, NCT00237757, <http://www.clinicaltrials.gov>.

Key words: behavioral science, clinical trials, effectiveness, interdisciplinary teams, patient care team, rehabilitation outcomes, research methods, stroke rehabilitation, team training, Treatment Implementation.


Abbreviations: TI = Treatment Implementation, VA = Department of Veterans Affairs.
*Address all correspondence to Dale C. Strasser, MD; Atlanta VA Medical Center, 1670 Clairmont Rd (11B), Decatur, GA 30033; fax: 404-417-2912.
Email:Dale.Strasser@va.gov
DOI: 10.1682/JRRD.2006.09.0120
INTRODUCTION

Most published research reports on rehabilitation interventions focus on specific treatment techniques, with little attention given to the role of the interdisciplinary team, now the dominant model of care delivery in the inpatient setting. Prior studies by this research team demonstrate the importance of team functioning for patient outcomes [1-3]. These studies have characterized the structure of rehabilitation teams, including interprofessional differences among team members, and identified team characteristics that predict patient outcomes. Yet controlled outcomes studies on the effect of team functioning or teamwork on patient outcomes have not proliferated.

Conducting intervention or outcomes studies on interdisciplinary teams has inherent challenges. Given the considerable variance in the structure and function of interdisciplinary teams across rehabilitation providers, limited consensus exists on the characteristics of effective teams. Additionally, the magnitude (type and dose) of rehabilitation care provided depends on numerous factors, such as individual patient complexity, caregiver support, funding sources, and availability of rehabilitation services. Challenges to conducting valid intervention research can, however, be addressed with research methods from psychosocial intervention research [4].

Treatment Implementation (TI) is a well-established methodological framework that guides the delivery of a clinical intervention. By ensuring accurate and consistent delivery of the intervention, TI methods promote both the internal and external validity of the research [5]. The TI framework is based on the fundamental assumptions that a study's reliability and validity depend partially on the researcher's ability to deliver the intervention according to a specific and predefined protocol and that participants perceive and understand the treatment as intended. TI methods enhance internal validity by structuring the delivery of the intervention and by measuring the dose of the intervention delivered to research participants. Furthermore, TI methods document actual change in participants' behaviors in response to the intervention at two levels: (1) their ability to engage in the desired skills or behaviors during intervention sessions and (2) their ability to enact the skills or behaviors outside the intervention or training setting. Documentation that behavior change generalized beyond the study setting proves ecological validity.

This article demonstrates the contributions of a TI framework adapted from social science intervention methods for use in a rehabilitation clinical trial. The intervention was based on a conceptual model of rehabilitation treatment effectiveness in which the rehabilitation team plays a pivotal role in patient outcomes [6-7]. We report specific intervention strategies prescribed by the TI framework and indicators that quantify the dose of intervention delivered to research participants. We also assess knowledge transfer to the research participants and document the extent to which the research participants enacted skills taught in the team-training intervention. Patient outcome data from the clinical trial revealed that patients treated by staff who participated in the team-training intervention demonstrated greater functional improvement than patients treated by the control teams, whose staff received information on team characteristics [8]. (For more details, see NCT00237757, <http://www.clinicaltrials.gov>.)

METHODS

We used the TI framework of delivery, receipt, and enactment to organize a complex behavioral intervention directed at rehabilitation teams. The core component of the intervention was a "train-the-trainer" approach in which two rehabilitation team leaders from each site received training to improve team functioning at their specific hospital. We used the TI framework to measure and promote consistent and accurate presentation of the intervention and to gather information on the study participants' receipt and enactment of the intervention strategies.

Participants and Training Workshop

Staff from 15 Department of Veterans Affairs (VA) inpatient rehabilitation units participated in the experimental condition of a cluster randomized clinical trial of a team-training intervention. Direct contact between research staff and VA sites was limited to two self-identified team leaders (this commonly included the physician and a senior member of one of the rehabilitation disciplines, or nursing staff). The rehabilitation team leaders (research participants) participated in the 2.5-day workshop hosted by the Atlanta VA. Team leaders from two to four different sites were represented during a given workshop. A total of 29 rehabilitation professionals from the 15 intervention sites participated in one of five identical workshops. Rehabilitation medicine was the most commonly represented discipline, with 12 physicians attending one of the workshops. Other professions represented at the workshops included nursing (two registered nurses), physical therapy (four physical therapists), occupational therapy (two occupational therapists), kinesiotherapy (three kinesiotherapists), social work (one social worker), speech-language pathology (three speech-language pathologists), and two rehabilitation administrative personnel. The team-training intervention was delivered by a research project staff of rehabilitation medicine, occupational therapy, and psychology professionals.

Treatment Implementation Procedures

Lichstein et al.'s TI framework [9] influenced the design and implementation of the team-training intervention. Three categories of TI (delivery, receipt, and enactment) prescribed the inclusion of intervention strategies that would increase the probability that study participants successfully received and understood the intervention components. Summaries of the intervention strategies follow and are also detailed in Table 1. To obtain quantitative evidence that each TI category had been considered during the design and implementation of the intervention, we established TI indicators for treatment delivery and enactment. TI indicators documented the use of intervention strategies, the dose of training delivered, and the sequence of intervention activities.


Table 1.
Intervention by Treatment Implementation category.
Intervention Component
Treatment Implementation Category
Delivery*
Receipt
Enactment
Interactive Workshop
(General Skills Training)
· Workshop participation encouraged via travel funds, protected time, & participation of local national leaders.
· Multiple standardized workshops to maximize attendance.
· Problem issue identified & problem solving applied.
· Informal exchange of ideas & experiences at social gathering.
· Coordination with site
principal investigator & local service chief to verify receipt of preworkshop materials.
· Five workshops offered over 12 months to accommodate schedules.
· Customized action plan developed from workshop activities.
· Participants surveyed at end of workshop.
· Postworkshop debriefing by project personnel.
       
Written Documents
(Team-Specific Feedback)
· Rehabilitation team characteristics formatted in user-friendly manner.
· Summary letter on rehabilitation team functioning.
· Summary of problem-solving workshop activity into action plan.
· Receipt of formal site-specific action plans.
· Receipt of team-functioning characteristics & summary letter.
-
       
Financial Resources
(Team-Specific Feedback)
· $1,000 transferred to each site.
· Receipt confirmed by email.
· Project personnel categorize use of funds.
       
Structured Telephone/
Videoconference
(Consultation)
· Semistructured follow-up discussion via telephone or videoconference.
· Follow-up conference
calls with project personnel & team leaders on intervention implementation.
· Implementation survey to participants.
· Project personnel categorize implementation from notes of follow-up conferences.
*Strategies that promote research staff's accurate and consistent delivery of intervention components.
Strategies that enhance research participants' receipt and understanding of intervention components.
Strategies that promote generalization of knowledge and skills to situations outside of intervention training sessions.
Delivery

Fidelity procedures ensured accurate and consistent delivery of intervention activities by the research personnel. The goal was to achieve a match between the written protocol of the intervention and the research staff's actual delivery of the treatment. Before the training, workshop participants received workbooks containing an introductory letter, an overview of the workshop, published articles relevant to the concept of teamwork, patient vignettes framing patient care, and teamwork issues to be discussed during the workshop. We asked participants to review these materials before arriving at the workshop. Funds were provided to participants to eliminate financial barriers to travel. The research staff selected the intervention components and strategies for the workshop and combined them into a comprehensive training program. To ensure consistency and accuracy in the delivery of the workshop, we developed information feedback, individual consultations, and detailed outlines of all intervention components and established a timeline to ensure timely delivery of all intervention activities. All feedback documents followed a template that summarized team members' opinions and attitudes regarding team functioning. The entire research team reviewed and approved all documents before dissemination. We enacted these procedures to promote delivery. Actual delivery was defined as therapeutic contact between research staff and participants, primarily through training workshops, information feedback, and remote consultations. Discrete specifics from each phase of the intervention were determined and measured prospectively.

Receipt

The defining principles of treatment receipt are that the research participant understands the treatment and can engage in the skills or behaviors inherent to the intervention. Receipt requires an effective transfer of knowledge from the research staff to participants. This transfer was achieved with role playing that demonstrated specific skills (e.g., providing instructive feedback to a team member) and contextual information relevant to the environment in which new skills and practices were expected to be used (e.g., using VA Functional Status Outcomes Database to monitor and improve patient outcomes).

Treatment receipt was monitored during workshop sessions through research staff participation and systematic data collection. Research staff conducted observations (e.g., monitoring participants during intervention sessions) and convened throughout the 2.5-day workshops to share notes and comments. Indicators of treatment receipt were active participation in workshop exercises and discussions and verbal confirmation from participants that principles were learned (e.g., probing questions from presenters to assess comprehension of material). Training activities were adjusted to accommodate participants' specific needs. For example, for two of the participating sites, the role of social work and the function of case management were major priorities. For another site, problematic/troublesome staff members were an important topic. When participants identified such themes, research staff incorporated those themes into workshop activities. We monitored receipt of written documents to ensure timely delivery of written information to the designated leaders. We asked research participants to complete confidential questionnaires that evaluated the workshop and assessed receipt of information delivered outside the workshop setting.

Enactment

Enactment of new team-functioning skills was the optimal goal of the training activities. Enactment was defined as generalization of knowledge and skills to situations outside intervention training sessions (i.e., the participant's VA unit). We included characteristics of the intervention (e.g., inclusion of the one-on-one consultations) and provision of monetary resources to encourage the rehabilitation team to enact new skills. Workshop activities stressed transfer of concepts into action. For example, sites were encouraged to use monetary resources provided by the project to initiate team-building activities (e.g., T-shirts that acknowledge obtainment of a unit goal). Indicators of treatment enactment were limited to participants' self-reports.

Assessment of Treatment Implementation

The project used three methods of data collection to comprehensively assess TI indicators. First, research project records were fundamental to the assessment of TI indicators. Research staff maintained detailed records of all intervention activities. Project records provided information on the delivery of training materials and feedback documents to participants, as well as documentation of participant attendance at workshop sessions and consulting sessions with research staff. Second, formal evaluations were conducted at the conclusion of each workshop. Workshop participants were given time at the conclusion of the workshop to confidentially evaluate workshop activities and presenters. Data were summarized across the five workshops. Third, participants were asked to report changes in team skills, new team behaviors, and new programs resulting from their participation in the team-training intervention. A 15-item questionnaire was sent to the research project's primary contact (the senior member of the two individuals represented at the workshops) at each VA approximately 2 months postintervention. We encouraged the participants responsible for the completion of the questionnaire to seek input from other staff members. Direct observation of participants in everyday work settings engaging in team interactions was not possible because of the diverse geographic locations of the 15 intervention sites.

Team-Training Intervention

We designed the rehabilitation team-training intervention to help members of the rehabilitation team gain knowledge and use the new team-functioning skills. The intervention was presented in three phases: (1) general skills training in team process (e.g., team effectiveness and problem-solving strategies), (2) informational feedback (e.g., action plans to address team-process problems and a summary of team-functioning characteristics as reported by rehabilitation staff), and (3) telephone and videoconference consultation (e.g., advice on implementation of action plans and facilitation of team-process skills).

General Skills Training

As the first phase of the intervention, the team intervention provided skills training in topics thought to influence team functioning. Training activities were interwoven with the model of rehabilitation treatment effectiveness [6-7]. The model postulates that team process is a function of team actions and team relations. Team actions exist on a continuum from leadership functions to managerial practices, while team relations exist on a spectrum from social climate to interprofessional relations. The model provided a common framework for understanding team process, created a shared language for workshop participants, and facilitated the discussion of alternative ways of addressing challenges to team functioning. Research staff used a problem-solving framework during training to activate concepts embedded in the model.

Problem-solving skills were presented as a mechanism to address team-process issues. Problem-solving skills effectively foster an environment in which professionals from multiple backgrounds (i.e., physicians, nurses, therapists, and psychologists) can engage in the process of identifying and addressing issues that affect team functioning. The process allows for discussion of current and pressing problems as well as identifies situations that may foster future problems. The problem-solving technique was offered as a way for team members to find effective and workable solutions to specific threats to team functioning.

We used a four-step problem-solving model: (1) Definition and Formulation, (2) Possible or Alternative Solutions (Brainstorming), (3) Decision Making, and (4) Implementation and Verification (e.g., action plans). These four progressive skills represent a basic set of activities for finding and implementing solutions to challenges in clinical practice and have proven successful in other healthcare settings [10].

General skills training was conducted in workshops led by an interdisciplinary research team that included a physiatrist, a geriatric psychologist, a rehabilitation psychologist, an occupational therapist, and a research psychologist familiar with VA rehabilitation inpatient settings. In addition, two clinical administrators from VA Central Office participated in the workshops and provided content and VA policy expertise. Self-identified team leaders from each of the 15 intervention sites participated in one of five identical workshops. The 2.5-day (16 contact hours) workshop was divided into four sessions, which began on a Tuesday afternoon and ended at noon on Thursday. Participants practiced applying learned skills to common problems by using hypothetical situations at several levels, such as challenging patients, hospital cutbacks, unfunded administrative directives, clinical practice guidelines, and accreditation issues. Each workshop participant actively engaged in the process in small groups and then in the larger workshop group, which allowed all to comment, critique, and refine specific strategies.

To ensure the relevancy of the material discussed in the workshop, we used clinically valid vignettes, developed and piloted with other rehabilitation teams not participating in the experiment. Before the workshop, all participants received the same material. One vignette highlighted the challenges posed by a stroke patient's overinvolved spouse, who split staff into "good" and "bad." In another vignette, the patient's poor bladder control could be managed most effectively by an interdisciplinary effort of nursing, therapy, and medical staff. Other vignettes highlighted the challenges to effective team functioning that arise from outside the team, such as organizational influences. Vignettes, role-playing, and other group activities emphasized an amalgam of team-process challenges that generalize across clinical scenarios. These techniques proved to be valuable teaching strategies. While participants worked through hypothetical scenarios, the value of information feedback was incorporated into training activities. This training was based on the assumption that information feedback is critical to the implementation of successful process improvement initiatives [11]. Participants learned about team-functioning profiles based on responses from their team members and were reminded that patient outcome data were available through the VA Functional Status Outcomes Database.

Participants also engaged in formal social activities (e.g., a group dinner in a principal investigator's home) with other participants and project staff over the course of the 2.5-day workshop. During these informal sessions, participants often exchanged experiences from their own hospitals in context to the material that had been presented, giving participants an additional opportunity to learn new skills.

Team-Specific Feedback

The second phase of the intervention was focused on two written feedback documents: a summary of team-functioning characteristics and a formal action plan that addressed problematic areas of team functioning. The team-functioning summary provided feedback in the domains of social climate, interprofessional relations, leadership functions, and managerial practices. Documents were based on a 67-item survey completed by staff members of the participating rehabilitation team before the workshop. Individual team-functioning summaries were not shared across the participating sites, although each site received normative data on the national sample. Copies of the questionnaire and examples representing the four domains are available from the corresponding author (DCS).

Consultation

Consultation, the third intervention phase, was intended to solidify the insights gained from the skills training and written feedback phases and facilitate implementation of new skills and action plans. Two to three months following receipt of the customized action plans and summary of team characteristics, we conducted individual consultations with each intervention site. At least two research project members participated in either a video teleconference (n = 3) or telephone conference (n = 12) with the workshop participants. Each site that participated in these conferences had its individual team performance ratings, along with a letter summarizing the findings and highlighting potential areas on which to focus quality improvement initiatives. Material from the workshop was reviewed along with the action plans and summary of team characteristics. Progress toward implementation of the action plan was discussed and suggestions offered with respect to perceived barriers. Further problem solving and modification of implemented plans occurred during the consultation. In some cases, participants reported successful implementation of their intended plan upon returning to their own rehabilitation team. Others found varying levels of engagement with the quality improvement ideas and suggestions that workshop participants presented to their own team.

RESULTS
Treatment Delivery

Treatment delivery records maintained by the research staff indicate that all 15 participating rehabilitation units received workshop materials before attending the workshop. Two team leaders from 14 of the 15 sites attended one of the four workshops. One site was represented by only one person. Likewise, all 15 rehabilitation units received written documents relevant to the intervention (second phase of the intervention), including annotated information feedback documents delivered to leaders who had participated in the workshop. Fourteen rehabilitation units received and accepted the $1,000 monetary incentive to participate in the study. One site returned the money to the research project, stating that VA regulations prohibited their access to the funds. Leaders of all 15 units received a minimum of one consultation from research staff (third phase of the intervention), while most received between two and four.

Treatment Receipt

Participant responses on the anonymous evaluations conducted at the end of each workshop suggest that research staff delivered a clear and consistent message across the five workshops. When asked to rate the workshop's delivery of its stated goals and objectives (i.e., to emphasize team functioning and how team functioning affects patient outcomes), participants either strongly agreed (94.6%) or agreed (5.4%) that the workshop was successful. Similarly, all participants strongly agreed (81%) or agreed (19%) that the workshop provided skills that help rehabilitation teams analyze, understand, and enhance team functioning. The same pattern of responses was made to statements that the workshop emphasized the link between team functioning and patient outcomes. The majority strongly agreed (54%) or agreed (32%) that the workshop related team functioning to broader issues, such as policy, procedures, and quality management. Five participants were undecided on this question.

Of the 15 VA sites that attended the workshop and received individualized written feedback documents summarizing the team's team-functioning characteristics, 12 sites reported reviewing the document. Three sites failed to respond to our questions on this topic. More than half (7 out of 12, 58%) of the leaders who reported reviewing the document also reported sharing information from the document with other rehabilitation team members. At four of these sites, the document was shared with Š80 percent of the team. All participants (100%) stated that they were better able to design a process improvement action plan for their own teams.

Participants' willingness and ability to use the skills and procedures of the intervention component were assessed by their self-report of the usefulness of specific intervention components. Participants overwhelmingly reported that all intervention content areas were useful. Responses from participants are summarized individually by intervention content in the Figure.


Figure. Usefulness of intervention components as reported by participants of rehabilitation team-training intervention.
Treatment Enactment

Questionnaires completed by representatives from each site suggest that the majority of participating services enacted new team activities and procedures to enhance team functioning. Sites also reported a differential response to the various intervention components. More change (i.e., greater enactment) was associated with workshop training than with written feedback.

Implementation of team activities, including enactment of solutions to the team-functioning challenges revealed during the workshop's problem-solving exercise, was reported by 9 of the 15 sites (60%). These nine sites (responders) implemented one or more changes in their rehabilitation team environment and functioning in response to the workshop-based problem-solving component. Most sites reported that they had implemented three to four strategies to improve team functioning. Strategies included improved team-member communication, implementation of a "rehabilitation round-table meeting" to discuss issues or problems, "pulling" of nursing staff into the concept of rehabilitation, improved communication between the unit secretary and rehabilitation staff (to avoid patients missing therapy), improved employee morale by providing more personal activities (such as monthly birthday parties at lunch), and improved quality of patient care by implementing "morning huddles" to discuss important patient issues. Sites reported sequential implementation of multiple solutions when the initial solution was not feasible or failed to resolve the problem. For example, one site attempted a structured telephone call reminder system to facilitate communication between therapy departments and nursing. This system was later changed to a daily "morning huddle," which "improved communication, efficiencies of staffing." Actions that sites took to address threats to team functioning are summarized in Table 2.


Table 2.
Threats to effective team functioning identified by team leaders and actions to address threats.
Team-Identified Threats
Team Actions (Team-Functioning Domain*)
Nursing Staff Lacked Basic Skills to Generalize Therapy
Services to Nursing Floor
Implemented structured program to crosstrain for nursing & therapy staff (IR, MP).
Posted Functional Independence Measure scores to encourage nursing staff's investment in patients' functional improvement (MP, IR).
Elicited registered nurse to advocate for nursing retention (LF, IR).
   
Demands on Staff in Preparation for Committee on
Accreditation of Rehabilitation Facilities (CARF) Survey
Scheduled potluck lunches & distributed customized team
T-shirts to foster team cohesion (SC).
Established communication sessions for staff to express concerns & for leaders to acknowledge positive staff contributions toward CARF accreditation (SC, IR).
   
Physical Therapy (PT) & Occupational Therapy (OT)
Personnel Do Not Cotreat Patients
Established more inclusive rehabilitation leadership team focused on CARF accreditation (LF, IR).
   
Poor Definition of Case Manager's Role Caused Conflict
Between Nursing & Social Work
Conducted monthly team process meetings (SC, IR, MP).
Added refrigerator, coffee machine, & fax machine for all staff (SC).
   
Lack of Continuity in Nursing Staff
Registered nurses invited to rehabilitation team meetings (IR, MP).
   
Supervisors Wanted to Attend Therapists' Monthly
Roundtable Team Meetings
Physician leader attempted to create more positive environment in roundtable meetings & lessen supervisors' concerns (LF, SC).
Meeting minutes provided to supervisors in lieu of attendance (MP).
   
Passive Involvement of Physician Adversely Affected
Establishment of Interdisciplinary Patient Goals
Established monthly team-process meetings for education, dissemination, & case presentations (MP, SC).
Additional meetings with administrators & supervisors to overcome limited physician involvement (LF, SC).
   
New Team Created by Merger of Two Hospitals
Team leaders conducted informal discussions on importance of staff communication (LF, SC).
Special presentation from outside expert on team dynamics.
   
Professional Roles Not Well Defined Among OT, PT, &
Nursing
PT supervisor initiated regular contact with nursing & OT staff on nursing unit to encourage communication & skill development (LF, IR, MP, SC).
PT supervisor learned & practiced new leadership skills (LF).
   
Provision of Patient & Family Education Not Well
Documented
Created checklist of educational activities to be provided to patients & families (MP, LF).
*Team-functioning domains: interprofessional relations (IR), leadership functions (LF), managerial practices (MP), and social climate (SC). For full review of conceptual model of rehabilitation effectiveness, see Strasser DC, Falconer JA. Linking treatment to outcomes through teams: Building a conceptual model of rehabilitation effectiveness. Top Stroke Rehabil. 1997;4(1):15-27; and Strasser DC, Falconer JA. Rehabilitation team process. Top Stroke Rehabil. 1997;4(2):34-39.

Sites were less responsive to written information that summarized responses from a survey of team-functioning characteristics. The survey data were obtained from members of the rehabilitation unit. Only 5 of the 15 sites reported that the information provided on the feedback document promoted a change in unit activities or the enactment of strategies addressing problems identified in the written feedback. Seven sites reported having made no changes, and three sites did not respond to this question on the self-report questionnaire.

Enactment of new team-functioning activities can be limited by the availability of nonclinical time dedicated to team-functioning activities. Therefore, we asked participating VA sites to rate the time demands of participating in the intervention and, more specifically, of enacting new activities to promote team functioning. Twelve sites (80%), all of which participated in all training, feedback, and consultation activities, felt that the time expended in informal or formal meetings leading to the implementation of new skills was very reasonable (n = 10) or reasonable (n = 2). Two sites reported that the time demands were unreasonable. Eleven sites reported that the time required to actually implement the new skills for team functioning within the clinical environment was very reasonable (n = 7) or fairly reasonable (n = 4). Two sites reported that the time spent in planning or executing team-functioning skills was unreasonable. One site did not respond to these items on the self-report questionnaire.

DISCUSSION

This report demonstrates both the possibilities and challenges of conducting clinical trials in rehabilitation settings. The intervention required multiple skill-training activities delivered sequentially across three phases. Research participants were engaged in various training activities and received information feedback documents, with the goal of improving the overall functioning of the rehabilitation team.

The intervention targeted the behaviors of team members so that team functioning and, consequently, patient outcomes would improve. In rehabilitation, as in other healthcare fields, interventions customarily target patients directly (i.e., patients are in direct contact with the intervention and patient outcomes are assessed). Interventions with rehabilitation staff are relatively less common and often based on the development and use of new skills and changes in behavior among team members, which require unique methodologies and training strategies distinct from clinical trials in which the treatment is delivered directly to the source of outcomes data (e.g., pharmacological trials).

In the case of our team-training intervention, TI methods influenced the design of the study as well as the intervention strategies. We used additional TI methods to monitor the research participants' responses to the intervention. Our application of TI to a complex intervention is consistent with Bellg and colleagues, who, writing for the National Institutes of Health Behavior Change Consortium, emphasized the importance of rigorous attention to TI in randomized clinical trials that target a change in the participant's behavior [5]. This study also followed their recommendation that the participants' responses to the intervention be assessed at two levels: their ability to engage in the desired skills or behaviors during intervention sessions and their ability to enact the skills or behaviors outside the intervention or training setting.

Additional support for the feasibility and efficacy of using TI is provided by Burgio and colleagues, who delineated TI application across six psychosocial interventions for family caregivers of persons with dementia [4]. Building on this work, investigators systematically employed TI techniques in the second phase of this family caregiving research: the Resources for Enhancing Alzheimer's Caregiver Health II trial [12]. This trial faced the unique challenge of implementing a multicomponent psychosocial intervention across five geographically distinct research sites. TI methods structured each component of the intervention and TI indicators quantified treatment implementation across intervention components and research sites. TI data provided evidence of the internal validity of the clinical trial. The investigators also used the TI data to judge research participants' receptivity to the multiple intervention components. For example, participants were less likely to participate in a telephone-based support group than in face-to-face supportive interactions with a trained interventionist [12].

The experience of our project supports the use of TI procedures in rehabilitation research. Fidelity procedures strengthened the design of the project and increased the internal and external validity of the research. Internal validity was supported by documentation that the intervention was delivered as intended. TI procedures helped organize the information content and the presentation of the content to participants, making the intervention more practical for use in a real-world setting. Staff and patient outcomes from this project will be easier to interpret because of these fidelity procedures. Internal consistency of implementation of the intervention protocol allowed treatment to be delivered equivalently across the treatment groups. This consistent delivery permits maximum external validity of the protocol and is an equally important dimension, since the intervention rests on the team's willingness to apply effective team-functioning practices to the healthcare environment.

CONCLUSIONS

Rehabilitation researchers should consider using TI methods when designing and implementing complex rehabilitation interventions. Our experience suggests that TI methods allow consistent and reliable organization of the information content and delivery and provide a framework for monitoring the actual enactment of the intervention. TI does not offer specific content but rather a way to organize content to enhance potency and allow for its implementation in different locations. The ability to consistently replicate any intervention is essential to rigorous research in clinical medicine.

ACKNOWLEDGMENTS

Parts of this material were presented in preliminary form at the annual meetings of the Association of Academic Physiatrists (paper presentation; Tucson, Arizona; February 2005) and the American Congress of Rehabilitation Medicine (poster presentation; Chicago, Illinois; September 2005).

This material was based on work supported by the VA, Rehabilitation Research and Development Service (merit review grants B2367R and 03225R).

The authors have declared that no competing interests exist.

REFERENCES
1. Strasser DC, Smits SJ, Falconer JA, Herrin JS, Bowen SE. The influence of hospital culture on rehabilitation team functioning in VA hospitals. J Rehabil Res Dev. 2002;39(1): 115-25. [PMID: 11926323]
2. Smits SJ, Falconer JA, Herrin JS, Bowen SE, Strasser DC. Patient-focused rehabilitation team cohesiveness in Veterans Administration hospitals. Arch Phys Med Rehabil. 2003; 84(9):1332-38. [PMID: 13680570]
3. Strasser DC, Falconer JA, Herrin JS, Bowen SE, Stevens AB, Uomoto JM. Team functioning and patient outcomes in stroke rehabilitation. Arch Phys Med Rehabil. 2005; 86(3):403-9. [PMID: 15759219]
4. Burgio L, Lichstein KL, Nichols L, Czaja SJ, Gallagher-Thompson D, Bourgeois M, Stevens AB, Ory M, Schulz R, REACH Investigators. Judging outcomes in psychosocial interventions for dementia caregivers: The problem of treatment implementation. Gerontologist. 2001;41(4):481-89.
[PMID: 11490046]
5. Bellg AJ, Borrelli B, Resnick B, Hecht J, Minicucci DS, Ory M, Ogedegbe G , Orwig D, Ernst D, Czajkowski S, Treatment Fidelity Workgroup of the NIH Behavior Change Consortium. Enhancing treatment fidelity in health behavior change studies: Best practices and recommendations from the NIH Behavior Change Consortium. Health Psychol. 2004;23(5):443-51. [PMID: 15367063]
6. Strasser DC, Falconer JA. Linking treatment to outcomes through teams: Building a conceptual model of rehabilitation effectiveness. Top Stroke Rehabil. 1997;4(1):15-27.
7. Strasser DC, Falconer JA. Rehabilitation team process. Top Stroke Rehabil. 1997;4(2):34-39.
8. Strasser DC, Falconer JA, Herrin J, Stevens AB, Uomoto JM, Herrin JS, Bowen SE, Burridge AG. Team training and stroke rehabilitation outcomes: A cluster randomized trial. Arch Phys Med Rehabil. In press 2007.
9. Lichstein KL, Riedel BW, Grieve R. Fair tests of clinical trials: A treatment implementation model. Adv Behav Res Ther. 1993;16(1):1.
10. D'Zurilla TJ, Nezu AM. Problem-solving therapy: A social competence approach to clinical intervention. 2nd ed. New York (NY): Springer Publishing Co; 1999.
11. Hayes RP, Ballard DJ. Review: Feedback about practice patterns for measurable improvements in quality of care-A challenge for PROs under the Health Care Quality Improvement Program. Clin Perform Qual Health Care. 1995;3(1): 15-22. [PMID: 10141395]
12. Belle SH, Burgio L, Burns R, Coon D, Czaja SJ, Gallagher-Thompson D, Gitlin LN, Klinger J, Koepke KM, Lee CC, Martindale-Adams J, Nichols L, Schulz R, Stahl S, Stevens A, Winter L, Zhang S, Resources for Enhancing Alzheimer's Caregiver Health (REACH) II Investigators. Enhancing the quality of life of dementia caregivers from different ethnic or racial groups: A randomized, controlled trial. Ann Intern Med. 2006;145(10):727-38. [PMID: 17116917]
Submitted for publication September 28, 2006. Accepted in revised form February 27, 2007.

Go to TOP

Go to the Contents of Vol. 43 No. 5

Last Reviewed or Updated  Wednesday, August 22, 2007 7:30 AM