VA Research and Development LOGO

Logo for the Journal of Rehab R&D
Volume 42 Number 5, September/October 2005
Pages 693 — 704


Use of the Delphi Technique for developing national clinical guidelines for prescription of lower-limb prostheses

Harmen van der Linde, MD, PhD;1* Cheriel J. Hofstad, MSc;1 Jacques van Limbeek, MD, PhD;1
Klaas Postema, MD, PhD;2 Jan H. B. Geertzen, MD, PhD2

1Rehabilitation Centre Sint Maartenskliniek, Nijmegen, the Netherlands; 2Department of Rehabilitation, University Hospital Groningen, Groningen, the Netherlands
Abstract — The aim of this project was the development of evidence- and consensus-based clinical practice guidelines for lower-limb prosthesis prescription for achieving transparency and consensus among clinicians, manufacturers, and insurance companies. This article describes a modified Delphi Technique, which is based on different methods of collecting evidence, and its role in the development of national clinical guidelines for prosthesis prescription. We used a multimethod approach to develop guidelines for the clinical practice of prosthesis prescription for lower-limb amputees. The Delphi Technique was central in the process, and the panel was made up of experts from three key disciplines on a national level. Our approach involved various methods: a systematic review, a survey of national clinical practice on prosthesis prescription, and interviews with experts. These activities resulted in 45 postulates about prosthesis prescription. The views of the national expert panel were then presented at a consensus development conference. The participants in the Delphi Technique sessions reached a consensus on 37 of the postulates on prosthesis prescription for lower-limb amputees. The postulates were categorized according to amputation level and partitioned into different domains. The total process resulted in the development of draft clinical guidelines comprising guidance for prescribing prostheses for the lower limb. The scope and applicability of these guidelines will have to be measured and evaluated in future work.
Key words: artificial limbs, consensus-based guideline development, consensus development conference, Delphi Technique, lower-limb amputation, lower-limb prosthesis prescription, practice guidelines, prosthesis prescription format.

Abbreviations: ADL = activities of daily living, ICT = information and communications technology.
This material is the result of work supported with resources of the Dutch Health Care Insurance Board, Diemen, the Netherlands.
*Address all correspondence to Harmen van der Linde, MD, PhD; Rehabilitation Centre Sint Maartenskliniek, PO Box 9011, 6500 GM Nijmegen, the Netherlands; +31-24-365-9329; fax: +31-24-365-9154. Email: h.vanderlinde@maartenskliniek.nl
DOI: 10.1682/JRRD.2003.11.0172
Introduction

Broad interest exists in improving the quality of healthcare, not only among clinicians and consumers but also among politicians and insurers. Clinical guidelines are important clinical tools for improvement. These guidelines are systematically developed postulates that guide practitioners in making decisions about appropriate treatments and healthcare for patients [1-4]. Clinical guidelines ought to be based on the best evidence available and, when possible, on scientific research. However, aspects of prosthetics care exist for which little literature is available.

The Dutch College of Health Care Insurances and the Dutch Ministry of Health Care commissioned the Dutch Society of Physical and Rehabilitation Medicine to develop national guidelines for the prescription of lower-limb prostheses. Currently, prosthesis prescription for lower-limb amputees is based primarily on empirical knowledge. Many options exist for prosthetic components; however, prescription criteria can be derived from the experiences of physicians, therapists, prosthetists, and patients [5-6]. On the other hand, third-party payers frequently ask for solid justification for prescription of costly prostheses [6]. Therefore, patients with identical clinical problems may receive different care depending on their clinician, hospital, and/or location. These variations in service among providers, hospitals, and geographic regions are of interest because of the assumption that at least some of these variations stem from inappropriate care. These variations can also be caused by a lack of accurate scientific information about prosthetic components and biomechanical capabilities of the patient. Hence, an intrinsic desire exists for healthcare professionals to offer-and for patients to receive-the best possible care [3]. Developing guidelines is seen as one of the most potentially useful tools for achieving changes in behavior and, therefore, more uniform, high-quality care [7]. Clinical guidelines also make healthcare more consistent and efficient and may highlight knowledge gaps in the available literature [3].

The first step is to extract as much scientifically based knowledge from the literature as possible [8]. However, some difficulties exist in the use of results from studies on biomechanical aspects and functional characteristics of certain prosthetic components for prescription criteria [9]. Outcome measures differ from study to study; therefore, comparison or meta-analysis of the results is difficult. However, explicit knowledge derived from literature is needed to develop clinical guidelines [3].

Despite a huge amount of literature, considerable gaps remain in our formal clinical knowledge of the effects of different prosthetic components and their mechanical characteristics on human functioning with a lower-limb prosthesis [9]. Therefore, with regard to prosthetic guideline development, we must still, to a large extent, rely on clinical consensus among experts. The integration of knowledge from research together with the expert opinions of clinical professionals and the opinions and wishes of consumers can form a solid basis for guideline development for prosthesis prescription.

To create consensus-based clinical guidelines, we should use a method that creates consensus. The ability to make effective decisions in situations where contradictory or insufficient information exists has led to an increased use of consensus methods: namely, brainstorming, nominal group techniques, and the Delphi Technique [10]. The Delphi Technique was originally developed in the 1950s by Olaf Helmer and Norman Dalkey as an iterative, consensus-building process to forecast future events. It has since been deployed as a generic strategy to develop consensus and make group-based decisions in a variety of fields [11].

The Delphi Technique is a structured communication aimed at producing detailed critical examination and discussion, not a quick compromise. The Delphi Technique may be characterized as a method for structuring a group communication process so that a group of individuals can deal with complex problems [12]. Several modifications of the method exist and the computer-based, or electronic version, is one of them. An important property of the computer-based Delphi Technique is that members of a group can participate in an asynchronous manner. A participant can take part in the group communication process when they want and only contribute to those aspects that they feel best able to contribute. In a face-to-face approach, the participants have to take a sequential path through a group problem-solving process. The Delphi Technique allows the individual participant to express a personal judgment [11].

Perhaps the property that most characterizes the Delphi Technique is the anonymity in participants' responses. The objective of this is to allow the introduction and evaluation of ideas and concepts by removing some of the common biases that normally occur in a face-to-face group process [11].

The aim of this project was the development of evidence- and consensus-based clinical practice guidelines for lower-limb prosthesis prescription for achieving transparency and consensus among clinicians, manufacturers, and insurance companies. This article describes a modified Delphi Technique, based on different methods of collecting evidence, and its role in the development of national clinical guidelines for prosthesis prescription.

Methods

Our pragmatic approach for developing guidelines for adults with lower-limb amputations required various methods: a systematic review, a survey of national clinical practice on prosthesis prescription, and interviews with experts. The views of a national expert panel who used the Delphi Technique were then combined with a consensus development conference. The overall process of developing the guidelines is shown in Figure 1.


Figure 1.Prosthetic guideline development process. Time frame is 18 months.
Sources of Evidence
Systematic Review

We performed a systematic literature analysis of clinical studies to identify the issues of human functioning with a lower-limb prosthesis, in accordance with the criteria of the Cochrane Collaboration [13]. For our purpose, two types of studies can be distinguished: (1) clinical studies that address subjects' motor performance and/or activities of daily living (ADL) with a lower-limb prosthesis and (2) technical studies that focus on the mechanical characteristics of prosthetic components, without addressing human functioning. For clinical guideline development, only studies of motor performance and ADL were considered relevant. All relevant studies were assessed with a checklist of 13 criteria for internal, statistical, and external validity. The studies were divided into three levels of evidence according to these criteria [9].

Survey of Clinical Practice on Prosthesis Prescription

Recommendations based solely on practitioners' judgment and clinical experience are likely to be more susceptible to bias and self-interest. Therefore, after deciding what role the expert opinion has to play, our next step was to decide how to collect and assess expert opinion. Currently no optimum method for this exists, but the process needs to be as explicit as possible [8]. A multicenter, cross-sectional study was carried out for observation of the prosthesis prescriptions of a group of lower-limb amputees in the Netherlands. The purpose of the study was to get insight into possible similarities in prescription criteria in practice and determine if prosthesis prescription was based primarily on the amputee's level of activity or the intended use (ADL performance or functional capabilities) of the prosthesis. Data were collected from inpatient and outpatient amputees.

Data on current clinical prosthesis prescription practices (implicit knowledge) were gathered during visits of two members of the project team to the consultation hours in 16 rehabilitation clinics throughout the Netherlands. The results of this study are published elsewhere [14].

Interviews with Experts

To collect implicit knowledge about prosthesis prescription, a research assistant contacted local consultants. Semistructured interviews were conducted that covered prosthesis prescription of transfemoral, transgenual, and transtibial amputations [15].

Delphi Technique Procedures

From the existing consensus methods, we chose the Modified Delphi Technique, which was developed by the Rand Corporation (Santa Monica, California) [16]. It is purported to be the most commonly used method for developing clinical guidelines [17]. In the present study, this formal consensus method consisted of two postal rounds and a final consensus meeting. The two postal rounds were conducted via the Internet. An advantage of the computer-mediated Delphi Technique is "collective intelligence." This is the ability of a group to produce a result that is of higher quality than any single individual in the group could achieve on his or her own. This rarely occurs in face-to-face groups [11].

The Project Team

A project team was formed to initiate this research; it consisted of all the authors of this article. The project team comprised a methodologist, who is also a clinician and who has a background in statistics; three specialists in rehabilitation medicine, who specialize in amputation and prostheses; and a research assistant. The project team's responsibilities were editing the postulates, selecting the participants, developing the questionnaires, and the analyzing the responses. During the Internet postal rounds, the project team was assisted by an information and communications technology (ICT) company that specializes in the computerized Delphi Technique. This company served only as a technical advisory organization and did not actively participate in the guideline development process.

Selection of Participants

Groups whose activities would be covered by the guidelines or who had other legitimate reasons for having input into the process participated in the guideline development [17-18]. This is important for ensuring adequate discussion of the evidence (or its absence) when guidelines are developed from the recommendations [8]. Therefore, the participants in this project were physicians (66%), prosthetists (25%), and physiotherapists (9%), who specialize in rehabilitation of amputees and prosthesis prescription. We formed a participant group of 32 members who represent the above-mentioned three key disciplines in the fields of amputation and prosthesis prescription in the Netherlands.

Selection of Postulates

The postulates for the Delphi Technique developed by the project team combined information from a systematic literature review [9], a survey of clinical practice on prosthesis prescription [14], and interviews with experts [16] (Table 1 for examples of postulates). The postulates concerned functional and technical aspects of prosthesis prescription. Aspects of cost or cost control were not considered. The postulates were graded according to their evidence as follows-

1. Based on a well-performed, randomized, controlled trial with sufficient control for confounding factors.
2. Based on a randomized trial with some control for confounding factors/extraneous variables.
3. Based on limited scientific evidence that does not meet all the criteria considered.
4. Based on opinion of expert clinicians.

Table 1.
Examples of foot postulates for Delphi Technique and citations from different information sources.
Postulate*
Citations
Energy-storing foot indicated for highly active transtibial amputees.
Highly active transtibial amputees able to walk at variable speeds and grades prefer energy-storing feet to conventional ones [1-2].
Information from interviews clarifies that clinicians prescribe Flex-Foot more often for young, active amputees than less active ones.
For active people, especially those involved in jumping sports, Flex-Foot is more suitable for sports activities, but in all likelihood it will be too lively for comfortable use for other activities [3].
   
Reaching foot flat early in stance phase is important parameter for choice of prosthetic foot.
Compared with Seattle and Flex-Foot, a single-axis foot reached foot flat earlier, which promotes preservation of limb stability [4-6].
Information from interviews clarifies that clinicians prescribe SACH foot more often because it reaches foot flat earlier in stance phase, which means stability and feeling of safety for amputee.
1. Casillas JM, Dulieu V, Cohen M, Marcer I, Didier JP. Bioenergetic comparison of a new energy-storing foot and SACH foot in traumatic below-knee vascular amputations. Arch Phys Med Rehabil. 1995;76(1):39-44.
2. MacFarlane PA, Nielson DH, Shurr DG, Meier K. Gait comparisons for below-knee amputees using a Flex-Foot versus a conventional prosthetic foot. J Prosthet Orthot. 1991;3(4):150-61.
3. Menard MR, McBride ME, Sanderson DJ, Murray DD. Comparative biomechanical analysis of energy-storing prosthetic feet. Arch Phys Med Rehabil. 1992;73(5):451-58.
4. Perry J, Boyd LA, Rao SS, Mulroy SJ. Prosthetic weight acceptance mechanics in transtibial amputees wearing the Single Axis, Seattle Lite, and Flex-Foot. IEEE Trans Rehabil Eng. 1997;5(4):283-89.
5. Postema K, Hermens HJ, Vries de J, Koopman HF, Eisma WH. Energy storage and release of prosthetic feet. Part 1: Biomechanical analysis related to user benefits. Prosthet Orthot Int. 1997;21(1):17-27.
6. Rao SS, Boyd LA, Mulroy SJ, Bontrager EL, Gronley JK, Perry J. Segment velocities in normal and transtibial amputees: prosthetic design implications. IEEE Trans Rehabil Eng. 1998;6(2):219-26.
*Grade of evidence 1; based on a well-performed, randomized controlled trial, with sufficient control for confounding factors.
SACH = solid ankle cushion heel.
Delphi Internet Postal Rounds

The ICT company developed a Web site where the participants could enter a personal code and password, after which the pages with the postulates opened. Participants were asked whether they did or did not agree with the postulates. We invited participants to give reasons for their choices [19]. The participants were given the opportunity to react to the arguments of the other (anonymous) participants [19]. These comments were added to the matching postulates and presented on the Web site immediately.

Two Internet Delphi rounds were considered sufficient to reach consensus; consensus was defined as a "general agreement of a substantial majority" (>75%) [19].

The first Delphi round (Delphi-1) consisted of 45 postulates. The project team analyzed every postulate and the comments on the postulates. When there was general agreement of >75 percent, the postulate was entered into the set of accepted draft clinical guidelines. In the case of 60 to 70 percent agreement, postulates were changed with the aid of the participants' comments. In the second Delphi round (Delphi-2), the participants were asked whether they agreed with the modified postulates or not. A few newly formulated postulates were presented in Delphi-2, which were developed from participants' comments on postulates from Delphi-1. Participants were invited to give reasons for their decisions for these newly formulated postulates only. Postulates with no agreement (40%-60% agreement) were included in the consensus development conference (Figure 1). The project team had only an editorial role in modifying the postulates.

After Delphi-2, the project team drafted a feedback report and distributed it to the participants to inform them of the opinions and arguments of their colleagues.

Consensus Development Meeting

In a consensus development conference, a selected group is brought together to consider certain topics and, in light of information presented there, attempt to reach a consensus. However, the group is also encouraged to include minority or alternative views when consensus cannot be achieved [17]. Formal methods ensure that all members have a chance to voice their views, all options are discussed, feedback is provided, and judgments are made confidentially [17]. A chairperson is one of the most important elements in a successful conference; he or she facilitates the exchange of relevant information [17]. Groups generate more alternatives when leaders encourage members to present different opinions rather than encourage consensus [8,17]. Leaders stimulate discussion and allow the group to identify genuine agreement but do not contribute their own opinion in the process. The Delphi consensus meeting was chaired by a member of the project team with both clinical and group process skills. The chairman helped ensure that the process ran smoothly and that good-quality decisions were made.

Participants of the consensus development meeting discussed the postulates for which no agreement (40%-60%) had been reached and the postulates for which a minor agreement (60%-75%) had been reached in Delphi-2. After the discussion of each postulate, participants voted anonymously. Eventually, participants had to vote on the domains in which accepted draft guidelines should be placed and whether an accepted draft guideline should be prescriptive or additive.

Results
Participants

For the expert panel, we started with 32 persons; i.e., 21 physicians, 8 prosthetists, and 3 physiotherapists, of whom 32 (100%) responded at Delphi-1 and 31 at Delphi-2. At the consensus development meeting, 12 physicians, 5 prosthetists and 2 physiotherapists (60%) were present. The primary reason mentioned for not attending the meeting was lack of time.

Delphi-1

All 32 participants responded. Many comments on the postulates were received and analyzed by the project team. The feedback report of Delphi-1 presented all the items with the expert agreement scores in percentages. Eleven postulates reached major agreement and were included in the draft clinical guidelines. Twenty-three postulates reached minor agreement and were reformulated and presented in Delphi-2. Eleven postulates reached no agreement (40%-60%) and were included in the consensus development conference (Figure 2).


Figure 2. Expert agreement percentage on postulates at different stages of Delphi Technique for development of draft clinical guidelines. > 75% means more than 75% of participants agreed with  postulate (bold), < 60%  means no agreement on postulate, 60-75%  means minor agreement on postulate.
Delphi-2

This round consisted of 23 postulates. Of these, 15 reached major agreement and were included in the draft clinical guidelines. Six postulates reached minor agreement and were included in the consensus development meeting, together with the two postulates that reached no agreement (Figure 2).

Consensus Development Meeting

After the two Delphi rounds, 19 postulates reached no or minor agreement. Because of a fully scheduled consensus development meeting, the project team deleted three postulates: postulates with the smallest level of evidence that also showed some overlap with other postulates. The participants discussed the 16 remaining postulates with no or minor agreement. After each postulate, participants voted anonymously whether they agreed or not. At the meeting, 11 postulates reached major consensus and 5 postulates did not reach any consensus and were subsequently excluded from the guidelines.

Eventually 37 postulates reached major agreement (Figure 2). These postulates were categorized with respect to amputation level and partitioned into subdomains with respect to components (Table 2).


Table 2.
Subdivision of domains by amputee type.
Amputee Type
Domains
Transtibial
General
Socket
Foot
-
Transgenual
General
Socket
Knee
Foot
Transfemoral
General
Socket
Knee
Foot

The postulates were categorized according to the subdivision table by the participants; they subsequently decided whether the postulates were prescriptive or additive. The prescriptive postulates were prioritized within each subdomain. Some postulates fit into multiple amputation levels and/or domains.

Draft Guideline Specific Format

By categorizing and prioritizing the postulates, the participants created a specific format for the draft guidelines (Figure 3).


Figure 3. Format of draft guidelines.

The feedback report of the consensus development meeting was sent to all participants. It presented all the postulates and their expert agreement scores in percentages and the format for the draft version of the guidelines. Participants were given the opportunity to make comments, which were incorporated into the draft of the guideline.

These draft guidelines included-

A summary of all the postulates and the expert agreement percentages.
Agreement percentages with respect to the participants' field of expertise; i.e., specialists in rehabilitation medicine, physiotherapists, and prosthetists.
A philosophy of care that made suggestions about the environment within which the recommendations in the guidelines should be implemented.
Evidence-based recommendations to identify which amputees could wear certain prosthetic:
- Feet.
- Knees.
- Sockets.
Discussion

In this project, we used a multimethod approach to develop guidelines for clinical practice of prosthesis prescription for lower-limb amputees. The Delphi Technique was central in the process of achieving consensus on postulated guidelines among a panel of experts in three key disciplines involved in prosthetics prescription. The format of the prescription guidelines that were developed consisted of 37 postulates (Tables 3-5); these were based on the scientific evidence from a systematic review of the literature and integrated with the expert opinions of clinicians. The total process resulted in the development of draft clinical guidelines for prescription of prostheses for lower-limb amputees.


Table 3.
Prescription format for prostheses for transtibial amputees with expert agreement percentages in parentheses for each postulate.
Format
General
 
Socket
 
Foot
Prescriptive
Activities that include many rotational movements are indication for prescription of rotator (96%).
 
Wearing gel liner cushions shear forces between socket and skin
of distal residual limb (90%).For extremely short transtibial residual limb, gel liner should be prescribed (83%).
If amputee suffers from sensibility disorders of residual limb, gel liner should be prescribed (81%).
Limited weight-bearing residual limb is indication for transtibial prosthesis with suspension above knee (95%).
 
Walking on even ground is indication for prescription of multiflexible foot (100%).
Early foot flat during stance phase of prosthetic leg provides early stance phase stability, which is important parameter in prescribing prosthetic foot (90%).
Energy-storing foot should be prescribed for highly active transtibial amputees (84%).
           
Additive
None developed.
 
Excessive perspiration of residual limb is not contraindication for prescription of gel liner (81%).
For transtibial residual limb without specific residual limb problems, transtibial prosthesis with a suspension above knee is not common daily practice prescription (79%).
Gel liner is not indication for improving total contact between socket and residual limb (100%).
If donning gel liner requires assistance, it is not contraindication for prescription of gel liner (100%).
 
When walking at high speed, prosthetic foot should have wide range of dorsiflexion (84%).

Table 4.
Prescription format for prostheses for transgenual amputees with expert agreement percentages in parentheses for each postulate. Common daily practice prosthesis prescription for amputee with average activity level, normal residual limb length, and no specific residual limb and skin problems.
Format
General
Socket
Knee
Foot
Prescriptive
Activities that include many rotational movements are indication for prescription of rotator (96%).
Common daily practice for prescription of transgenual prostheses includes hard socket combined with poly-form inner socket (76%).
Wearing gel liner cushions shear forces between socket and skin of distal residual limb (90%).
If amputee suffers from sensibility disorders of residual limb, gel liner should be prescribed (81%).
Common daily practice for prescription of transgenual prostheses includes 4-axis knee unit (84%).
Walking on uneven ground is indication for prescription of multiflexible foot (100%).
         
Additive
None developed.
Reduced femur condyle contours can be indication for gel liner to improve suspension (79%).
Excessive perspiration
of residual limb is not contraindication for prescription of gel liner (81%).
Prescription of open socket vs closed socket could improve comfort during sitting (90%).
If problems exist passing femur condyles open socket can be prescribed (77%).
Gel liner is not indication for improving total contact between socket and residual limb (100%).
If donning gel liner requires assistance, it is not contra-indication for prescription of gel liner (100%).
7-axis knee unit provides more stability during stance phase than 4-axis knee unit (89%).
Early foot flat during stance phase of prosthetic leg provides early stance phase stability, which is important parameter in prescribing prosthetic foot (90%).
When walking at high speed, prosthetic foot should have wide range
of dorsiflexion (84%).

Table 5.
Prescription format for prostheses for transfemoral amputees with expert agreement percentages in parentheses for each postulate. Common daily practice prosthesis prescription for amputee with average activity level, normal residual limb length, and no specific residual limb and skin problems.
Format
General
Socket
Knee
Foot
Prescriptive
Activities that include many rotational movements are indication for prescription of rotator (96%).
Common daily practice for prescription of transfemoral socket includes combination of NML and quadrilateral principles for socket (84%).
If amputee suffers from sensibility disorders of residual limb, gel liner should be prescribed (81%).
If transfemoral prosthesis is only used for making transfers, nonsuction pelvic-belt suspension is preferred (82%).
Electronic knee units indicated for patients with high demand for stability control (94%).
Knee-lock mechanism is only prescribed if insufficient stability control during stance phase exists (84%).
If balance training does not improve poor balance, control knee-lock mechanism should be prescribed (81%).
Walking on even ground is indication for prescription of multiflexible foot (100%).
Energy-storing foot should be prescribed for highly active transfemoral amputees (90%).
If knee-lock mechanism is prescribed, combination with single- or multiple-axis foot is indicated (81%).
         
Additive
Improving comfort during sitting is indication for lotus-adaptor (87%).
Weight of prosthesis is not essential criterion in prosthesis prescription for young transfemoral amputees (84%).
To increase stability over hip joint of transfemoral amputees, an RPB should be prescribed (89%).
Wearing gel liner cushions shear forces between socket and skin of distal residual limb (90%).
Excessive perspiration of residual limb is not contraindication for prescription of gel liner (81%).
Gel liner is not indicated for improving total contact between socket and residual limb (100%).
If donning gel liner requires assistance from others, it is not contraindication for prescription of gel liner (100%).
If suction socket is not sufficient, elastic pelvic bandage should be prescribed to improve suspension (79%).
If of insufficient vascularization of upper leg, nonsuction, pelvic-belt suspension socket should be prescribed (75%).
If amputee has lower activity level, NML socket should not be prescribed (79%).
7-axis knee unit provides more stability during stance phase than 4-axis knee unit (89%).
Single-axis knee unit is not primary prescription for transfemoral amputees with low activity level (89%).
5-axis or 7-axis knee unit is not primary prescription for transfemoral amputees with high activity level (100%).
Early foot flat during stance phase of prosthetic leg provides early stance phase stability, which is important parameter in prescribing prosthetic foot (90%).
When walking at high speed, prosthetic foot should have wide range
of dorsiflexion (84%).
NML = narrow medial lateral, RPB = rigid pelvic band.
Advantages and Disadvantages of the Method

The Delphi Technique, as a tool, has reached a stage of maturity. It is now used fairly extensively in organizational settings in either the paper and pencil mode or in combination with face-to-face meetings and nominal group techniques. Compared with other consensus methods, the computerized Delphi Technique has several advantages, for example-

Participants react anonymously, which means a decrease in mutual influence.
Participants contribute to the group communication process when they feel they want or are able to.
Communication through the Internet removes geographical obstacles and takes less time.
Information sent to participants by the Internet is quick, which makes more participants join the process; this has a positive effect on the results achieved [17,19].

Our aim was to develop guidelines based on objective evidence. In our opinion, the modified Delphi Technique can make the knowledge from clinical experience more explicit (by converting it into postulates) than would be possible in conventional standards and guideline development practices. A disadvantage is the chance of limiting the knowledge input. However, the technique creates less chance of biased information, fractions of information, or controversies. An essential point in this procedure is the choice for creating a scientific base that can be extended in the next steps of the guideline development process. In these steps, more clinical information will be added when it actually has significant value.

Choice of Participants

In a consensus procedure, the choice of the participants is crucial. In the process of selecting the participants, our aim was to achieve a broad representation of all different points of view about prescribing prostheses for lower-limb amputees from three different groups of experts. However, the chance exists that subjectivity could interfere. Psychosocial interactions within the group could have been present. The process design, however, minimizes this aspect because most parts were anonymous and the nominal group meeting was managed by skilled and objective professionals. The participation of all disciplines involved in clinical practice also provides a solid foundation for implementation of the resultant guidelines.

The number of participants in this procedure can be debated. In alternative conventional methods, larger groups of clinical experts are assembled, which may account for more knowledge and postulates than derived with a Delphi Technique. The choice for this procedure, however, was based on time, related aspects, and an efficient manner of achieving consensus. The future steps in the procedure still give more experts the opportunity to participate in and make changes in the guidelines.

Another point of discussion is the absence in this part of the procedure of patients and insurers. The Delphi Technique does not rule them out as participants; however, an important aim was to develop first-draft guidelines based on scientific and explicit clinical knowledge. It might be necessary to invite patients to this first phase of a modified Delphi Technique when it is applied in other countries.

We recognize the importance of consumer opinions on the prescription process and the differences from those of clinical professionals. Therefore, consumers will participate in the subsequent steps in the guideline procedure.1

Decisions of Project Team

In the Delphi Technique, the project team has to decide the procedural steps. Their decisions can vary from fully autocratic to fully democratic ones. Because of the expected fundamental differences, we assumed that too directive a role by the project team would be ineffective. Therefore, the project team had only an editorial role. The initial postulates derived from the literature search and the observational study were edited to a manageable form for presentation in the postal rounds. No selection was performed based on the content of the postulates by the project team. Additionally, we decided to allow all Delphi-1 postulates with minor agreement (60%-75% consensus) a second chance. The Delphi-2 data showed much more agreement, and we belived that a consensus could be achieved. After the consensus development meeting, the participants seemed satisfied with the resulting guideline format.

At the consensus meeting, 59 percent of the participants were present, whereas in the postal rounds, all participants took part. However, in our opinion, the three key disciplines were sufficiently represented (63% physicians, 26% prosthetists, and 11% physiotherapists) during the meeting, and a 70 percent consensus on the postulates was reached during the postal rounds. For none of the postulates did substantial differences in agreement percentage exist among the three disciplines.

The success or failure of the consensus meeting largely depends on the experience and leadership skills of the chairman. In this, the method does not differ from a conventional face-to-face development meeting.

Scientific evidence obtained from a systematic literature review consisted of information about the functional aspects of prosthetic feet, knee mechanisms, sockets, and prosthesis weight. Specific prescription criteria could not be gained from the literature. Therefore, one limitation of this process is the lack of explicit information available on prescription criteria. Guideline development based only on the limited information available reduces the chances that studies are included that are not randomized controlled trials. However, randomized controlled clinical trials are not the only way to scientifically prove a hypothesis. Limiting recommendations to where evidence exists would reduce the scope of guidelines and limit their value to clinicians and policy makers who need to make decisions in the presence of imperfect knowledge [20]. Guidelines based on a mixture of evidence from literature and consensus opinion are of greater scope and applicability.

The consensus-based guideline development process presented here has created the opportunity for collaboration of the three disciplines active in prosthesis prescription in the Netherlands. This collaboration is important for the clinicians who are going to use the guidelines and will improve the implementation process. It also gives clinicians the opportunity to control the effectiveness of the guidelines that are developed and add adjustments.

The content of the prescription format will now be reviewed by clinical professionals and experts on guideline development who were not involved in the consensus development process. Representatives of patient groups will be involved in this review process also. The resulting guidelines will then be evaluated nationally in clinical practice. Furthermore, we need to incorporate assessment of functional capability of amputees because it forms the basis of the prescription format.

The method presented has been useful for the specific situation in the Netherlands. However, we state that it could also be useful in other industrialized countries, after adjustments based on differences in regulations or disciplines involved in the prescription process. It could also form a basis for an international discussion on prescription criteria.

Conclusion

The participants in the Delphi Technique achieved consensus about 37 postulates on prosthesis prescription for lower-limb amputees. This resulted in a set of draft clinical guidelines for prosthesis prescription. The adoption of this core set by the participants may be the first step toward a minimum reference standard of quality measures for clinical practice. Our intention is not to replace existing individual clinical expertise, but we suggest that these postulates should be used alongside the views of clinicians. The scope of content and applicability of the guidelines that were developed will have to be measured and evaluated in the near distant future.

References
1. Field MJ, Lohr KN. Guidelines for clinical practice: from development to use. Washington (DC): National Academies Press; 1992.
2. Fowkes FG, Roberts CJ. Introducing guidelines into clinical practice. Eff Health Care. 1984;1(6):313-23.
3. Woolf SH, Grol R, Hutchinson A, Eccles M, Grimshaw J. Potential benefits, limitations, and harms of clinical guidelines. BMJ. 1999;318(7182):527-30.
4. National Health and Medical Research Council. A guide to the development, implementation and evaluation of clinical practice guidelines. Health Technol Assess. 1998;2(3):1-73.
5. Goh JC, Solomonidis SE, Spence WD, Paul JP. Biomechanical evaluation of SACH and uniaxial feet. Prosthet Orthot Int. 1984;8(3):147-54.
6. Menard MR, McBride ME, Sanderson DJ, Murray DD. Comparative biomechanical analysis of energy-storing prosthetic feet. Arch Phys Med Rehabil. 1992;73(5):451-58.
7. Trickey H, Harvey I, Wilcock G, Sharp D. Formal consensus and consultation: a qualitative method for development of a guideline for dementia. Qual Health Care. 1998;7(4):192-99.
8. Shekelle PG, Morton SC, Clark KA, Pathak M, Vickrey BG. Systematic review of risk factors for urinary tract infection in adults with spinal cord dysfunction. J Spinal Cord Med. 1999;22(4):258-72.
9. Van der Linde H, Hofstad CJ, Guerts ACH, Postema K, Geertzen JHB, Van Limbeek J. A systematic literature review of the effects of different prosthetic components on human functioning with a lower limb prosthesis. J Rehabil Res Dev. 2004;41(4):555-70.
10. Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs. 2000;32(4): 1008-15.
11. Turoff M, Hiltz SR. Computer based Delphi processes. In: Adler M, Ziglio E, editors. Gazing into the oracle: the Delphi method and its application to social policy and public health. London (England): Kingsley Publishers; 1995. p. 56-88.
12. Linstone H, Turoff M. The Delphi method: techniques and applications. Boston (MA): Addison Wesley; 1975.
13. Cochrane Reviewers' Handbook 4.2.2. In: Alderson P, Green S, Higgins JPT, editors. The Cochrane Library, issue 1. Chichester (England): John Wiley & Sons, Ltd; 2003.
14. Van der Linde H, Geertzen JH, Hofstad CJ, Van Limbeek J, Postema K. Prosthetic prescription in the Netherlands: an observational study. Prosthet Orthot Int. 2003;27(3):170-78.
15. Van der Linde H, Geertzen JH, Hofstad CJ, Van Limbeek J, Postema K. Prosthetic prescription in the Netherlands: an interview with clinical experts. Prosthet Orthot Int. 2004; 28(2):98-104.
16. Bernstein SJ, Hofer TP, Meijler AP, Rigter H. Setting standards for effectiveness: a comparison of expert panels and decision analysis. Int J Qual Health Care. 1997;9(4):255-64.
17. Murphy MK, Black NA, Lamping DL, McKee CM, Sanderson CF, Askham J, Marteau T. Consensus development methods, and their use in clinical guideline development. Health Technol Assess. 1998;2(3):i-iv, 1-88.
18. Grol R. Personal paper: Beliefs and evidence in changing clinical practice. BMJ. 1997;315(7105):418-21.
19. Verhagen AP, De Vet HC, De Bie RA, Kessels AG, Boers M, Bouter LM, Knipschild PG. The Delphi list: a criteria list for quality assessment of randomized clinical trials for conducting systematic reviews developed by Delphi consensus. J Clin Epidemiol. 1998;51(12):1235-41.
20. Rycroft-Malone J. Formal consensus: the development of a national clinical guideline. Qual Health Care. 2001;10(4): 238-44.
Submitted for publication November 24, 2003. Accepted in revised form May 31, 2005.
1Van der Linde H, Hofstad CJ, Geertzen JHB, Postema K, Van Limbeek J. From satisfaction to expectation: measurement of patients' experience with everyday practice. Unpublished observations; 2004.

Go to TOP  

Go to the Contents of Vol. 42 No. 5

Last Reviewed or Updated  Wednesday, March 22, 2006 11:50 AM