Volume 47 Number 1, 2010
Pages 73 — 82
Abstract — The goal of this study was to investigate the use of custom-made orthopedic shoes (OS) and the association between the use of OS and the most relevant aspects of their usability. Over a 6-month period, patients meeting the inclusion criteria were recruited by 12 orthopedic shoe companies scattered throughout the Netherlands and asked to complete a questionnaire composed of a pre- and post-OS section. Patients with different pathologies were included in the study (n = 339; response 67%). Mean age of the patients was 63 +/- 15 years, and 38% were male. Three months after delivery, 81% of the patients used their OS frequently (4-7 days/week), 13% occasionally (1-3 days/week), and 6% did not use their OS. Associations were found between use and all measured aspects of usability (p-values varied from <0.001 to 0.028). Patients who used their OS more often had a more positive opinion regarding all the aspects of usability. We conclude that all aspects of the usability of OS are relevant in relation to their use and should be taken into account when prescribing and evaluating OS.
Key words: diabetes mellitus, foot disorders, medical devices, orthotics, patient satisfaction, rehabilitation, rheumatoid arthritis, shoes, usability, use.
Custom-made orthopedic shoes (OS) are used for a large variety of serious foot and/or ankle problems: for example, to prevent recurrence of foot ulcers, reduce plantar pressure, diminish pain in the feet or ankles during standing and walking, support foot deformities, enhance mobility, or provide stability [1-3]. OS are prescribed for patients with a wide range of disorders, such as diabetes, rheumatoid arthritis, degenerative foot disorders, spasticity, and muscular diseases . These patients do not fit standard off-the-shelf footwear. OS, including the insole, are completely individually designed and fabricated from a positive model cast from the patient's foot.
In order to be effective, OS must be used by those for whom they are prescribed. However, it has been frequently reported that patients do not use their OS [4-13]. Varying rates of nonuse have been reported, ranging from 20 to 25 percent for first-time users [5,10] and from 4 to 19 percent for experienced users [4,6-9,11-13]. Generalization or comparison of the results of these studies is almost impossible for three general reasons. First, most studies have been performed in one specific group of patients, which is different from the diversity seen in clinical practice. Second, follow-up times vary from 3 months to 2 years after supply of the first pair of OS. Third, and most important, use of OS has been defined in many different ways but mainly as in the following three main categories: frequent, occasional, and nonuse. These categories have been analyzed using two main combinations (frequent + occasional use vs nonuse [8,10,13] or frequent use vs occasional + nonuse [5,7,9]), without any definitive rationale for the specific combination. Other studies define use in categories that are not clearly described and sometimes not mutually exclusive [4,11-12].
The use of OS has been associated with several aspects of their usability. Usability is "the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction, in a specified context of use" (International Organization for Standardization [ISO], 9241-11). Within the domains of usability stated, a more positive score on any of the following aspects has been associated with a higher frequency of use of OS: benefits of OS with regard to walking capacities, wound healing, or pain reduction (domain effectiveness); comfort and ease of use and the efficiency of the delivery process of OS (domain efficiency); and cosmetic appearance and communication with the medical specialist or orthopedic shoe technician (domain satisfaction) [4-21]. However, only one study has been conducted in which the associations between use and aspects of all domains of usability have been studied and that was conducted only with patients with degenerative disorders of the foot . The focus of other studies has been on some aspects of one or two domains of usability and often in one specific group of patients only [4-9,11-21].
The aim of the present study was to investigate the use of OS in a group of patients that reflects the diversity seen in clinical practice and to investigate the associations between the use of OS and the most relevant aspects of their usability.
A prospective cohort study with internal comparison was conducted. Over a 6-month period, patients meeting the inclusion criteria were recruited by 12 orthopedic shoe companies scattered throughout the Netherlands. A specially developed questionnaire comprising pre- and post-OS sections was used. After patients had given written informed consent, during the visit at which foot measurements were taken, the orthopedic shoe technician gave the pre-OS section to the patients. These patients' personal data were sent to the researchers. The pre-OS section had to be completed and returned to the researchers before actual delivery of the OS. Three months after delivery of the OS, the researchers sent the post-OS section of the questionnaire to all patients who had previously completed the pre-OS section. However, in the current article, only analysis of the data acquired from the post-OS section of the questionnaire is presented.
Patients who did not complete either the pre- or post-OS section of the questionnaire within a month were contacted once by telephone by the researchers, who asked about the reason for the delay and possible problems and again asked the patients to complete the questionnaire.
A series of patients who were provided with their first-ever pair of OS by 12 Dutch orthopedic shoe companies was included. Patients who were previous users of OS were excluded because large differences exist between first-time and experienced users . Other inclusion criteria were (1) 16 years of age or older, (2) able to read Dutch, and (3) able to complete the questionnaire without help related to cognitive or physical impairments.
The patient flow chart is shown in the Figure. Mean age of the 339 patients included in the study was 63 ± 15 years, and 38 percent were male. Of these patients, 85 had diabetes mellitus, 60 rheumatoid arthritis, 237 a foot disorder, 23 a muscular disease, and 104 another disorder (e.g., cerebral vascular accident, spinal cord injury, psoriasis, leather allergy, and others). Disorders were indicated by patient self-report, and it was possible to indicate more than one disorder. The age and sex of the patients included in the study were comparable with the nonresponders (patients who gave written informed consent but did not complete both sections of the questionnaire; mean age 59 ± 17 years and 41% male) and with all patients of the 12 orthopedic shoe companies to whom a first-ever pair of OS was provided between January and June 2007 (data obtained via administration system of the orthopedic shoe companies; mean age 63 ± 17 years and 39% male).
We were able to contact 34 of the 51 patients who did not respond to the post-OS section of the questionnaire. Reasons indicated for not responding were lack of interest (n = 15), lack of time (n = 10), not using OS and dissatisfied (n = 3), not using OS because of change in medical situation (n = 2), questionnaire missing in post (n = 2), and no reason specified (n = 2).
We used the Monitor Orthopedic Shoes (MOS), which is a practical and reproducible questionnaire that can be used for a wide range of patients . The MOS consists of a pre- and post-OS section. For the purposes of this study, we used only the post-OS section, which was designed to measure use and the most relevant aspects of usability of OS from a patient's perspective through multiple choice and visual analog scale questions .
We defined three categories of use of OS: frequent use (4-7 days/week), occasional use (1-3 days/week), and nonuse (not using OS). We further asked patients to indicate the average daily duration of use in hours. Use of OS was not specified into activities during which OS were used or location of use (e.g., indoor vs outdoor).
Within the domains of usability as defined by the ISO, the following aspects were measured: change in walking capacity, wound healing, change in pain, and change in sprains (domain effectiveness); donning and doffing OS, fit of OS, ease of walking with OS, weight of OS (domain efficiency); and cosmetics and communication with medical specialist and orthopedic shoe technician (domain satisfaction).
We assessed differences between the three groups with a Kruskal-Wallis test. We used this test because of the nonnormal distribution of the data. We performed post hoc analyses to assess differences between the groups separately. We used a chi-square test for categorical variables and calculated the effect size with Cramér's V. We used a Mann-Whitney U-test for scale measures with a nonnormal distribution of the data and calculated the effect size with the formula (effect size = Z / (n1 + n2) ). Alpha level was set at p < 0.05. We analyzed data using SPSS for Windows, version 16.0 (SPSS Inc; Chicago, Illinois).
Three months after delivery of OS, 81 percent of the patients used OS frequently (4-7 days/week), 13 percent used OS occasionally (1-3 days/week), and 6 percent did not use OS. No significant differences were found between the patient characteristics of the three groups (Table 1). Patients who use their OS frequently have a significantly higher daily duration of use than patients who use them occasionally (Table 1).
Table 1.Patient characteristics, categorized with regard to frequency of use of their custom-made orthopedic shoes (OS).
Variable Frequent Use
(81%, n = 275)
(13%, n = 43)
(6%, n = 21)
Sex Male 40 (110) 37 (16) 14 (3) 0.061 Female 60 (165) 63 (27) 86 (18) Age (yr), Mean ± SD 63 ± 14 63 ± 17 63 ± 12 0.998 Main Reason† Pain 54 (147) 70 (30) 57 (12) NA‡ Wounds 9 (25) 2 (1) 0 (0) Foot Deviation 23 (62) 12 (5) 19 (4) Leg Length Difference 3 (7) 0 (0) 10 (2) Other 12 (34) 16 (7) 14 (3) General Health Improved 19 (51) 7 (3) 29 (6) NA‡ No Change 71 (194) 79 (34) 52 (11) Deteriorating 10 (28) 12 (5) 14 (3) Missing 1 (2) 2 (1) 5 (1) Daily Duration of Use (hours/day) >12 24 (65) 5 (2) NA <0.001¶ 8-12 34 (93) 2 (1) NA 4-8 30 (83) 33 (14) NA 1-4 11 (29) 47 (20) NA <1 1 (1) 14 (6) NA Missing 1 (4) 0 (0) NA
Note: Values are % (n) or otherwise as indicated. Frequent Use = using OS 4-7 days/week, Occasional Use = using OS 1-3 days/week, Nonuse = not using OS. Percentages may not add to 100 because of rounding.*p-Value for differences between three groups on that aspect is shown.†Main reason was indicated by patient self-report.‡Chi-square test was not applicable because more than 25% of cells had expected count <5.§General health refers to change in general health (not feet specifically) between pre- and postmeasurements.¶p-Value for difference between frequent and occasional users only is shown: this value was not applicable (NA) for nonusers.SD = standard deviation.
The associations between the use of OS and patients' opinions with regard to aspects of the usability of OS are shown in Table 2. Significant differences between the three groups were found for all aspects of usability of OS, with a more positive score for patients who use their OS more often. The observations also indicate that more improvement was noted with regard to walking capacity in patients who frequently use their OS and that no differences were found with regard to wound healing, although a p-value could not be calculated for these aspects (Table 2). Considerable variation was present within the groups (Table 2).
Table 2.Association between use of custom-made orthopedic shoes (OS) and aspects of their usability within domains effectiveness, efficiency, and satisfaction.
Variable Frequent Use
(81%, n = 275)
(13%, n = 43)
(6%, n = 21)
Effectiveness Walking Capacity Improved, because of OS 76 (208) 33 (14) 0 (0) NA† Improved, not because of OS 1 (3) 5 (2) 33 (7) No change 14 (39) 47 (20) 33 (7) Deteriorated, not because of OS 7 (18) 9 (4) 0 (0) Deteriorated, because of OS 2 (4) 2 (1) 19 (4) Missing 1 (3) 5 (2) 14 (3) Wounds After OS‡ Patients with wounds before OS (n = 49, 10, 0) More 4 (2) 10 (1) - NA† Same 27 (13) 20 (2) - Less 22 (11) 20 (2) - Healed 47 (23) 50 (5) - Patients with no wounds before OS (n = 226, 33, 21) Wounds 6 (13) 6 (2) 0 (0) NA† No wounds 94 (212) 94 (31) 71 (15) Missing 0.5 (1) 0 (0) 29 (6) Change in Pain (skin)§ (n = 166, 25, 8)‡ 82 (58.5-95) 66 (28.5-89.5) 22.5 (7.5-46) <0.001 Change in Pain (muscles) § (n = 212, 33, 9)‡ 79.5 (60-92) 58 (35-79.5) 20 (8-48) <0.001 Change in Sprains§ (n = 147, 23, 4)‡ 92 (75-97) 83 (50-96) 45.5 (31.25-78.75) 0.013 Efficiency Donning/Doffing OS§ 75 (49.5-89) 63 (27.5-79.5) 51.5 (15.75-94.5) 0.017 Fit of OS§ 84 (68-95) 74 (38-89) 64 (36-89.5) 0.002 Ease of walking with OS§ 80 (62.25-92) 52 (31-81.5) 5 (4-17.5) <0.001 Weight of OS§ 48 (27-52) 26.5 (8.5-49.25) 17 (6.75-36.25) <0.001 Satisfaction Cosmetics: patient's opinion§ 58 (46-80) 49 (17-87) 16 (2.5-50.25) <0.001 Cosmetics: opinion of others Ugly or very ugly 8 (22) 19 (8) 38 (8) <0.001 Neutral 37 (102) 47 (20) 19 (4) Attractive or very attractive 40 (110) 19 (8) 10 (2) Do not know or missing 15 (41) 16 (7) 33 (7) Communication with doctor§ 85 (67.25-94) 79 (50-88) 53.5 (17-97.25) 0.028 Communication with ST§ 87 (64-94.75) 78 (41-90) 64 (15.5-96.25) 0.001
Note: Frequent Use = using OS 4-7 days/week, Occasional Use = using OS 1-3 days/week, Nonuse = not using OS. Values are % (n) or median (interquartile range). Percentages may not add to 100 because of rounding.*p-Value for differences between three groups on that aspect is shown.†Chi-square test was not applicable because more than 25% of cells had expected count <5.‡Not all patients had wounds, pain, or sprains; therefore, number of patients for these questions is indicated for each use group, respectively.§Scores could range from 0 (most negative score possible) to 100 (most positive score possible).NA = not applicable, ST = orthopedic shoe technician.
Results of the post hoc analyses are shown in Table 3. Change in pain (both in the skin and in the muscles) and ease of walking with OS differed significantly between all groups. Both aspects of cosmetics (patient's opinion and opinion of others) differed significantly between patients who frequently use their OS and both other groups and approached significance between patients who occasionally use and those who do not use their OS. The largest effect sizes were found for ease of walking with OS and change in pain in the muscles. Finally, a significant effect for gender was found: compared with patients who frequently use their OS, significantly more female patients did not use their OS. Because of this gender effect, two more post hoc analyses were performed. First, the associations between the use of OS and aspects of their usability were also calculated for the three groups of patients matched for gender. The results were the same: significant differences were found between the three groups for all aspects of usability (results not shown). Second, gender differences were calculated for all aspects of usability within the group of patients who frequently use their OS. No significant differences were found between the opinion of men and women on any of the aspects of usability (results not shown).
Table 3.Post hoc analyses on significant main effects of frequency of use of custom-made orthopedic shoes (OS) in domains of usability.
Variable Frequent Use × Nonuse Frequent Use × Occasional Use Occasional Use × Nonuse p-Value* ES p-Value* ES p-Value* ES Effectiveness Change in pain (skin) <0.001 0.31 0.027 0.16 0.010 0.45 Change in pain (muscles) <0.001 0.26 <0.001 0.23 0.013 0.38 Change in sprains 0.013 0.20 0.081 0.13 0.109 0.31 Efficiency Donning/doffing OS 0.145 0.09 0.010 0.15 0.974 0.00 Fit of OS 0.035 0.12 0.003 0.17 0.813 0.03 Ease of walking with OS <0.001 0.36 <0.001 0.22 <0.001 0.59 Weight of OS 0.001 0.22 <0.001 0.20 0.313 0.13 Satisfaction Cosmetics: patient's opinion <0.001 0.22 0.003 0.17 0.079 0.23 Cosmetics: opinion of others <0.001 0.34 0.004 0.20 0.066 0.33 Communication with doctors 0.094 0.11 0.025 0.13 0.464 0.10 Communication with ST 0.030 0.13 0.001 0.18 0.613 0.06 Patient Characteristics: Sex† 0.019 0.14 0.728 0.02 0.059 0.24
Note: Frequent Use = using OS 4-7 days/week, Occasional Use = using OS 1-3 days/week, Nonuse = not using OS. Nonsignificant results for patient characteristics "age," "main reason," and "general health" not shown.*p-Value for difference between groups is shown; significant difference indicates that patients who use their OS more often have a significantly more positive opinion with regard to that aspect.†Significant p-value indicates that men use their OS more frequently than women.ES = effect size, ST = orthopedic shoe technician.
In the present study, we found that 3 months after delivery, 81 percent of the patients used their OS frequently, 13 percent occasionally, and 6 percent did not use their OS. All measured aspects of usability were associated with the use of OS; patients who use their OS more have a more positive opinion of the aspects of their usability. Comparison of these results with previous studies is almost impossible because in other studies patients with different pathologies were investigated, follow-up measurements were taken at different time points, and use was defined in various ways. In general, the rather small amount of nonuse found in this study supports the conclusion that nonuse of OS is not a major issue .
Even though nonuse of OS does not seem to be a major issue, gaining insight into factors associated with the frequency of use (and nonuse) is still relevant because of the high costs and efforts for the patient, the orthopedic shoe technician, and the medical specialist involved in every pair of OS. We measured aspects of usability within the domains effectiveness, efficiency, and satisfaction. All these aspects of usability were positively associated with use. This result indicates that when prescribing and evaluating OS, prescribers should not focus on the effectiveness of OS alone: aspects of efficiency and satisfaction are as important as effectiveness in relation to use of OS, which is in-line with previous research .
The considerable variation in patients' opinions of aspects of usability within the groups indicates that large individual differences exist. Some patients have extremely negative opinions about some aspects of the usability of their OS, yet still use them frequently. Others do not use their OS, despite a very positive opinion about some aspects of the usability. This finding may be explained by the importance that patients attach to specific aspects, a factor we did not take into account in our measurements. For example, for a patient with very severe pain, reduction of that pain may outweigh a negative opinion of the cosmetics of the OS: the patient will use the OS every day. Whereas for another patient, the same pain reduction may still not outweigh a negative opinion of the cosmetics: the patient will not use the OS. Future research should take the importance that patients attach to specific aspects of OS usability into account. Clinically, this finding implies that it is essential to take all aspects of usability into account for every patient and find out which aspects (if any) are more important than others. The importance of structured evaluation in every individual has been stressed in recent Dutch research as well .
The present study has some limitations. First, we may have underestimated the rate of nonuse of OS. However, characteristics of patients included in the study were comparable with all patients who were provided with a pair of OS in that period and with nonresponders. Second, it could be hypothesized that patients who do not use their OS and have negative opinions of the usability of their OS are less willing to respond. To minimize this, we administered the pre-OS section of the questionnaire in an early stage of the process, approximately 3 months before delivery of OS. Not responding in that phase was not likely related to dissatisfaction with use and aspects of the usability of OS. Not responding to the post-OS section of the questionnaire was more likely to be related to dissatisfaction with the use and usability of OS. We therefore called these patients and were able to contact more than half of them. Only a few did not use their OS and even fewer did not respond because of dissatisfaction; all other patients did use their OS and had other reasons for not responding. Therefore, an underestimation of the rate of nonuse will, in our opinion, only be a small one.
A third limitation is that we could not perform multivariate analyses. There were two reasons for that. First, the MOS has two types of answer categories, which was a deliberate choice in the development of the questionnaire, during which emphasis was put on patients' preference . Second, it was not possible to perform multivariate analyses on those parts of the aspects of usability with similar answer categories because of the (unexpectedly) large differences in the number of patients in the three groups: more than 80 percent of the patients were found in one group. Because of this imbalance, it was not possible to investigate the importance of the aspects of usability relative to each other.
A general limitation in the field of research regarding use and usability of OS are the three differences between all studies with regard to (1) patients, (2) time period of follow-up, and (3) definition of use, which make comparison of results almost impossible:
We conclude that the more positive opinion on all measured aspects of the usability of OS of patients who use their OS more often indicates that it is essential to take all aspects of the usability of OS into account when prescribing and evaluating OS. The effectiveness (e.g., a change in pain after OS), the efficiency (e.g., the ease of walking with OS), and the satisfaction (e.g., the patient's opinion of the cosmetics of OS) are all relevant in relation to the use of OS.
Go to TOP
Last Reviewed or Updated Thursday, March 11, 2010 11:24 AM