Volume 48 Number 7, 2011
Pages 763 — 774
Abstract — This article presents an assessment technique for testing the quality of prosthetic socket fabrication processes at computer-aided manufacturing facilities. The assessment technique is potentially useful to both facilities making sockets and companies marketing manufacturing equipment seeking to assess and improve product quality. To execute the assessment technique, an evaluator fabricates a collection of test models and sockets using the manufacturing suite under evaluation, then measures their shapes using scanning equipment. Overall socket quality is assessed by comparing socket shapes with electronic file (e-file) shapes. To characterize carving performance, model shapes are compared with e-file shapes. To characterize forming performance, socket shapes are compared with model shapes. The mean radial error (MRE), which is the average difference in radii between the two compared shapes, provides insight into sizing quality. Interquartile range (IQR), the range of radial error for the best-matched half of the points on the compared socket surfaces, provides insight into regional shape quality. The source(s) of socket shape error may be pinpointed by separately determining MRE and IQR for carving and forming. The developed assessment technique may provide a useful tool to the prosthetics community and industry to help identify problems and limitations in computer-aided manufacturing and give insight into appropriate modifications to overcome them.
Key words: blister forming, CAD/CAM, carving, central fabrication, computer-aided manufactured sockets, prosthetics, shape quality, size quality, socket design, transtibial amputation, vacuum forming.
Computer-aided manufacturing (CAM) can potentially reduce the time and cost of making prostheses for people with limb amputation. Unlike traditional socket fabrication methods, CAM automates the mold-making and socket-forming processes. This automation reduces the amount of manual technical work, potentially saving time and money, and allows the practitioner to spend more time with patients. Additional benefits may include lower operating expenses and less required laboratory space, thus lower overhead, as well as lower equipment, supply, and maintenance costs.
Despite its potential benefits, CAM is estimated to be regularly used by only 24 percent of practitioners . Inconsistent or poor performance is believed to be part of the reason for low use. This is likely caused by some fabrication facilities not making the shape designed by the practitioner on the computer . A wide range of quality exists in the industry. Clinical evaluations have shown that the mean number of computer sockets required to properly fit subjects range from 1.5 to 5.0, depending on the investigation [3-8]. Part of the need for so many sockets may be poor manufacturing quality. If many fabrication runs are required to achieve a socket of acceptable fit, then the advantages of CAM are lost and traditional methods may as well be used.
Facilities implementing CAM to fabricate practitioner-designed model or socket shapes and manufacturers of fabrication equipment face an important challenge. No practical means exist for assessing socket quality or carving and forming performance, nor are there testing standards. If facilities could regularly assess the shape quality of their models and sockets with a widely accepted protocol, then they could identify and correct errors, resulting in higher-quality products delivered to the practitioner and patient. Documentation demonstrating good manufacturing practices could be disseminated, building clinician confidence in these facilities. This would likely result in the increased acceptance and use of computer-aided design as a clinic tool.
The purpose of this article is to present an assessment technique for CAM. The assessment technique is designed to help facilities test the quality of their models and sockets, identify fabrication problems and limitations, provide insight into needed equipment modifications, and document their performance.
Our assessment technique is to fabricate a collection of test models and sockets using the CAM equipment under evaluation. We measure the shapes of the fabricated models and sockets, then compare them with each other and with the electronic file (e-file) shapes from which they were fabricated (Figure 1). We conduct quantitative analyses to provide insight into the severity of error and the likely source(s).
Diagram summarizing assessment technique. In (a) size quality plot, carving error is mean radial error (MRE) of model compared with electronic file (e-file) shape; forming error is MRE of socket compared with model. In (b) regional shape quality plot, vertical axis is number of points on sur-face. In (c) shape difference plot, radial error (mm) between socket and model is illustrated. IQR = interquartile range.
Click Image to Enlarge. View as PowerPoint Slide
We select the test shapes to represent a range of socket shapes encountered in clinical practice in amputation prosthetics. We used three transtibial amputation socket shapes in the example presented here, though additional socket shapes or other types of prostheses could also be tested. The examples used here are from a commercial software package for prosthetic socket design (ShapeMaker, Sheck & Siress; Chicago, Illinois). The first model was a patellar tendon-bearing socket design for a person with a short and conical residual limb with a prominent fibular head. The second model was a total surface-bearing socket design for a person with a fleshy cylindrical residual limb with minimal bony prominences. The third model was of a patellar tendon-bearing socket design for a person with a bony and cylindrically shaped residual limb. Model lengths from the midpatellar tendon to the distal end for the three models were 12.8, 14.2, and 15.7 cm, respectively. We chose these models because they represent a range of shapes, contours, and lengths, which is important when evaluating carving and forming performance.
The facility's carving equipment creates a model for each of the socket shapes. We select the settings used (e.g., bit size, carving speed) based on the analysis of interest to the evaluator. For example, if a facility seeks to evaluate the quality of carves (models) and sockets that they send to customers regularly, then they should use their usual carver settings. If a facility is interested in evaluating the influence of carving speed on their product, then they should carve one set of test models at a slow speed and another set at a fast speed, then see how the quantitative analyses results differ.
We measure the shapes of the models using an accurate scanning instrument. The scanning instrument should not contact the model so that the model shape is not distorted during measurement. If a laser scanner is used, measurement error is lower if the plane of laser light is projected perpendicular to the model surface rather than at an acute or obtuse angle. The line that the laser light makes on the model surface is less distorted.
Different emerging technologies may satisfy the measurement need. We have found that a high-quality three-dimensional laser scanner (3D Scanner HD, NextEngine, Inc; Santa Monica, California) has acceptable performance for this application. The manufacturer states an accuracy of approximately 0.1 mm. The scanner uses multiple laser stripes to cross-validate the geometry data it collects, which helps minimize error in regions of complete shape. In the example presented here, we used a high-definition setting of 2,000 points per square inch, which took approximately 40 min for each model scan.
The facility's regular forming practices creates sockets over each of the three carved models. The socket material is selected based on the analysis of interest to the evaluator. If test socket fabrication is to be evaluated, then test socket materials should be used. If definitive socket fabrication is of interest, then definitive socket materials should be used. Comparing the influence of material selection on socket shape quality might also be of interest. In that case, one set of models should be made with one material and another set of models made with the other material, then differences in results compared.
We measure the shapes of the sockets using an accurate scanning instrument. Unlike model shape assessment, the scanning instrument must measure within a confined space, i.e., within the prosthetic socket. We are not aware of any currently available commercial products that are small enough and meet the required accuracy threshold. The contact scanner we developed had an accuracy of approximately 0.1 mm for a wide range of socket shapes tested and sampled at 800 points per slice  with a 0.8 mm slice spacing. These performance qualities are important, not necessarily because this level of accuracy is needed in socket manufacturing, but because current algorithms to properly align socket shapes with model or e-file shapes require such accuracy. In other words, poor socket shape measurement may introduce socket-to-model or socket-to-file alignment errors that substantially distort the results of interest in the evaluation. Potentially, new alternative alignment algorithms could be created to overcome this restriction, but we are unaware of any such technologies currently available.
To determine overall error, we compare the shapes of the sockets with the e-file shapes. Since the socket shape data and the e-file shape data are in different reference frames, they first need to be aligned to each other before comparisons can be made between them. We use an optimization procedure to do this. The optimization procedure minimizes a quantitative measure of clinical relevance. For example, commercially available software packages in the prosthetics industry typically implement an optimization procedure that minimizes the volume difference between the two shapes of interest. In other words, the shapes are moved relative to each other until the volume within them is as nearly equal as possible. This procedure performs well if the radii differences between the shapes are relatively even over the surface or if the anatomical landmarks are distinct and prominent in the shapes being aligned . It does not perform well if there are highly localized regions of shape difference and if there are not distinct and prominent anatomical landmarks in the shapes. In these cases, research has shown that an optimization procedure that minimizes volume difference while maximizing shape similarity performs better . Maximizing shape similarity means that regions on the two surfaces that are very similar in shape are given higher priority (weighting) than regions that are highly dissimilar in shape. Computationally, with the inverse hyperbolic tangent of the dot product of corresponding surface normals as an optimization variable, the highest weighting is given to surface normals that are close together, while the lowest weighting is given to surface normals that are far apart. Thus, if the fibular head region of the socket is shaped poorly and oversized but the socket is shaped correctly everywhere else, the algorithm would produce an image that is highlighted blue at the fibular head, indicating a need for reduction, with little to no color elsewhere on the surface.
In addition to comparing the socket shapes with the e-file shapes, the models need to be compared with the e-file shapes and the sockets compared with the models. These additional comparisons are necessary because they will identify how much carving and forming each contribute to fabrication error. We execute an algorithm similar to that described earlier for aligning sockets to e-file shapes. The algorithm minimizes volume difference and maximizes shape similarity .
Images of shape differences between the sockets and e-file shapes provide a quick visual assessment of overall quality (Figure 2). Locations of undersizing are red, and locations of oversizing are blue. A sparsely colored image with dense color away from sites of clinical relevance (e.g., Figure 2(a)) demonstrates a socket of high quality, while an image that is densely colored in weight-bearing areas (e.g., Figure 2(b)) indicates a socket of low quality. The low quality socket should be further evaluated to identify the source(s) of error.
Examples of overall socket quality. Images of shape differences between sockets and e-file shapes. (a) Socket with sparse color over surface with dense color away from sites of clinical relevance (edge of brim) is high quality, while (b) socket with dense color over surface is reduced quality. Blue = oversized, red = undersized.
Click Image to Enlarge. View as PowerPoint Slide
To quantify to what degree carving and forming are sources of error, we plot the mean radial error (MRE) for all socket-model (forming) and model-file (carving) shape pairs (Figure 3). MRE is the average difference in radii between two surfaces. A box around the three points, one point for each socket-model and model-file shape pair, indicates consistency of error. If the box is small, then both carving and forming are performed consistently. If the box is large then there is an inconsistency problem.
Size quality graph—interpreting shape of box. Shape of box indicates whether sizing problems exist in carving and forming. Carving error is mean radial error (MRE) of models relative to e-files, and forming error is MRE of sockets relative to models. (a) Carving performed well; forming not performed consistently. (b) Forming performed well; carving not performed consistently. (c) Carving and forming both performed consistently. However, models are oversized (~1.8 mm radius over surface), which results in oversizing of sockets. (d) Carving and forming performed consistently; carved models are accurate. However, sockets are oversized (~1.3 mm radius over surface) because of problem in forming. (e) Carving and forming not performed consistently. Further testing is needed to identify source(s) of error.
Click Image to Enlarge. View as PowerPoint Slide
Figure 3 illustrates how the size quality graph is interpreted. If the box is long, narrow, and close to the x-axis (Figure 3(a)), then carving is performed well but forming is not performed consistently. An inconsistent shrinkage problem using the polymer possibly exists. If the box is tall and thin (Figure 3(b)), then forming is performed well but carving is not performed consistently. The facility is possibly changing the bit from one carve to the next, and the bit is not inserted to a consistent depth each time in its holder. If the box is small but far from the x-axis (Figure 3(c)), then carving and forming are both performed consistently. However, the models are oversized (~1.8 mm radius over the surface in this example), and that results in oversizing of the sockets. There is possibly a calibration offset error in the carver. If the box is small but far from the y-axis (Figure 3(d)), then carving and forming are both performed consistently. The carved models are accurate; however, the sockets are oversized (~1.3 mm radius over the surface in this example). A thick sock is possibly being put over the models before forming. If the box is large and away from both axes (Figure 3(e)), then carving and forming are not performed consistently. Further testing is needed to identify the source(s) of error.
While presentation of the size quality data provides insight into sizing errors and the degree of inconsistency, we need a different chart to characterize regional shape quality. Regional shape quality pertains to how well the manufactured shape matches that designed by the practitioner. Figure 4 shows a histogram that illustrates regional shape error. A histogram shows the distribution of radial error. If the histogram is tall and narrow (Figure 4(a)), then the regional shape error is low, which means that the radial error is relatively even over the surface. However, if the histogram is short and wide (Figure 4(b)), then the radial error is high at some locations and low at other locations. In other words, the shape is distorted. We may use the interquartile range (IQR) to characterize the degree of distortion. The IQR is the range of radial error for the best-matched half of points on the surface, which are the central 50 percent of the points about the mean in the histogram.
Example histograms. (a) Distribution of radial error is narrow, thus, little distortion exists. (b) Distribution of error is broad, indicating substantial distortion. IQR = interquartile range.
Click Image to Enlarge. View as PowerPoint Slide
While the histogram and IQR quantify distortion, they do not indicate where the distortions are located on the surface. Visual images of shape difference show locations of undersizing in red and locations of oversizing in blue (Figure 5). With these images, we can visually inspect whether shape errors are localized to certain locations, for example, at the fibular head and anterior tibial crest where there is much curvature (Figure 5(a)). Alternatively, the error might be more uniformly distributed over the surface, indicating uniform undersizing or oversizing (Figure 5(b)). For cases where shape errors are localized, quantifying and displaying the closeness of surface normals might facilitate presentation. This presentation would distinguish shape error from volume error. The three charts (size quality, regional shape quality, and shape difference) summarize fabrication performance from which likely sources of error can be deduced.
Example shape difference images. (a) This socket, compared with its model, shows undersizing (red) over tibial crest and fibular head. (b) This socket, compared with its model, is uniformly undersized except at medial and lateral tibial plateaus. Blue = oversizing.
Click Image to Enlarge. View as PowerPoint Slide
The example case presented here illustrates how this assessment technique is implemented (Figure 6(a)-(b)). The example is from a fabrication facility, labeled here as company XXX, that requested an analysis of its overall socket quality as well as their model- and socket-forming processes.
Example assessment report evaluating company XXX’s carving and forming procedures. (a) Socket shapes are compared with e-file shapes. (b) Carving (model shapes compared with e-file shapes) and forming (sockets shapes compared with model shapes) results are shown. IQR = interquartile range. Blue = oversized, red = undersized.
Click Image to Enlarge. View as PowerPoint Slide
Results from this analysis show that company XXX's sockets are consistently oversized compared with the e-file shapes from which they were fabricated. The overall quality images are all dark blue in color (Figure 6(a)). Further analysis helps identify the source(s) of oversizing.
Company XXX has very consistent carving and forming processes. The box in the size quality chart is very small (Figure 6(b), top graph), meaning that the range of MRE for both carving and forming is small. However, because the box is offset relative to the x-axis, the models are consistently oversized. An error in the company's carver is causing a 0.5 mm radial (1.0 mm diameter) oversizing of all models, which is equivalent to an approximately 1.7 percent volume error. This oversizing remains after forming, as seen in the size quality image (Figure 6(b), top graph); the green box is on the y-axis, meaning that errors during carving are propagated through forming. From the carving shape difference plot (Figure 6(b), bottom left graph), it is clear that the radial error is relatively uniform over the surface.
Thus, we suspect a calibration error during carving. The location of the tip of the bit may not be correctly programmed into the carver. Calibration error may be corrected by recalibrating the carver using the manufacturer's recommended calibration procedures.
Company XXX's socket forming practices (shape forming) are remarkably accurate. The tall, narrow histograms (Figure 6(b), center right graph) show that their sockets are almost exactly the same shapes as their models. The very light colors in the forming shape difference plot (Figure 6(b), bottom right graph) further support that their socket forming practices are exceptionally good.
Clearly shown in previous reports in the literature [2,10], variability exists in quality and consistency among different fabrication facilities that make prosthetic models and sockets for patients with amputation. It would therefore seem reasonable for facilities to assess their manufacturing equipment as part of their regular practice. Manufacturers of computer-aided socket fabrication equipment seeking to improve their products could facilitate these efforts by including a means for assessment as part of the technology they sell. Regular evaluation would quickly inform a fabrication facility whether something in its process needs correction. Such efforts might reduce downtime, enhance quality, and lower operating costs in the prosthetic socket fabrication industry. The ultimate goal is for practitioners to receive a socket shape well-matched to the shape created on the computer and for patients with amputation to consistently receive a high-quality product.
It is important to recognize two issues in fabrication quality: consistency and accuracy. Both can be evaluated through the assessment technique described here. However, the ease with which the test results are interpreted is strongly influenced by the presence of consistency. In the case demonstrated here, we easily identified the source of fabrication error because it was present in each socket-model pair made by the facility. Inconsistent errors are more difficult to interpret but, in general, point to a human technician issue. One example of a human technician issue would be differences in the amount of material removed during model sanding (most facilities sand their models by hand before forming). If assessment technology were implemented, fabrication facilities would be provided with the information necessary to find and correct error and improve their model and socket products. If many technicians are making models and sockets, then this assessment technique could help identify which technician's procedures are different from others.
Evaluation technology potentially provides a base for standards or certification of CAM facilities and equipment. One approach to certification could be to rate fabricators on a scale based on product consistency and accuracy, where 1 = highly consistent and highly accurate and 5 = highly inconsistent and highly inaccurate.
From our experience comparing socket manufacturing accuracy and practitioner-rated socket fit , we know that some patients are more tolerant of shape error than others. We expect a threshold error that most patients using prosthetic limbs and practitioners would deem acceptable. In other words, for most patients there is no need to expend time and funds to make sockets more accurate than this threshold level, because there is no difference in clinical fit or the patient's functional outcome. Prosthetics research has yet to identify this threshold, but in our opinion, it is only a matter of time before it is determined. While we expect a threshold that holds for most patients, we do not expect that it will hold for all patients. Select patients likely require a special, more accurate socket shape. An analogy to this concept is in the podiatry field: some patients require custom orthotic inserts for their shoes while many others do fine with off-the-shelf orthosis products. In prosthetics, some patients might require a special, more accurate socket shape while other patients might do fine with the normal quality shape. Presumably, the cost of fabricating a very accurately shaped socket is higher, particularly if the latest technology is used, i.e., additive fabrication techniques [12-16]. One possible means for managing these differences in patient adaptability is to distinguish the minimally adaptable patient by a special rating, for example a rating termed "P-1." A patient with a P-1 rating is highly unusual, does not easily adapt, and is intolerant to even small shape errors. An example might be a severely diseased patient with thin skin over bony prominences and minimal residual-limb soft tissue. A patient with a normal rating, who adapts to a normal shape quality socket, might be termed a P-2 patient. A facility with a rating 1 would be capable of fabricating sockets accurately for both patients. A facility with a rating 3 (range: 1-5, as described previously) could only manufacture acceptable sockets for the P-2 patients who adapt to shape errors. Selection of a facility with a rating 1 is thus imperative and justified for P-1 patients while both facilities are appropriate for P-2 patients.
An important challenge toward this assessment technique is that we do not currently know how much socket fabrication error is acceptable or what determines which group a patient should be assigned to. Data presented in the literature on the influence of patient volume change on fit provide a starting point for establishing socket and model volume accuracies and radial accuracies needed for CAM sockets. Fernie and Holliday state that a patient using a sock ply that is 10 percent or more of his or her limb volume should be fitted with a new smaller socket . When the sock ply is between 5 and 10 percent of the limb volume, the socket is "acceptable," and when the sock ply is between 0 and 5 percent of the limb volume, the socket is "good." In 1982, Fernie and Holliday developed these ranges based on their experience using a water displacement method in the clinic to measure limb volume change of patients at various stages postamputation. Perhaps the clinical community would benefit from a revised and updated set of standards created using the current methodology. Using the sockets in the present investigation, we found that 5 and 10 percent volume errors correspond to MREs of approximately 1.8 and 3.4 mm, respectively.
Studies on patients with amputation are needed to quantify the clinical effect of socket shape error on patient discomfort, performance, function, and duration of acceptable socket fit. For example, do CAM sockets with much MRE and IQR error need to be replaced sooner, thus actually increasing healthcare costs over other methods that do not have these errors? Obtaining meaningful results from clinical studies will be challenging because simultaneous confounding factors influence fit. The patient's health status, tissue tolerance, disease, and other characteristics might influence tolerance to socket shape error. Patient weight changes over time are considered influential to limb volume and, thus, fit . Furthermore, some patients can easily adapt to accommodate socket shape errors while others cannot. The degree of acceptable error is likely patient-dependent, and a challenge for prosthetics researchers will be determining which patient characteristics correlate with low threshold to socket shape error and which do not.
One means for overcoming manufacturing error in prosthetics is to use a different modality. Direct socket fabrication approaches have been pursued in research investigations and potentially have less error than current CAM techniques [12-16,18]. While moving to one of these technologies can potentially overcome shape fabrication error, these technologies are not yet commercially available for prosthetics. High cost, licensing, and mechanical strength issues are part of the limitations. Further, it is not clear if, from a socket quality standpoint as discussed earlier, such technologies are necessary. There may, however, be other advantages to these technologies outside of just fabrication accuracy.
Another approach to correcting for socket shape error for individual fabrication facilities is to characterize the nature of the error and then correct it in the carving process . In a recent investigation, most of the time (7 of 10 facilities tested), all three models tested for a company had relatively consistent error, meaning that a modeling approach should be capable of characterizing an error . The model can then be used to correct the fabrication process by altering the e-file shape sent to the manufacturing equipment. Using models to correct fabrication processes like these is a technique pursued in other fields, for example, the automotive industry . In engineering, this effort is known as "solving the inverse problem." Such algorithms proceed as an optimization procedure in which the manufacturing e-file shape is locally altered until the shape predicted by the model is within a specified tolerance for the application at hand. These sorts of correction algorithms do not currently exist in prosthetic socket manufacturing. However, they could potentially improve socket fabrication by automating correction.
An issue related to shape error pertains to the point density specified in e-file shapes used to make models and sockets. Too fine a density can detrimentally affect computing speed and disk storage space as well as slow down manufacturing speed. Too low a density potentially introduces shape error that may make the socket fit unacceptable. Once the accuracy needs of models and sockets are better determined by clinical outcomes assessment, then it should be clear whether prosthetic CAM point densities need adjustment from their current levels. Quantification of the spatial frequency content of a population of well-fit, traditionally made sockets would be another approach to gaining insight into an appropriate point density. Spatial frequency analysis has been investigated by Hastings et al. as related to digitizer performance, but not for model or socket fabrication .
With recent enhancements in imaging systems, such as the laser scanner used to assess models in this study, characterizing the shapes of positive models is simplified over custom techniques described previously . Small scanners that fit within a prosthetic socket are starting to emerge in the market and should soon be able to replace custom socket-scanning instruments. Once commercially available technology is demonstrated to have sufficiently low error to meet the need, fabrication facilities will have a viable means to conduct in-house assessment using a methodology similar to that described here to determine whether they have manufacturing errors.
It is important to recognize that when engineering an evaluation technology for socket shape, an important harmony exists between the measurement technique and the means for aligning measured shapes (e.g., models, sockets, and e-file shapes). The optimization algorithm described in this investigation combined maximization of shape similarity and minimization of volume difference . In the literature, researchers have described other methods for alignment, including minimization of volume difference , anatomical landmarks [24-26], and top and bottom slice centroids . We developed this new assessment technique because our investigations demonstrated that alignment of residual-limb shapes using a mean absolute difference (volume minimization) algorithm alone or a surface normal optimization algorithm alone depended on the nature of the shape difference, which was not acceptable . Use of a combination of minimization of mean error and alignment of surface normals did not depend on the nature of shape difference and, thus, produced better results. If socket and model shapes are to be compared with each other or to e-file shapes, then care must be taken to ensure that the alignment algorithm is not sensitive to the nature of the shape difference. An additional goal is to create an alignment optimization algorithm that is minimally sensitive to measurement errors in the scanning instrument with which it is coupled.
Potentially, the assessment technique described in this article can be applied to other aspects of CAM, namely, the limb shape measurement phase. In 2007, Geil showed that manual, optical, and electromagnetic shape-capture systems produced anthropometric measurements similar to each other, thus suggesting minimal advantage to using electronic measurement systems for shape assessment . In 2008, McGarry et al. showed that an electromagnetic contact sensor system did not perform as well on transtibial models as on cylindrical models . Geil and McGarry et al.'s evaluations were very helpful toward telling us where the technology stood in 2007 and 2008, respectively, but it still remains to be determined how accurate scanning technologies must be to effectively implement CAM methods.
This article presents an assessment technique to evaluate the quality of CAM models and sockets. The prosthetics computer-aided socket fabrication industry may benefit from creating and implementing model and socket assessment standards to enhance quality and consistency across the industry. Such an effort might improve practitioner confidence in CAM, serve as a basis for enhancement, reduce patient healthcare costs, and serve as a means for documenting performance. Further research is needed to establish a relationship between CAM socket fabrication error and patient functional outcome, prosthesis performance, and the duration of time the socket is usable by the patient with amputation.
Go to TOP
Last Reviewed or Updated Thursday, August 18, 2011 1:02 PM