Journal of Rehabilitation Research & Development (JRRD)

Quick Links

  • Health Programs
  • Protect your health
  • Learn more: A-Z Health
Veterans Crisis Line Badge
 

Volume 48 Number 6, 2011
   Pages 619 — 628

Target Achievement Control Test: Evaluating real-time myoelectric pattern-recognition control of multifunctional upper-limb prostheses

Ann M. Simon, PhD;1* Levi J. Hargrove, PhD;1-2 Blair A. Lock, MS;1 Todd A. Kuiken, MD, PhD1-3

1Center for Bionic Medicine, Rehabilitation Institute of Chicago, Chicago, IL; 2Department of Physical Medicine and Rehabilitation, Feinberg School of Medicine, Northwestern University, Chicago, IL; 3Biomedical Engineering Department, Robert R. McCormick School of Engineering and Applied Science, Northwestern University, Evanston, IL

Abstract — Despite high classification accuracies (~95%) of myoelectric control systems based on pattern recognition, how well offline measures translate to real-time closed-loop control is unclear. Recently, a real-time virtual test analyzed how well subjects completed arm motions using a multiple-degree of freedom (DOF) classifier. Although this test provided real-time performance metrics, the required task was oversimplified: motion speeds were normalized and unintended movements were ignored. We included these considerations in a new, more challenging virtual test called the Target Achievement Control Test (TAC Test). Five subjects with transradial amputation attempted to move a virtual arm into a target posture using myoelectric pattern recognition, performing the test with various classifier (1- vs 3-DOF) and task complexities (one vs three required motions per posture). We found no significant difference in classification accuracy between the 1- and 3-DOF classifiers (97.2% +/- 2.0% and 94.1% +/- 3.1%, respectively; p = 0.14). Subjects completed 31% fewer trials in significantly more time using the 3-DOF classifier and took 3.6 +/- 0.8 times longer to reach a three-motion posture compared with a one-motion posture. These results highlight the need for closed-loop performance measures and demonstrate that the TAC Test is a useful and more challenging tool to test real-time pattern-recognition performance.

Key words: multifunctional prosthesis, myoelectric control, pattern recognition, performance test, proportional control, prosthesis, surface electromyography, transradial amputation, upper limb, virtual environment.

Abbreviations: DOF = degree of freedom, EMG = electromyography, LDA = linear discriminate analysis, MAV = mean absolute value, TAC Test = Target Achievement Control Test, TMR = targeted muscle reinnervation, ULPOM Group = Upper Limb Prosthetics Outcome Measures Group.
*Address all correspondence to Ann M. Simon, PhD; Center for Bionic Medicine, 345 E Superior St, Room 1309, Chicago, IL 60611; 312-238-1158; fax: 312-238-2081.
Email: asimon@ric.org
DOI:10.1682/JRRD.2010.08.0149
INTRODUCTION

Myoelectric control systems based on pattern recognition have been proposed for the next generation of multifunctional upper-limb prostheses [1-3]. Ideally, a multifunctional prosthesis will restore functionality to users and measurably improve their quality of life. Unfortunately, the only validated prosthetic outcomes measure is the Assessment for Capacity of Myoelectric Control [4-5], which measures a user's ability to perform a series of two-handed tasks and requires a physical prosthesis under volitional control. The Upper Limb Prosthetics Outcome Measures Group (ULPOM Group) was formed in 2005 to address the lack of outcomes measures for upper-limb prosthetics [6]. In 2009, the ULPOM Group presented findings identifying a wide range of variables that contribute to prosthesis usability. One variable, "control of the prosthesis," included the need for a measurement tool sensitive enough to both differentiate between control schemes and show changes in the ability to control the prosthesis over time [7]. The ULPOM Group recognized that the evolution of prosthesis design constitutes a continuum of research, development, clinical work, and ultimately, home use. The ULPOM Group recommended that a series of tests be used to iteratively test each preceding stage in the continuum. In this article, we work on the development portion of the continuum by attempting to develop a test that may be used to quantify previously researched control algorithms before implementing them in physical devices that may or may not yet exist. Specifically, we developed a test to measure the user's ability to control pattern-recognition systems of varying complexity and comprising different components.

During pattern-recognition control, a computer program identifies a user's intended movements by using the pattern produced by several channels of surface electromyography (EMG) signals [8]. The computer classifies the pattern and sends a movement command to the prosthesis. A large focus of pattern-recognition research is providing better EMG decoding by using various classifiers and feature sets [3,9-12]. The performance of a classifier is commonly assessed by calculating its classification accuracy after all data has been collected. Classification accuracy is the capability of the algorithm to correctly decode user movements. Pattern classification techniques such as linear discriminate analysis (LDA) [1,13], fuzzy logic [3,14], or artificial neural networks [10-11,13] commonly achieve offline classification accuracies ~95 percent.

However, how a pattern classifier's performance in offline tests translates to its performance in real-time closed-loop control is relatively unclear [15]. Data are collected while the user tries to produce a specific motion. After the experiment is over, the data are processed and classification accuracy is calculated as the percent of times the classifier correctly identifies the motion. Therefore, classification accuracies are calculated during an open-loop task during which the user has no feedback. At the beginning stages of development, offline accuracies provide useful information without needing a multifunctional prosthesis. With offline performance established, the need for evaluation tools based on real-time performance becomes more apparent because it is important to investigate what happens to performance when the user is interacting with the decoded movement.

Virtual environments can provide an alternative setting for evaluating real-time pattern-recognition perform-ance [16-19]. Through the use of a virtual clothespin task, Hargrove et al. showed that system controllability and functional performance improved when they included the transient portion of the EMG signals in classifier training [17]. This is of noted importance because including this information may have the opposite effect on offline classification accuracy by lowering the reported performance of the system [17]. More recently, Kuiken et al. designed a virtual test called the Motion Test that examined the clinical robustness and accuracy of pattern-recognition control [18] in users with amputation who had undergone targeted muscle reinnervation (TMR) surgery [20-21]. The Motion Test instructs users to follow prompts for a movement and observe a virtual prosthesis decoding their movements. Users must maintain their muscle contractions until the virtual prosthesis moved through its full range of motion [18]. Previous Motion Test results suggest that the reinnervated muscles of users with amputation and TMR can produce sufficient EMG information for real-time pattern-recognition control [18]. Although this test provides real-time performance metrics, the required task is oversimplified; motion speeds are normalized and unintended movements (i.e., misclassifications) are ignored.

We included these considerations (i.e., motion speeds and misclassifications) in a new, more challenging virtual test called the Target Achievement Control Test (TAC Test). The TAC Test evaluates user control and positioning of a multifunctional prosthesis. We instructed subjects to move a virtual prosthesis into a target posture and maintain it for a period of time (i.e., 2 s) (Figure 1). If the subject overshot the target posture or produced unnecessary movements (either through volitional control or motion misclassifications), these had to be corrected to achieve success.


Figure 1. Target Achievement Control Test (TAC Test). Subjects moved multi-functional virtual prosthesis into target posture. Virtual hand turned green when target was reached within acceptable tolerances (+/-5?? for each degree of freedom). Figure illustrates starting and ending posi-tions for successful trials. (a) Example trial from conditions 1 and 2 requiring one motion to reach target posture (e.g., wrist flexion).(b) Example trial from condition 3 requiring three motions to reach target posture (e.g., wrist flexion, wrist supination, and hand close).

Figure 1.

Target Achievement Control Test (TAC Test). Subjects moved multi-functional virtual prosthesis into target posture. Virtual hand turned green when target was reached within acceptable tolerances (±5?? for each degree of freedom). Figure illustrates starting and ending posi-tions for successful trials. (a) Example trial from conditions 1 and 2 requiring one motion to reach target posture (e.g., wrist flexion).(b) Example trial from condition 3 requiring three motions to reach target posture (e.g., wrist flexion, wrist supination, and hand close).

Click Image to Enlarge. View as PowerPoint Slide

In this study, subjects with a transradial amputation controlled a virtual prosthesis using myoelectric pattern recognition with proportional control. To illustrate the flexibility of the new virtual performance test, subjects performed the TAC Test with two classifier complexities (a 1-degree of freedom [DOF] classifier and a 3-DOF classifier) and two task complexities (one or three motions required to achieve target posture success). Results of subjects testing both complexities showed that the TAC Test provides valuable information about user myoelectric control and pattern-recognition control algorithms that could not be obtained using existing performance measures (e.g., offline classification accuracy) or existing real-time virtual performance tests (e.g., Motion Test).

METHODS
Subjects

Five individuals with a transradial amputation participated in this study. Table 1 presents demographic information about the subjects.


Table 1.
Demographics of subjects with transradial amputation.

Subject
Age (yr)
Arm with Amputation
Arm
Tested
Time Since Amputation
Type of
Prosthesis Used

1
53
Right
Right
20 yr
Myoelectric
2
62
Right
Right
25 yr
Myoelectric
3
55
Bilateral
Right
32 yr
Body-powered
4
24
Left
Left
9 mo
Body-powered
5
32
Bilateral
Right
3 yr
Body-powered


Electromyographic and Pattern-Recognition
Configuration

We used six self-adhesive silver/silver chloride bipolar surface electrode pairs (Noraxon Dual electrodes, Noraxon; Scottsdale, Arizona) to record muscle activity. The electrode pairs had a 1 cm diameter circular conductive area and a 2 cm interelectrode distance. We placed four electrode pairs in a ring at the proximal portion of the residual forearm around the apex of the muscle bulge (2-3 cm distal to elbow crease) and positioned the other two electrode pairs on the distal end of the residual limb. For subjects with short residual limbs, we placed all six electrodes in the ring at the proximal portion of the forearm. The EMG signals were amplified, sampled at a frequency of 1 kHz and high-pass filtered at 20 Hz, to reduce motion artifact.

Subjects trained the system to recognize seven motion classes (wrist flexion, wrist extension, wrist supination, wrist pronation, hand open, hand closed, and no movement). To train the pattern-recognition system, we prompted subjects by demonstrating each movement and asked them to perform the movement at a comfortable and consistent level of effort. In training, subjects held each contraction for 3 s, repeated eight times. We split the data into two groups with 12 s of data used to train the LDA classifier and 12 s of data used to test the LDA classifier. The pattern-recognition system segmented data from all EMG channels into a series of 150 ms analysis windows with a 50 ms window increment. We extracted four time-domain features (mean absolute value [MAV], number of zero crossings, waveform length, and number of slope sign changes) from each analysis window. With this LDA classifier, only one class decision was made at a time (i.e., sequential control). This pattern-recognition scheme has been previously described [1] and has shown to produce effective real-time control [18,22]. After subjects trained the classifier, it was used to predict user commands and control a virtual prosthesis in real time. We assessed classification accuracy offline by dividing the number of correct class decisions by the total number of class decisions. For this experiment, we calculated the proportional movement speed by averaging the MAV of all channels (k) of EMG signals for a given data window and multiplying by a class gain factor (G) [15,23]:

Equation 1.

where i = class. We configured desired speed gains for each class such that subjects could achieve full dynamic range where the maximum EMG amplitude corresponded to 100?? per second. Subjects practiced in the virtual environment for 5 to 10 min prior to testing.

Target Achievement Control Test

A screen displayed a target posture and a virtual prosthesis that responded to classifier output (Figure 1). For the TAC Test, the subjects moved the virtual prosthesis from a nonneutral position to a neutral posture (target). The neutral position was 0?? of wrist flexion or extension and 0?? of wrist rotation (see Figure 1, successful trial end). To provide visual feedback, the virtual hand turned green when it was within an acceptable tolerance of the target (±5?? for each DOF) (Table 2). Subjects completed tests more quickly if they only produced the motion(s) necessary to reach the target. If a subject overshot the target posture or produced unnecessary movements, he or she had to correct those motions to achieve success. TAC Test trials ended successfully when subjects were able to keep the virtual prosthesis in the target for 2 s. Target postures were never at the end of DOF ranges, which ensured controlled stopping and dwelling within the target posture as part of the required task. TAC Test trials ended unsuccessfully if subjects were unable to achieve and maintain the target posture by the specified time out. We tested three conditions in this study.


Table 2.
Target Achievement Control Test configurable parameters.

Parameter
Description
 
Study Setting

Test Complexity
Number of motions required to reach target posture.
 
1 (C1, C2); 3 (C3)
       
Movement Distance
Distance between initial position of virtual hand and target posture for each tested motion. Larger or smaller distances can be used to test gross or fine motor control.
 
75??
       
Target Width
Acceptable tolerance for reaching target posture. Smaller target widths lead to more challenging trials.
 
±5??
       
Dwell Time
Length of time virtual prosthesis has to continuously remain in target posture for trial to be considered successful.
 
2 s
       
Trial Time Out
Length of time in which trial must be completed. Trial is considered failed if time out is reached without success.
 
15 s (C1, C2); 45 s (C3)

C = condition.


Condition 1

Subjects controlled a 1-DOF virtual prosthesis and performed the TAC Test with one required motion per trial. We tested each DOF separately. For wrist rotation, we used only the data for wrist supination, wrist pronation, and no movement to build and test the LDA classifier. The target posture required subjects to either supinate or pronate the virtual wrist across a movement distance of 75?? to achieve success. Subjects repeated the protocol for wrist flexion and extension and hand open and closed. We presented the 1-DOF classifiers to the subjects in random order. For each 1-DOF classifier, subjects performed four sets of the TAC Test; each set consisted of two repetitions of each target posture (two postures) with a trial time out of 15 s. Condition 1 consisted of a total of 48 trials.

Condition 2

Subjects controlled a 3-DOF virtual prosthesis and performed the TAC Test with one required motion per trial. We used data for all seven motion classes to build and test the LDA classifier. Similarly to condition 1, the target posture only required subjects to perform one motion across a movement distance of 75?? to achieve success. Unlike condition 1, all three DOFs were active during each trial. For example, if a subject was trying to pronate his or her wrist and the hand closed, the subject needed to reopen the hand before achieving the target posture. Subjects performed four sets of the test; each set consisted of two repetitions of each target posture (six postures) with a trial time out of 15 s. Condition 2 consisted of a total of 48 trials.

Condition 3

Subjects controlled a 3-DOF virtual prosthesis and performed the TAC Test with three required motions per trial. Similarly to condition 2, we used data for all motions to build and test the LDA classifier. Unlike condition 2, target postures required subjects to perform three motions, such as wrist flexion, wrist supination, and hand open, to achieve success. Each trial required moving the virtual prosthesis across 75?? for each required motion. Therefore, subjects had to move the virtual prosthesis over a total 225??. Subjects performed four sets of the test; each set consisted of one repetition of each target posture (eight postures). Since the pattern-recognition algorithm used in this study allowed only sequential motions, the trial time out for condition 3 was 45 s. Condition 3 consisted of a total of 32 trials.

Prior to testing, we gave subjects at least 5 min to familiarize themselves with each condition. We presented the conditions to the subjects in random order. We used the first test set of each condition as practice and subsequent sets for data analysis. We analyzed the effects of classifier complexity by comparing conditions 1 and 2 and the effects of task complexity by comparing conditions 2 and 3.

TAC Test performance metrics included completion time, completion rate, and path efficiency. We set completion time as the time from trial start to the successful achievement of the target posture, not including the 2 s dwell time. We set completion rate as the percentage of successfully completed postures in a set of trials. We calculated path efficiency as the shortest path to the target divided by the total distance traveled by the virtual hand [24]. Therefore, a trial with a path efficiency equal to 100 percent indicated that the subject was able to move the virtual prosthesis directly into the target posture and stop within the acceptable tolerance. We only reported completion time and path efficiency for successful trials.

Statistical Analysis

We performed a paired t-test to assess the statistical difference between classification accuracy, completion rate, completion time, and path efficiency across the two levels of classifier and task complexities.

RESULTS
Classifier Complexity (Comparison of Conditions 1 and 2)

Classification accuracy was not significantly different between the 1- and 3-DOF classifiers (p = 0.14). Average classification accuracy was 97.2 ± 2.0 percent (mean ± standard deviation) across all 1-DOF classifiers (condition 1) and 94.1 ± 3.1 percent across all 3-DOF classifiers (condition 2).

When the TAC Test required only one motion per posture, subjects completed significantly more trials and completed them significantly faster while using the 1-DOF classifier compared with using the 3-DOF classifier (p = 0.002 for completion rate and p < 0.001 for completion time) (Figure 2, Table 3). Figure 3 displays the position and decision history of an example trial using the 3-DOF classifier. Path efficiency measures demonstrated a similar trend of significantly decreased performance with the 3-DOF classifier compared with the 1-DOF classifier (p = 0.03).


Table 3.
Target Achievement Control Test performance measures by condition.

Condition
Classifier Complexity (DOF)
Task Complexity (No. of Motions)
Classification Accuracy (%)
Completion
Rate (%)
Completion
Time (s)
Path
Efficiency (%)

1
1
1
97.2 ± 2.0*
99.4 ± 1.2*
2.9 ± 1.0*
92.8 ± 3.9*
2
3
1
94.1 ± 3.1*
68.9 ± 9.3*†
5.6 ± 0.9*†
81.1 ± 5.0*†
3
3
3
94.1 ± 3.1
92.1 ± 7.6
20.1 ± 4.0
54.7 ± 11.1

*Paired t-test indicates significant difference between conditions 1 and 2 (p < 0.05).
Paired t-test indicates significant difference between conditions 2 and 3 (p < 0.05).
DOF = degree of freedom.



Figure 2. Average completion rate curves for all three conditions. Solid line indicates performance during trials that required only one motion per posture using 1-degree of freedom (DOF) classifier (condition 1). Dashed line indicates performance during trials that required only one motion per posture using 3-DOF classifier (condition 2). Dotted line indicates performance during trials that required three motions per posture using 3-DOF classifier (condition 3). Shaded regions represent±1 standard error.

Figure 2.

Average completion rate curves for all three conditions. Solid line indicates performance during trials that required only one motion per posture using 1-degree of freedom (DOF) classifier (condition 1). Dashed line indicates performance during trials that required only one motion per posture using 3-DOF classifier (condition 2). Dotted line indicates performance during trials that required three motions per posture using 3-DOF classifier (condition 3). Shaded regions represent±1 standard error.

Click Image to Enlarge. View as PowerPoint Slide

Task Complexity (Comparison of Conditions 2 and 3)

When the TAC Test required subjects to perform three motions to achieve each posture (condition 3), subjects completed significantly more trials (p = 0.03) in a significantly longer time (p = 0.001) compared with performing only one motion (condition 2) (Figure 2, Table 3). Note that the trial time out length for condition 3 was three times that of condition 2. Figure 4 displays the position and decision history of a subject using the 3-DOF classifier to reach a posture that required three motions. The average completion time for achieving a three-motion posture was 3.6 ± 0.8 times longer than the average completion time for a one-motion posture. The average path efficiency for the three-motion posture was significantly lower than that for the one-motion posture (p = 0.01).


Figure 3. Position and decision history during example Target Achievement Control Test (TAC Test) trial requiring one motion to reach condition 2 target posture using 3-degree of freedom (DOF) classifier. Virtual prosthesis began in 75?? wrist extension, 0?? wrist rotation, and hand 50% closed. User had to flex wrist to reach target posture (0?? flexion or extension, 0?? wrist rotation, and hand 50% closed). Gray bars indicate target position for each DOF. Since TAC Test required all DOFs to match target position, subject had to correct any misclassifications (e.g., wrist pronation). Virtual arm reached target position at 5.3 s (indicated by T*). Trial ended at 7.3 s after subject was able to remain in target posture for 2 s.

Figure 3.

Position and decision history during example Target Achievement Control Test (TAC Test) trial requiring one motion to reach condition 2 target posture using 3-degree of freedom (DOF) classifier. Virtual prosthesis began in 75?? wrist extension, 0?? wrist rotation, and hand 50% closed. User had to flex wrist to reach target posture (0?? flexion or extension, 0?? wrist rotation, and hand 50% closed). Gray bars indicate target position for each DOF. Since TAC Test required all DOFs to match target position, subject had to correct any misclassifications (e.g., wrist pronation). Virtual arm reached target position at 5.3 s (indicated by T*). Trial ended at 7.3 s after subject was able to remain in target posture for 2 s.

Click Image to Enlarge. View as PowerPoint Slide

Figure 4. Position and decision history during example Target Achievement Control Test trial requiring three motions to reach condition 3 target posture using 3 degrees of freedom. Virtual prosthesis began in 75?? wrist flexion, 75?? wrist supination, and hand 25% closed. User had to extend and pronate wrist and close hand to reach target posture(0?? flexion or extension, 0?? wrist rotation, and hand 75% closed). Gray bars indicate target position for each degree of freedom. Virtual arm reached target position at 18.2 s (indicated by T*). Trial ended at 20.2 s after subject was able to remain in target posture for 2 s.

Figure 4.

Position and decision history during example Target Achievement Control Test trial requiring three motions to reach condition 3 target posture using 3 degrees of freedom. Virtual prosthesis began in 75?? wrist flexion, 75?? wrist supination, and hand 25% closed. User had to extend and pronate wrist and close hand to reach target posture(0?? flexion or extension, 0?? wrist rotation, and hand 75% closed). Gray bars indicate target position for each degree of freedom. Virtual arm reached target position at 18.2 s (indicated by T*). Trial ended at 20.2 s after subject was able to remain in target posture for 2 s.

Click Image to Enlarge. View as PowerPoint Slide

DISCUSSION

We investigated subject performance with various classifier and task complexities to highlight the TAC Test. The existing offline measure of classification accuracy is a limited metric of control caused by a ceiling effect. Classification accuracy is bounded by a maximum value of 100 percent, with pattern-recognition algorithms commonly reporting accuracies ~95 percent. The result of decades of research into classifier types and feature sets is very minimal increases, if any, in classification accuracy and how these changes relate to controllability is unclear. Existing virtual performance measures such as the Motion Test are oversimplified. The Motion Test prompts subjects to perform one motion until the virtual prosthesis moves through its full range of motion [18]. In a previous study by Li et al., users with a transradial amputation controlled 11 motions of a virtual prosthesis with an average classification accuracy of 79 ± 11 percent [22]. Users successfully completed 72 percent of the Motion Test trials [22]. In the current study, subjects who performed the TAC Test controlled only seven motion classes of a virtual prosthesis (condition 2) with an average classification accuracy of 94.0 ± 3.1 percent. Even with fewer classes and much higher classification accuracy, subjects successfully completed only 69 percent of the TAC Test trials, highlighting the need for closed-loop performance measures. The TAC Test is challenging because subjects are required to "undo" unintended movements and command DOF stopping, because all DOFs are needed to match the target posture. Also unlike the Motion Test, the TAC Test allows subjects to move the virtual prosthesis at a slow or fast rate, depending on the intensity of their muscle contraction.

The TAC Test does not exhibit similar ceiling effects because a wide range of testing difficulties can be achieved by modifying test parameters (Table 2). For example, if a subject was able to achieve a 100 percent completion rate with a ±10?? tolerance on the target posture, the experimenter or clinician can reduce the tolerance to ±5?? to make the test more difficult. In the current study, we asked subjects to position the virtual arm into postures that required either one or three movements. An interesting observation was that subjects did not seem to be as affected by misclassifications at the beginning of movement while attempting to achieve a three-motion posture compared with a one-motion posture. While performing the TAC Test with one required motion per posture, subjects would often correct unintended movements as they happened (Figure 3). During trials that required three motions per posture, many subjects did not correct movement misclassifications right away but rather waited until they were closer to the target posture to correct the movements as needed (Figure 4). In this case, misclassifications may actually have helped complete the motion. It is also possible that subjects were unable to determine whether the virtual hand was at the target in one DOF until the other DOFs were close to their target positions. These observations are not possible with other existing virtual performance measures. Similar observations are harder to track when subjects use a physical prosthesis since current physical prostheses do not include position-tracking.

In addition to testing subject performance, the TAC Test provided a good environment for subjects to practice pattern-recognition control. To succeed in the test, subjects needed to plan their movements and produce repeatable muscle patterns. Movement timing and sequential control are other key pattern-recognition concepts. Subjects needed to control their muscle contraction length and intensity to properly position the virtual prosthesis and relax their muscles without eliciting another motion in order to keep the virtual arm at the target. Because the pattern-recognition algorithm used in the current study only allowed sequential control, subjects needed to perform only one motion at a time. Since the TAC Test does not depend on the type of control, algorithms that provide simultaneous and proportional control [25] can be tested within this virtual environment if they prove beneficial. The variable configuration (movement distance, time limit, acceptable tolerance, etc.) allows subject testing with different performance levels while still engaging subjects and maintaining motivation (Table 2).

The TAC Test must not be confused with a validated upper-limb prosthesis usability outcome measure. The ability to control a prosthesis is only one important component in a user's overall ability to use a prosthesis, which may include many other variables such as terminal device type, functional level, motivation, and level of therapy [7]. We developed the TAC Test to specifically test control algorithms so that differences in control strategies may be measured and compared. This aligns with the findings from the ULPOM Group's critical recommendation that the information to be captured is specific to the area in which it is being tested. We believe that the TAC Test captures important information about the control algorithms being tested; therefore, we assume that systems that score higher on the TAC Test will be more controllable and ultimately, more usable. Future work must be completed to test that assumption.

One limitation of the TAC Test is that subjects interact with a virtual, not a physical, environment. Although the goal of developing this virtual test was not to completely replicate the physical environment, it is important to acknowledge the differences. User control and perform-ance may differ between these two environments because the virtual environment does not model the prosthesis inertia. Wearing a physical prosthesis may alter the way users contract their muscles. Supporting the prosthetic weight may also affect how quickly muscles fatigue. During physical prosthesis control, not only position but also force and acceleration matter. While performing the TAC Test, users can successfully complete trials even with large terminal decelerations (e.g., they can stop abruptly in the target posture). Large terminal decelerations with a physical prosthesis may cause unwanted interactions (e.g., placing a cup down too fast may cause the liquid to spill). Finally, for patients to use prosthesis, they would still require significant therapy for pattern-recognition systems that prove to be controllable in a virtual environment.

CONCLUSIONS

Although fundamental differences exist between the virtual and physical environments, the TAC Test provides a good platform for pattern-recognition control practice and testing. The virtual test measures real-time performance in a variety of testing settings. Current results demonstrate that significant online differences can be seen even when no significant offline differences exist. Therefore, the virtual test provides more control information in the continuum of offline measure of classification accuracy and the full setup necessary for physical prosthesis testing.

ACKNOWLEDGMENTS
Author Contributions:
Study concept and design: A. M. Simon, L. J. Hargrove, B. A. Lock, T. A. Kuiken.
Acquisition and analysis of data: A. M. Simon.
Drafting of manuscript: A. M. Simon, L. J. Hargrove, B. A. Lock.
Obtained funding: T. A. Kuiken.
Financial Disclosures: The authors have declared that no competing interests exist.
Funding/Support: This material was based on work supported by the National Institutes of Health, National Institute of Child Health and Human Development (grants R01 HD058000-01 and R01 HD043137-05).
Additional Contributions: The authors thank Aimee Schultz for editing the manuscript.
Institutional Review: The Northwestern University Institutional Review Board approved the experimental protocol, and subjects gave written informed consent before participation.
Participant Follow-Up: The authors do not plan to inform participants of the publication of this study. However, once the article is
published, it will be posted to our laboratory's Web site.
REFERENCES
1. Englehart K, Hudgins B. A robust, real-time control scheme for multifunction myoelectric control. IEEE Trans Biomed Eng. 2003;50(7):848-54. [PMID: 12848352]
DOI:10.1109/TBME.2003.813539
2. Chu JU, Moon I, Mun MS. A real-time EMG pattern recognition system based on linear-nonlinear feature projection for a multifunction myoelectric hand. IEEE Trans Biomed Eng. 2006;53(11):2232-39. [PMID: 17073328]
DOI:10.1109/TBME.2006.883695
3. Ajiboye AB, Weir RF. A heuristic fuzzy logic approach to EMG pattern recognition for multifunctional prosthesis control. IEEE Trans Neural Syst Rehabil Eng. 2005;13(3): 280-91. [PMID: 16200752]
DOI:10.1109/TNSRE.2005.847357
4. Wright FV. Measurement of functional outcome with individuals who use upper extremity prosthetic devices: Current and future directions. J Prosthet Orthot. 2006;18(2): 46-56. DOI:10.1097/00008526-200604000-00006
5. Miller LA, Swanson S. Summary and recommendations of the Academy's State of the Science Conference on Upper Limb Prosthetic Outcome Measures. J Prosthet Orthot. 2009;21(9):83-89. DOI:10.1097/JPO.0b013e3181ae974d
6. Hill W, Stavdahl ??, Hermansson LN, Kyberd P, Swanson S, Hubbard S. Functional outcomes in the WHO-ICF model: Establishment of the Upper Limb Prosthetic Outcome Measures Group. J Prosthet Orthot. 2009;21(2):115-19.
DOI:10.1097/JPO.0b013e3181a1d2dc
7. Hill W, Kyberd P, Norling Hermansson L, Hubbard S, Stavdahl ??, Swanson S. Upper Limb Prosthetic Outcome Measures (ULPOM): A working group and their findings. J Prosthet Orthot. 2009;21(9):69-82.
DOI:10.1097/JPO.0b013e3181ae970b
8. Oskoei MA, Hu H. Myoelectric control systems-A survey. Biomed Signal Process Control. 2007;2(4):275-94.
9. Fukuda O, Tsuji T, Ohtsuka A, Kaneko M. EMG-based human-robot interface for rehabilitation aid. Proceedings of the 1998 IEEE International Conference on Robotics and Automation; 1998 May 16-20; Leuven, Belgium. Los Alamitos (CA): IEEE; 1998.
10. Hudgins B, Parker P, Scott RN. A new strategy for multifunction myoelectric control. IEEE Trans Biomed Eng. 1993;40(1):82-94. [PMID: 8468080]
DOI:10.1109/10.204774
11. Gallant PJ, Morin EL, Peppard LE. Feature-based classification of myoelectric signals using artificial neural networks. Med Biol Eng Comput. 1998;36(4):485-89.
[PMID: 10198534]
DOI:10.1007/BF02523219
12. Englehart K, Hudgins B, Parker P, Stevenson M. Classification of the myoelectric signal using time-frequency based representations. Med Eng Phys. 1999;21(6-7):431-38.
[PMID: 10624739]
DOI:10.1016/S1350-4533(99)00066-1
13. Hargrove LJ, Englehart K, Hudgins B. A comparison of surface and intramuscular myoelectric signal classification. IEEE Trans Biomed Eng. 2007;54(5):847-53.
[PMID: 17518281]
14. Chan FH, Yang YS, Lam FK, Zhang YT, Parker PA. Fuzzy EMG classification for prosthesis control. IEEE Trans Rehabil Eng. 2000;8(3):305-11. [PMID: 11001510]
DOI:10.1109/86.867872
15. Lock BA, Englehart K, Hudgins B. Real-time myoelectric control in a virtual environment to relate usability vs. accuracy. MyoElectric Controls Symposium; 2005 Aug; Fredericton, Canada.
16. Smith RJ, Huberdeau D, Tenore F, Thakor NV. Real-time myoelectric decoding of individual finger movements for a virtual target task. Conf Proc IEEE Eng Med Biol Soc. 2009;2009:2376-79. [PMID: 19965192]
17. Hargrove L, Losier Y, Lock B, Englehart K, Hudgins B. A real-time pattern recognition based myoelectric control usability study implemented in a virtual environment. Conf Proc IEEE Eng Med Biol Soc. 2007;2007:4842-45.
[PMID: 18003090]
18. Kuiken TA, Li G , Lock BA, Lipschutz RD, Miller LA, Stubblefield KA, Englehart KB. Targeted muscle reinnervation for real-time myoelectric control of multifunction artificial arms. JAMA. 2009;301(6):619-28.
[PMID: 19211469]
DOI:10.1001/jama.2009.116
19. Shenoy P, Miller KJ, Crawford B, Rao RN. Online electromyographic control of a robotic prosthesis. IEEE Trans Biomed Eng. 2008;55(3):1128-35. [PMID: 18334405]
DOI:10.1109/TBME.2007.909536
20. Kuiken T. Targeted reinnervation for improved prosthetic function. Phys Med Rehabil Clin N Am. 2006;17(1):1-13. [PMID: 16517341]
DOI:10.1016/j.pmr.2005.10.001
21. Kuiken TA, Dumanian GA, Lipschutz RD, Miller LA, Stubblefield KA. The use of targeted muscle reinnervation for improved myoelectric prosthesis control in a bilateral shoulder disarticulation amputee. Prosthet Orthot Int. 2004; 28(3):245-53. [PMID: 15658637]
22. Li G , Schultz AE, Kuiken TA. Quantifying pattern recognition-based myoelectric control of multifunctional transradial prostheses. IEEE Trans Neural Syst Rehabil Eng. 2010;18(2): 185-92. [PMID: 20071269]
23. Hargrove L, Zhou P, Englehart K, Kuiken TA. The effect of ECG interference on pattern-recognition-based myoelectric control for targeted muscle reinnervated patients. IEEE Trans Biomed Eng. 2009;56(9):2197-2201.
[PMID: 19692302]
DOI:10.1109/TBME.2008.2010392
24. Williams MR, Kirsch RF. Evaluation of head orientation and neck muscle EMG signals as command inputs to a human-computer interface for individuals with high tetraplegia. IEEE Trans Neural Syst Rehabil Eng. 2008;16(5): 485-96. [PMID: 18990652]
DOI:10.1109/TNSRE.2008.2006216
25. Jiang N, Englehart KB, Parker PA. Extracting simultaneous and proportional neural control information for multiple-DOF prostheses from the surface electromyographic signal. IEEE Trans Biomed Eng. 2009;56(4):1070-80.
[PMID: 19272889]
DOI:10.1109/TBME.2008.2007967
Submitted for publication August 10, 2010. Accepted in revised form February 28, 2011.
This article and any supplementary material should be cited as follows:
Simon AM, Hargrove LJ, Lock BA, Kuiken TA. Target Achievement Control Test: Evaluating real-time myoelectric pattern recognition control of multifunctional upper-limb prostheses. J Rehabil Res Dev. 2011;48(6): 619-28.
DOI:10.1682/JRRD.2010.08.0149
Crossref

Go to TOP

Go to the Table of Contents of Vol. 48 No. 6

Last Reviewed or Updated  Monday, July 11, 2011 9:40 AM

Valid HTML 4.01 Transitional