Journal of Rehabilitation Research & Development (JRRD)

Quick Links

  • Health Programs
  • Protect your health
  • Learn more: A-Z Health
Veterans Crisis Line Badge

Volume 48 Number 6, 2011
   Pages 707 — 718

Using virtual reality environment to facilitate training with advanced upper-limb prosthesis

Linda Resnik, PT, PhD, OCS;1-2* Katherine Etter, MS;2 Shana Lieberman Klinger, MA;1 Charles Kambe, BS2

1Providence Department of Veterans Affairs Medical Center, Providence, RI; 2Department of Community Health, Brown University, Providence, RI

Abstract — Technological advances in upper-limb prosthetic design offer dramatically increased possibilities for powered movement. The DEKA Arm system allows users 10 powered degrees of movement. Learning to control these movements by utilizing a set of motions that, in most instances, differ from those used to obtain the desired action prior to amputation is a challenge for users. In the Department of Veterans Affairs "Study to Optimize the DEKA Arm," we attempted to facilitate motor learning by using a virtual reality environment (VRE) program. This VRE program allows users to practice controlling an avatar using the controls designed to operate the DEKA Arm in the real world. In this article, we provide highlights from our experiences implementing VRE in training amputees to use the full DEKA Arm. This article discusses the use of VRE in amputee rehabilitation, describes the VRE system used with the DEKA Arm, describes VRE training, provides qualitative data from a case study of a subject, and provides recommendations for future research and implementation of VRE in amputee rehabilitation. Our experience has led us to believe that training with VRE is particularly valuable for upper-limb amputees who must master a large number of controls and for those amputees who need a structured learning environment because of cognitive deficits.

Key words: amputation, control mechanisms, DEKA Arm, medical devices, motor learning, motor pathways, prosthetics, prosthetic training methods, rehabilitation, virtual reality, virtual reality environment.

Abbreviations: 3-D = three-dimensional; DISC = DEKA Integrated Solutions, Inc; FSR = force-sensitive resistance; IMU = inertial movement unit; IVR = immersive virtual reality; VA = Department of Veterans Affairs; VRE = virtual reality environment.
*Address all correspondence to Linda Resnik, PT, PhD, OCS; Research Health Scientist, Providence VA Medical Center, 830 Chalkstone Avenue, Providence, RI 02908; 401-273-7100, ext 2368; fax: 401-863-3489.

Technological advances in upper-limb prosthetic design offer dramatically increased possibilities for powered movement. The full DEKA Arm (DEKA Research & Development Corporation; Manchester, New Hampshire), for example, allows up to 10 degrees of powered movement in addition to passive degrees of freedom associated with active degrees of freedom (Figure 1).

Figure 1. DEKA Arm displayed on manikin.

Figure 1.

DEKA Arm displayed on manikin.

Click Image to Enlarge. View as PowerPoint Slide

No previous prosthetic device has given users control over so many degrees of freedom. While these new capabilities are exciting and offer many more possibilities for using the arm in space, they also require the use of a more advanced controls scheme than do simpler devices with fewer degrees of freedom.

In 2008, the Department of Veterans Affairs (VA) launched a series of studies to optimize the DEKA Arm system. As of September 2010, we have fit 26 subjects with the DEKA Arm. The DEKA Arm system uses a range of "strap and go" options for controlling powered movements that are customized for the user. The control options include foot controls with force-sensitive resistance (FSR), foot controls with inertial movement units (IMUs), air bladders, switches, and myoelectric control sites. The majority of these controls require the amputee to employ a set of motions and activate muscles that, in most instances, differ from those used to obtain the desired action prior to amputation. When using foot controls, for example, users must learn to associate motor actions, such as pronation or supination of the foot, with specific motor outcomes (customized for the user), such as wrist pronation or supination. This contrasts with use of more invasive alternatives, such as targeted muscle reinnervation strategies, which (after successful surgery) can utilize existing neural pathways to achieve end movements [1-3].

The full DEKA Arm system employs Endpoint Control, a coordinated control scheme, which enables simultaneous control of multiple arm joints, minimizing the cognitive burden that would be required to control all upper-limb joints separately. Endpoint Control derives its name from controlling the end of the device, the terminal device in this case. In Endpoint Control, the device is positioned based on the desired direction of movement of the terminal device; i.e., the trajectory of the movements of the upper-limb joints is designed to move the terminal device in space. Regardless of the number of joints involved, only six control signals are needed to position the arm. The wrist and grip patterns can be operated separately from Endpoint Control, allowing the user to position the arm in space and then perform fine motor tasks using a different set of controls.

Although the DEKA Arm system uses Endpoint Control for arm movements, users must also learn to sequence these with movements of the hand to perform functional tasks. Lastly, they must learn to problem solve and preplan their actions. In short, a great deal of motor learning needs to take place for an amputee to become a skilled user of the DEKA Arm.

Stages of Motor Learning

Motor learning occurs in three stages: an initial phase, an intermediate phase, and an advanced phase [4]. During the initial or cognitive phase, subjects are engaged in basic acquisition of information about the motor skill, acquiring knowledge of the task's requirements and goals [5]. Information is derived from sensory experience, which may take the form of listening to verbal instructions, reading a written description of a task, or seeing a demonstration. The learner's greatest challenge during this phase is integrating knowledge of the instructions with the movements that are required to operate the controls and elicit the appropriate arm movement.

During the intermediate or associative phase, participants practice the task to refine the information derived in the cognitive phase. This practice is aimed at efficiently accomplishing the goals set out in the cognitive phase. Learners in the intermediate phase depend on sensory guidance for motor output and use feedback from both the sensory systems and external sources, such as therapists, to identify and correct their errors. The motor action becomes less disjointed during this phase, with a merging of steps [6].

During the final or autonomous stage, the steps of a task fuse into a continuous motor action. Learners become proficient at the desired skill and are now concerned with speed and accuracy of performance. Once this phase of learning has been reached, the learner no longer needs to give conscious thought to performing the action but can make adaptive changes to the action [6]. Thus, the goal of prosthetic training is to achieve the final stage of motor learning and enable skilled operation of the prosthesis.

We wanted the participants in the VA studies of the DEKA Arm to master the controls of their prostheses in the most efficient manner; we needed to move subjects through the study in a rapid time frame. Control of 10?? of powered movements creates a cognitive burden and can be challenging for some amputees. Thus, we attempted to develop the best prosthetic training program possible, informed by findings from the literature on motor learning.

Creating New Model of Motor Control of Prosthetic Arm

Complex task performance requires selection of appropriate sequencing and timing of movement. Previous investigators have developed models explaining the internal process of sensorimotor integration allowing for complex motor control [7-9]. According to Wolpert et al., motor commands must be mapped internally onto their sensory consequences using a forward model and sensory consequences must be mapped onto motor commands using an inverse model [7-9]. For example, in learning to use one of the control options, the foot controls of the DEKA Arm, the user needs to learn the kinematic transformation between the movement of the foot and the movement that is created with the prosthesis. Thus, when learning to use the Endpoint Control of the DEKA Arm, the user needs to transform the designed trajectory of movement into the necessary motor command to the foot to achieve the desired trajectory. In the forward model, the brain predicts the actual outcome of a motor command prior to movement and then compares the actual outcome of movement with the desired outcome through sensory feedback. Sensory feedback (visual and kinematic) is then used to fine-tune the movement control. In the inverse model, a desired sensory consequence must be transformed into the appropriate motor command. The development of both forward and inverse internal models is required to learn complex motor tasks.

When sound limbs are manipulated, sensory feedback as well as visual feedback concerning the movement and outcome comes directly from the motion of the limb. However, when using the controls (such as the foot control) of the DEKA Arm, users do not receive proprioceptive feedback from the DEKA Arm itself and must rely on proprioceptive feedback from other body parts (such as the foot and ankle) and the residual limb as well as visual feedback from the prosthetic arm's movement. Thus, increased visual monitoring of the mechanized arm is needed to compensate for the lack of proprioceptive feedback on arm and hand position, speed of movement, and orientation.

We believe that smooth and adept linking of new motor actions to prosthetic arm use requires an integration of the prosthesis into the amputee's sense of self and incorporation into the body image [10]. This process may involve a remapping of the motor representation area of the brain. A motor representation is an area of the brain involved in the planning and carrying out of actions by a particular muscle or muscle group [11]. Motor representations in the primary motor cortex of the brain have been shown to be plastic, changing in response to experience and practice [12], such that a motor representation may be enlarged or reduced by use and sensory input. Thus, for users of an upper-limb prosthesis that is controlled, in part, by foot movements, it is possible that new motor representations can be produced that create a kinematic transformation between the position and movement of the prosthesis and the motor commands of the foot.

Virtual Reality Environment

Given the challenges in developing internal models of motor control in the absence of proprioceptive feedback from the robotic arm, we attempted to facilitate motor learning using strategies that enhance visual feedback with the virtual reality environment (VRE) program provided by DEKA. DEKA's VRE program allows the user to practice controlling the prosthesis within a virtual environment, using the same controls used in operation of the actual arm. We expected that VRE training would provide a good environment for learning the motor patterns required to control movements of the DEKA Arm. The VRE allows the initiation of control-scheme refinement and can be used prior to completion of the interface fitting process. Use of the VRE also allows a gradual acclimatization to the arm: experience with the arm-control scheme prior to use of the physical arm allows a staged introduction of the new elements (control schemes and capabilities) of the arm system. In some sense, use of the VRE could be considered by new prosthetic users as similar to use of training wheels on a bicycle. This may be especially beneficial as the number of controls for more advanced upper-limb prostheses continues to grow with advancements in the technology.

In this article, we provide highlights from our experiences to date in implementing VRE in training amputees to operate the full DEKA Arm. The specific purposes of this article are to (1) discuss the use of VRE in amputee rehabilitation, (2) describe the VRE system used with the DEKA Arm, (3) describe our approach to training subjects using the VRE, (4) provide qualitative data from a case study of a subject using the DEKA powered shoulder to illustrate the use of VRE training, and (5) provide recommendations for future research and implementation of VRE in amputee rehabilitation.

Virtual Reality Environment in Rehabilitation

The term VRE refers to a wide variety of methods used to simulate an alternative or virtual world. VRE can be categorized into two basic forms based on viewing perspective: immersive or nonimmersive. In immersive virtual reality (IVR), the person retains the first-person viewing perspective as if in the real world. In contrast, nonimmersive VRE uses an avatar, which can take any form, to represent the user in the virtual world, creating a third-person perspective. Nonimmersive VRE requires the user to internalize the avatar's movements as his or her own through the use of visual perception processing [13]. Further categorization of VRE types is possible based on the delivery method of the image, such as through use of sensory goggles or projection onto a screen, as well as how much of the real world a user can see at any given time. Some VRE systems provide the user with visual, sensory (haptic), or auditory feedback [14-15]. Combining all three forms of feedback can help the user become immersed in the virtual world [15].

VRE was used in other scientific disciplines before its introduction into the field of rehabilitation and continues to be used in such areas as training of pilots and surgeons. The benefit of VRE for rehabilitation is that it enables stringent control over the environment with which the user interacts. This level of environmental control is not available in the real world [16]. Several studies report on the effectiveness of VRE for rehabilitation of patients with stroke, cerebral palsy, and severe spinal cord injuries [17-18]. The VRE-based Nintendo Wii system (Nintendo Corporation, Ltd; Kyoto, Japan) has been widely adopted in rehabilitation research and practice because of its low cost and interactive features, such as haptic feedback sensation in the form of controller vibrations, goal-oriented achievements, user control input, feedback on speed of action, and most recently, stability and movement position feedback [19-20].

Virtual Reality Environment in Amputee Rehabilitation

VRE has been used in amputee rehabilitation to treat phantom-limb pain and in training patients to operate myoelectric controls, as well as in prosthetic development research. Prior research using VRE in the amputee population has focused largely on its application for reducing phantom-limb pain. This method of pain reduction evolved from work using mirrors to create the visual illusion of a second sound limb and progressed to using VRE [21-22]. In an IVR system, the movements of the sound limb can be prerecorded through motion tracking and/or a sensor glove prior to the start of training [23]. Software then transposes and projects these movements into the area occupied by the phantom limb. As the amputee views a screen or wears display goggles, it appears as if the phantom limb is moving as the amputee thinks about moving both limbs simultaneously [23]. Specific tasks can be prerecorded depending on training structure; however, movements are limited to this precomposed library [23]. The reduction of pain has been shown to last for weeks following VRE training [22]. No studies have currently demonstrated the ability to produce longer lasting results following the end of training sessions. The effectiveness of this method has been demonstrated in unilateral amputees, though in theory the movements could be created by a surrogate for a bilateral amputee.

Virtual Reality Environment in Prosthetic Development

VRE can play an important role on the development side of a prosthesis, aiding engineers during the design phase and prior to product testing, as well as on the clinical side, assisting clinicians as a training and fitting tool [24]. Benefits of VRE use prior to product manufacturing include visualizing component layout and function in a three-dimensional (3-D) space, testing control algorithms, and exploring interactions of the prosthesis with simulated real-world objects for possible limitations [24].

Virtual Reality Environment for Prosthetic Training

VRE has promise as a teaching tool for training amputees to control and use a new myoelectric prosthetic device [25-26]. VRE, in its simplest form, can be used on a computer screen, and a simple software program could be taken home by users for practice. An early example of the use of VRE for prosthetic training was demonstrated by the MyoBoy trainer system introduced by Otto Bock (Duderstadt, Germany). The MyoBoy allows users to practice generating myoelectric signals, and can help users isolate and improve the strength of myosites used in prosthetic control. The user's myoelectric sites are connected to the MyoBoy, and visual feedback is provided on the activation and strength of the signal. As Otto Bock's myoelectric hand evolved, so did the MyoBoy to include a visual representation of the myoelectric hand. This visual representation of a virtual hand that responds to a user's input provides a link between control activation and prosthesis movement. Users practice generating the myoelectric signals required to move the arm and are provided with feedback via the image of the hand, which simulates movement of the actual prosthesis.

Additional computer-aided VRE training systems have been developed that provide external motivation in the form of task completion or achievements. Lovely et al. created a system that employs video games to help motivate pediatric amputees during the early myoelectric signals training phase of rehabilitation [27]. Using the system, the amputee points and shoots at targets presented at random locations on the computer screen. The pointer is operated by a combination of myoelectric signals from the residuum and a joystick controlled by the sound side. This system could be used with either single- or two-site myoelectric systems and could be tailored to enable therapist evaluation, with a system similar to that used in the University of New Brunswick Universal Myo-Electric Trainer [27].

Future applications of VRE for prosthetic training could involve early training, after amputation and prior to socket fitting, as training in most cases does not require the amputee to be able to wear the prosthesis [25]. In theory, users could become more proficient with the device controls by training in VRE prior to use.

Virtual Reality Environment System Used with DEKA Arm

The VRE system used in the VA "Study to Optimize the DEKA Arm" consisted of a real-time, 3-D avatar that simulated the movement of the DEKA Arm system. The avatar used with the VRE program consisted of a full torso and head, with both upper limbs intact, as well as additional visual information on selected grip and arm mode. Subjects used their prosthetic controls to operate the virtual arm of the avatar on the VRE screen.

Use of the VRE provided real-time visual feedback on use of the prosthetic controls, providing the user with valuable information about the aspects and dynamics of movement of the arm for each given command. The avatar in the VRE exhibited the same joint constraints as the arm; for example, maximum forearm pronation and supination in the VRE was the same as for the physical arm. Use of the VRE also enabled users to gain greater familiarity with the abilities and motion trajectories of the DEKA Arm, some of which are unique because of combination patterns used in Endpoint Control. Subjects could also experience tactile and auditory feedback from the arm system on mode selection during the use of the VRE. The VRE system used in our study did not provide opportunities to interact with a virtual environment.

The VRE is integrated with the controls-fitting software such that the prosthetist is able to see the control signals simultaneously with the VRE image. This allows the clinician to quickly modify the user's control scheme (e.g., change the action required to open or close the hand) based on the user's experience during the VRE training. This quick feedback and adjustment allows the prosthetist and user to refine their control-scheme selections during the VRE training process.

The VRE system was customizable, in that it could be configured to represent a left or a right amputee and could be used for all levels of upper-limb amputations. The avatar could be viewed from the front (Figure 2), back, side, above, or in a combination of views (Figure 3).

Figure 2. Virtual reality environment avatar from front.

Figure 2.

Virtual reality environment avatar from front.

Click Image to Enlarge. View as PowerPoint Slide

Figure 3. Virtual reality environment avatar from above and behind.

Figure 3.

Virtual reality environment avatar from above and behind.

Click Image to Enlarge. View as PowerPoint Slide

The user could zoom in to focus on particular joints. Thus, the user could see the movement of the virtual arm from a number of perspectives and change the perspective during the use of the VRE, as well as track the path and speed of movements resulting from activation of control inputs. The motions of the arm and hand segments could be viewed in relation to the rest of the body, providing visual information about position in space. We expected that this information might help users form a new visual cognitive framework to substitute for missing proprioceptive information typically obtained in a sound limb from sensory organs for position and movement.


Subjects in our research trained with the VRE prior to training with the DEKA Arm. All VRE sessions were guided by study occupational therapists. The subject viewed the VRE screen either on a laptop positioned in front of him or her or, in some cases to make the images larger and easier to see, on a screen projected onto a nearby wall. Subjects typically wore only the controls, not the DEKA Arm itself, at the beginning of VRE training and then wore the controls together with the nonactivated DEKA Arm during later VRE sessions. Prior to VRE training, subjects had spent minimal or no time using the controls to activate the arm itself. The VRE simulation provided an early experience activating the motor pathways required to operate the controls and gave subjects the opportunity to see the resultant gross movements of the virtual arm. Thus, subjects observed the motions that they created on the avatar before they acquired substantial physical procedural knowledge. Via this route, we expected that the VRE training would help create a preexisting mental framework that could be used when operating the actual DEKA Arm.

When using a full arm with a powered shoulder, users had to learn up to 16 control actions associated with about half as many control sites (depending on their setup scheme and version of Endpoint Control) (Figure 4). To enable the greatest degree of control over a variety of actions, study therapists assigned most control sites two separate movements, one accomplished while the arm was in "Arm" mode, and the other accomplished while the arm was in "Hand" mode. Additionally, subjects needed to become familiar with the stimuli and meaning of several notification signals. An auditory beep with accompanying vibration indicates grip selection, and a different sound and vibratory sensation occur when mode selection changes. Lastly, the DEKA Arm has a tactor that provides vibratory feedback for detection of changes in grip pressure.

Figure 4. List of all controls for user of DEKA Arm.

Figure 4.

List of all controls for user of DEKA Arm.

Click Image to Enlarge. View as PowerPoint Slide

During the controls-setup process, subjects were provided with a handout that summarized their controls scheme to reinforce the location of each control site and its assigned action in each mode. Subjects were asked to take a copy of this handout home between setup sessions and attempt to memorize the controls used for each movement.

During their VRE training, subjects had a copy of their controls handout placed on the table in front of them to use as a reference. As the VRE training progressed, the therapist removed the controls handouts to help the user actively remember the information and facilitate later retention of controls information. The handout was returned to the user for a reference guide as necessary.

In our study to date, all users of the full DEKA Arm use foot controls. Movements of the feet are required to operate these controls by exerting pressure on FSRs located on different aspects of the foot or by moving the foot and ankle to operate the IMUs worn on the top of the shoe. Activation of these types of foot controls requires somewhat different amounts of force or movement depending upon the subject's position. Because of this, we had our subjects practice controlling the arm in VRE in the sitting as well as standing position, and in some cases (while using the IMUs), with the feet off the ground while seated.

At the outset of VRE training, therapists asked subjects to trigger each of the controls in a specified order, first in one mode (Arm or Hand) and then to repeat the controls activation while the arm was in the other mode. Subjects repeated each of the movements of the virtual arm multiple times to ensure that they understood how to correctly control the foot action (or other control such as myoelectric or air bladder) and that they observed the resulting action of the arm on the avatar. The therapist then requested that the subject perform specific movements of the virtual arm in a random order.

In many cases, subjects had to learn new vocabulary terms to describe arm and hand movements (i.e., terms like pronation, supination, wrist flexion, wrist extension). Therapists introduced this terminology and employed a number of strategies to stimulate information retention, provide practice, and allow subjects to apply the new information to solve simple problems. As examples, subjects were asked to verbalize what actions they performed to achieve each of the arm movements, to demonstrate the specific movements requested on their sound side (if applicable), and to identify all of the control sites and explain their resulting movements.

The VRE training was also used as an opportunity to introduce the six different grip patterns of the DEKA Arm, allow subjects to practice opening and closing each grip and mimic the grip pattern with the sound side, and discuss applications of each of the patterns. Thus, subjects must learn the order of grips so that they recognize how many times they need to trigger the control movement to arrive at the grip of their choice. Some subjects also have a sensor that allows them to toggle through the grips in reverse order. Thus, subjects need to memorize the order of the grips in a forward and, if using backward toggle, backward direction. VRE training incorporated practice toggling through the grips and memorizing the grips in forward and reverse order (if applicable).

After subjects mastered control of gross movements, they were introduced to several more complex sequences that would be useful for performing basic functional tasks. Generally speaking, a certain amount of preplanning of motor activities is needed for users of the DEKA Arm to achieve complex tasks. For example, subjects were instructed in several strategies to pick up a spoon and bring the hand to the mouth and then return the spoon to the table. Examples of preplanning strategies might include selecting the appropriate grip pattern or prepositioning the wrist before initiating movement with Endpoint Control. Once they were taught these strategies, subjects had the opportunity to practice them using the VRE. They were also asked to verbalize the sequence of arm movements and associated controls necessary to achieve the desired functional task. The components of the VRE training program are summarized in Figure 5.

Figure 5. Virtual reality environment (VRE) training components.

Figure 5.

Virtual reality environment (VRE) training components.

Click Image to Enlarge. View as PowerPoint Slide


We present a case example of a recent subject who experienced the VRE training as part of the VA "Study to Optimize the DEKA Arm" to illustrate our experience using the VRE training protocol.

This subject was a 55-year-old white, male nonveteran, with a left unilateral forequarter amputation secondary to cancer about a year prior to participation in the study. At the time of the study, the subject had been using a body-powered prosthetic for approximately 3 months and indicated that he felt that he was still learning to use it. He used his device approximately 8 hours a day. He wore a figure nine harness to operate the device's control cable. By abduction of the scapula (on the sound side), he used the cable to control elbow flexion (with elbow unlocked) and open the terminal device (with elbow locked) (Otto Bock Ergo Arm [Otto Bock, 2383]). Nudge control was used for elbow lock. His shoulder device had two locking positions that could be used to place the shoulder into flexion and abduction (less than 90??).

The subject was fitted with a DEKA Arm attached via a thermoplastic X-frame socket design with a contralateral thoracic pad (also an X-frame shape). A pneumatic pressure sensor (air bladder) was attached to the external surface of the contralateral pad. He was fit with bilateral IMU foot controls that were worn on the top of his shoes, secured by a clip to his shoe laces. The subject used foot controls on both feet to provide most control inputs. A pressure-sensitive bladder was used to switch between arm modes and place the arm in standby when necessary. He had two notifications given by a tactor and auditory beeps: one for grip select and one for mode select.

The subject participated in approximately 3.5 hours of VRE training in 8 sessions over the course of 4 days led by an occupational therapist. The subject wore the socket and his controls for seven of the eight sessions and wore the deactivated DEKA Arm for one of the VRE training sessions. All VRE sessions were videotaped. Videotapes were analyzed by study staff to extract key observations and comments. Additionally, the subject completed several written surveys on the usefulness of VRE and was asked about the VRE during his final semiguided interview.

During his very first VRE session, the subject commented that he had a strong sensation of phantom-limb movement while he was creating and watching the movement of the avatar:

It's hard for me to explain how my mind is visually accepting what the avatar is doing. It's because I'm picturing my actual arm, you know, the left arm and hand that I used to have-I'm picturing. . . I see that picture in my mind through the use of this, which is you know, I don't know if you can see, but in the chair, I'm moving somewhat like this because I'm trying to make the phantom arm function like the avatar.

This phantom sensation, which the subject found pleasant, persisted into the second day of VRE training but was not experienced once he began operating the DEKA Arm itself without the visual feedback provided by the VRE:

I'm feeling it in the phantom arm-I don't have any pain; it's not phantom pain I'm experiencing. I'm experiencing phantom motion. Which is, this is really, really cool.

As per the training protocol, the subject practiced operating the controls in standing and sitting positions. This was clearly of value in helping him experience the subtle control of foot movement needed to operate the controls with precision. He commented that during VRE training his experience of operating the foot controls was different standing than it was sitting:

It takes less effort [in the sitting position]. When I'm standing, my weight is compressed, so it's a lot easier to move the feet, but this [sitting], I get more sensitivity because I'm not glued to the floor by this 190 pound body . . . .

The subject practiced learning to operate the IMU controls with his feet off the ground. The following quotation illustrates how the occupational therapist introduced use of the controls with the feet elevated:

So, let's say you are watching TV with your foot up and you want to reach for the remote control. And your arm is supinated, so you have to pronate. Instead of rolling [the] foot on floor, you move [the] foot this way (demonstrates movement of the foot with her hand). So how do you extend your wrist?

The subject recognized the amount of gross motor coordination that he needed to master the controls. In an early VRE session, he commented that he thought that an athletic (i.e., more coordinated) person would have an easier time in learning to operate the DEKA Arm than someone with a sedentary lifestyle:

Let's face it, this is, you know, this [learning to use the DEKA Arm] is going to take training, but depending upon the physical ability of the individual will depend upon the accuracy or acuity [with] which they'll be able to control it. You know, I mean if someone is an athletic individual, I think they'll have a much easier time functioning with this, both in a sedentary and an active manner. Whereas somebody who is primarily sedentary, they're going to have a little difficulty, I think, getting accustomed to [it].

By the end of the entire study protocol, the subject explained how mental clarity was helpful in motor planning-enabling him to operate the controls without thinking too much about them:

The more I use it, the more familiar I get with it, and the more that I utilize the powers of my brain before I attempt to move the arm and the hand and the shoulder, the better off I am . . . . The thinking process for me now is becoming almost second nature because I'm using all my senses-my hearing, my eyes, and my brain to operate the functions of the arm through my feet. If I know I want to reach 45 degrees across the table in front of me to go from left to right to pick up a cup of coffee by the handle, I know exactly what I have to do before I do it, which aids me in making the arm functional. The more proficient an individual can become mentally with the mechanical operating devices-the IMUs, the sensors-the better off they'll be.

This subject used the VRE in two views. He used the aerial view, with the perspective from above and slightly behind his head and looking down on the head and arms. In this view, the left and right sides of the avatar were located on the same side as the subject's own left and right sides. He also used the front view, in which he saw the avatar from the front, as if looking in a mirror. However, the left and right sides of the avatar were reversed from the perspective of the subject. The subject stated he preferred the front view:

The aerial view makes it a little more difficult with positioning the arm, the hand. The visual sensation of it is a lot different than seeing it in this position [front, mirrorlike]. This position, I find for me, is a lot easier to work with as far as control, motion, and the actual visual stimulation of watching the avatar's arm function.

There were some delays in wireless connectivity of the DEKA Arm system controls used, resulting in a slight lag time between controls activation and movement of both the avatar and the prosthesis itself. There were subtle differences in the responsiveness of the VRE avatar compared with the prosthesis. This subject reported that the VRE system responded more quickly to his commands than did the actual prosthesis. He reported that it required less effort to activate controls with the VRE than with the actual arm. He noted the differences when he transitioned from VRE training to use of the arm, commenting:

It [VRE training] made the use of the arm easier to a degree. It just felt that the VRE, the model, responded quicker to the commands than the DEKA Arm does.

The VRE system that we used did not allow for interaction with virtual objects, and therefore, the subject had no prior experience manipulating an object with the virtual hand. Thus, it is not surprising that the subject commented on the transition between using the VRE and the actual arm when asked to pick up a block for the first time:

This ain't as easy as it looked on the avatar. I'll tell you that.

At the end of the study, 27 days after his last VRE session and after 30 hours of actual training with the arm, the subject compared and contrasted his virtual (VRE) and actual experiences of training with the arm. He suggested improvements to the VRE that would allow him to interact with virtual objects in the VRE environment:

The VRE training was good. It did assist me in knowing how to position my feet, the different pressure-sensitive positions where I'd have to move, you know, to be able to move the arm or the hand, ah, that worked really good. But the reality of it, it is virtual, but you've got to put soda bottles, cans, pots and pans, you know, things like that in there instead of the individual having to imagine what he's doing . . . .

Our study protocol was developed to capitalize on the enhanced visual feedback provided by the VRE and the relative simplicity of learning to operate the multiple DEKA Arm controls in the absence of distractions that might result from interaction with the environment. Its use provides real-time visual feedback on success of controls activation in an environment free from potential concerns, such as dropping an object or worrying about the arm inadvertently bumping into the user's own body. Our study was not designed to assess the effectiveness or impact of VRE training, and thus, we have no method of formally comparing the ease or effectiveness of prosthetic training with and without prior use of the VRE. This subject became a competent user of the DEKA Arm and was able to perform many functional and recreational activities, including holding and strumming a guitar (Figure 6).

Figure 6. Subject using DEKA Arm to hold guitar.

Figure 6.

Subject using DEKA Arm to hold guitar.

Click Image to Enlarge. View as PowerPoint Slide

Our experience with the VRE, as exemplified by this case, raises multiple questions for further investigation and research. The subject in our case example reported strong sensations of phantom movement while using the VRE. This finding is in keeping with previous reports in the literature on the use of mirror therapy or VRE for the treatment of phantom pain. This subject was a relatively new amputee and, thus, may have been somewhat more susceptible to this type of experience. The uniqueness of this case is that the phantom movement was experienced in concert with the movement of the avatar, which resulted from the subject's operation of the foot controls. It is possible, then, that this type of kinesthetic experience facilitated development of an internal forward sensorimotor model, perhaps speeding up the learning process.

In this VRE case, some small variations in movement and timing existed between the avatar and the real DEKA Arm based on prototype data-transfer approaches. We expect that the more closely the movement and timing of movement of the avatar can be made to exactly match that of the actual arm, the better the transference of skills learned in the VRE to actual prosthetic use will be. Further research is needed to examine the effect of such differences on the motor learning process.


Our experiences to date have led to many interesting observations and new hypotheses that could be tested in future research. For example, future studies might evaluate the speed of learning to use complex controls with and without VRE for subjects with different levels of amputation, as well as for different types and numbers of controls, and explore changes in the motor representation in the brain that result from VRE training and/or from use of the foot to control a virtual upper limb.

The subject described in our case example was a forequarter amputee who used 16 controls to operate the DEKA Arm system. It appeared that use of the VRE facilitated his learning process. We believe that the value of virtual environment training for an upper-limb amputee is greater for those amputees who must master a greater number of controls. We also believe that VRE may be useful for amputees who need more structured learning environments because of cognitive deficits. Our preliminary feedback suggests that the effectiveness of VRE systems may be enhanced by the inclusion of virtual objects and interactive activities. Further research is needed to test these hypotheses.

Author Contributions:
Study concept and design: L. Resnik, S. L. Klinger.
Acquisition of data: L. Resnik.
Analysis and interpretation of data: L. Resnik, S. L. Klinger, K. Etter, C. Kambe.
Drafting of manuscript: L. Resnik, S. L. Klinger, K. Etter, C. Kambe.
Critical revision of manuscript for important intellectual content: L. Resnik, S. L. Klinger, K. Etter.
Obtained funding: L. Resnik.
Study supervision: L. Resnik.
Administrative, technical, or material support: C. Kambe.
Financial Disclosures: The authors have declared that no competing interests exist. We are thankful to DEKA Integrated Solutions, Inc (DISC) (Manchester, New Hampshire), who through funding from the Defense Advanced Research Project Agency, developed the DEKA Arm and provided it to our study. DISC operated under a Cooperative Research and Development Agreement with the VA but was not involved in the study design or data collection, analysis, or interpretation. However, they did receive summarized data case studies and reviewed the final version of this manuscript to determine whether or not proprietary information was included.
Funding/Support: This material was based on work supported by the VA Rehabilitation Research and Development Service (grants A6780 and A6780I). DEKA's support of the VA optimization studies was sponsored by the Defense Advanced Research Projects Agency and the U.S. Army Research Office.
Additional Contributions: The authors acknowledge the valuable work of study coordinator Kate Barnabe and members of the research team at the VA New York Harbor Healthcare System: Nicole Sasson, site Principal Investigator; Christopher Fantini, lead prosthetist; Roxanne Disla, occupational therapist; Ken Breuer, prosthetist; and Maryanne Garbarini, research assistant.
Institutional Review: This study received approval from the Providence VA Medical Center Institutional Review Board and the VA New York Harbor Healthcare System Institutional Review Board.
Participant Follow-Up: The authors do not plan to inform study participants of the publication of this study because of a lack of contact information.
Disclaimer: The information in this article does not necessarily reflect the position or policy of the Government; no official endorsement should be inferred.
1. Kuiken TA, Miller LA, Lipschutz RD, Lock BA, Stubblefield K, Marasco PD, Zhou P, Dumanian GA. Targeted reinnervation for enhanced prosthetic arm function in a woman with a proximal amputation: A case study. Lancet. 2007;369(9559):371-80. [PMID: 17276777]
2. Kuiken TA, Li G, Lock BA, Lipschutz RD, Miller LA, Stubblefield KA, Englehart KB. Targeted muscle reinnervation for real-time myoelectric control of multifunction artificial arms. JAMA. 2009;301(6):619-28. [PMID: 19211469]
3. Kuiken TA, Dumanian GA, Lipschutz RD, Miller LA, Stubblefield KA. The use of targeted muscle reinnervation for improved myoelectric prosthesis control in a bilateral shoulder disarticulation amputee. Prosthet Orthot Int. 2004; 28(3):245-53. [PMID: 15658637]
4. Fitts PM, Posner MI. Human performance. Belmont (CA): Brooks/Cole Publishing; 1967.
5. Feygin D, Keehner M, Tendick R. Haptic guidance: Experimental evaluation of a haptic training method for a perceptual motor skill. Proceedings of the 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems; 2002 Mar 24-25; Orlando, FL. Los Alamitos (CA): IEEE; 2002. p. 40-47.
6. Reznick RK, MacRae H. Teaching surgical skills-Changes in the wind. N Engl J Med. 2006;355(25):2664-69.
[PMID: 17182991]
7. Wolpert DM, Ghahramani Z, Jordan MI. An internal model for sensorimotor integration. Science. 1995;269(5232):1880-82.
[PMID: 7569931]
8. Wolpert DM, Ghahramani Z, Flanagan JR. Perspectives and problems in motor learning. Trends Cogn Sci. 2001; 5(11):487-94. [PMID: 11684481]
9. Wolpert DM, Ghahramani Z. Computational principles of movement neuroscience. Nat Neurosci. 2000;3 Suppl: 1212-17. [PMID: 11127840]
10. Van den Bos E, Jeannerod M. Sense of body and sense of action both contribute to self-recognition. Cognition. 2002; 85(2):177-87. [PMID: 12127698]
11. Sanes JN, Donoghue JP. Plasticity and primary motor cortex. Annu Rev Neurosci. 2000;23:393-415. [PMID: 10845069]
12. Classen J, Liepert J, Wise SP, Hallett M, Cohen LG. Rapid plasticity of human cortical movement representation induced by practice. J Neurophysiol. 1998;79(2):1117-23.
[PMID: 9463469]
13. Alessi SM, Hanson T, Wieners M, Petry NM. Low-cost contingency management in community clinics: Delivering incentives partially in group therapy. Exp Clin Psychopharmacol. 2007;15(3):293-300. [PMID: 17563216]
14. Saposnik G , Mamdani M, Bayley M, Thorpe KE, Hall J, Cohen LG , Teasell R; EVREST Steering Committee; EVREST Study Group for the Stroke Outcome Research Canada Working Group. Effectiveness of Virtual Reality Exercises in STroke Rehabilitation (EVREST): Rationale, design, and protocol of a pilot randomized clinical trial assessing the Wii gaming system. Int J Stroke. 2010;5(1): 47-51. [PMID: 20088994]
15. Sveistrup H. Motor rehabilitation using virtual reality. J Neuroeng Rehabil. 2004;1(1):10. [PMID: 15679945]
16. Keshner EA. Virtual reality and physical rehabilitation: A new toy or a new research and rehabilitation tool? J Neuroeng Rehabil. 2004;1(1):8. [PMID: 15679943]
17. Kizony R, Raz L, Katz N, Weingarden H, Weiss PL. Video-capture virtual reality system for patients with paraplegic spinal cord injury. J Rehabil Res Dev. 2005;42(5):595-608.
[PMID: 16586185]
18. Deutsch JE, Borbely M, Filler J, Huhn K, Guarrera-Bowlby P. Use of a low-cost, commercially available gaming console (Wii) for rehabilitation of an adolescent with cerebral palsy. Phys Ther. 2008;88(10):1196-1207.
[PMID: 18689607]
19. Butler DP, Willett K. Wii-habilitation: Is there a role in trauma? Injury. 2010;41(7):671-73. [PMID: 20398899]
20. Shih CH, Shih CT, Chiang MS. A new standing posture detector to enable people with multiple disabilities to control environmental stimulation by changing their standing posture through a commercial Wii Balance Board. Res Dev Disabil. 2010;31(1):281-86. [PMID: 19850444]
21. Ramachandran VS, Rogers-Ramachandran D. Synaesthesia in phantom limbs induced with mirrors. Proc Biol Sci. 1996;263(1369):377-86. [PMID: 8637922]
22. Mercier C, Sirigu A. Training with virtual visual feedback to alleviate phantom limb pain. Neurorehabil Neural Repair. 2009;23(6):587-94. [PMID: 19171946]
23. Murray CD, Patchick E, Pettifer S, Caillette F, Howard T. Immersive virtual reality as a rehabilitative technology for phantom limb experience: A protocol. Cyberpsychol Behav. 2006;9(2):167-70. [PMID: 16640472]
24. Hauschild M, Davoodi R, Loeb GE. A virtual reality environment for designing and fitting neural prosthetic limbs. IEEE Trans Neural Syst Rehabil Eng. 2007;15(1):9-15. [PMID: 17436870]
25. Takeuchi T, Wada T, Mukobaru M, Doi S. A training system for myoelectric prosthetic hand in virtual environment. Proceedings of the International Conference on Complex Medical Engineering. 2007 May 23-27; Beijing, China. Los Alamitos (CA): IEEE; 2007. p. 1351-56.
26. Soares A, Andrade A, Lamounier E, Carrijo R. The development of a virtual myoelectric prosthesis controlled by an EMG pattern recognition system based on neural networks. J Intel Inf Syst. 2003;21(2):127-41.
27. Lovely DF, Stocker D, Scott RN. A computer-aided myoelectric training system for young upper limb amputees. J Microcomputer App. 1990;13(3):245-59.
Submitted for publication July 7, 2010. Accepted in revised form November 9, 2010.
This article and any supplementary material should be cited as follows:
Resnik L, Etter K, Klinger SL, Kambe C. Using virtual reality environment to facilitate training with advanced upper-limb prosthesis. J Rehabil Res Dev. 2011;48(6): 707-18.

Go to TOP

Go to the Table of Contents of Vol. 48 No. 6

Last Reviewed or Updated  Monday, July 11, 2011 9:42 AM

Valid HTML 4.01 Transitional