VA Research and Development LOGO

Logo for the Journal of Rehab R&D
Volume 43 Number 5, August/September 2006
Pages 657 — 670


Tools for understanding and optimizing robotic gait training

David J. Reinkensmeyer, PhD;1-2* Daisuke Aoyagi, MS;1 Jeremy L. Emken, MS;2 Jose A. Galvez, PhD;1 Wade Ichinose, MS;1 Grigor Kerdanyan, MS;3 Somboom Maneekobkunwong, MS;3 Koyiro Minakata, BS;1 Jeff A. Nessler, MS;1 Roger Weber, BA;3 Roland R. Roy, PhD;4 Ray de Leon, PhD;5 James E. Bobrow, PhD;1 Susan J. Harkema, PhD;4 V. Reggie Edgerton, PhD4

Departments of 1Mechanical and Aerospace Engineering and 2Biomedical Engineering, University of California (UC) Irvine, Irvine, CA; 3Rancho Los Amigos National Rehabilitation Center, Downey, CA; 4Brain Research Institute, UC Los Angeles, Los Angeles, CA; 5School of Kinesiology and Nutritional Science, California State University Los Angeles, Los Angeles, CA
Abstract — This article reviews several tools we have developed to improve the understanding of locomotor training following spinal cord injury (SCI), with a view toward implementing locomotor training with robotic devices. We have developed (1) a small-scale robotic device that allows testing of locomotor training techniques in rodent models, (2) an instrumentation system that measures the forces and motions used by experienced human therapists as they manually assist leg movement during locomotor training, (3) a powerful, lightweight leg robot that allows investigation of motor adaptation during stepping in response to force-field perturbations, and (4) computational models for locomotor training. Results from the initial use of these tools suggest that an optimal gait-training robot will minimize disruptive sensory input, facilitate appropriate sensory input and gait mechanics, and intelligently grade and time its assistance. Currently, we are developing a pneumatic robot designed to meet these specifications as it assists leg and pelvic motion of people with SCI.
Key words: control strategies, gait training, locomotion, motor control, pelvic assist manipulator, pneumatically operated gait orthosis, rehabilitation, robotics, spinal cord injury, stroke.

Abbreviations: ARTHuR = Ambulation-assisting Robotic Tool for Human Rehabilitation, BWS = body-weight support, DOF = degrees of freedom, PAM = pelvic assist manipulator, POGO = pneumatically operated gait orthosis, SCI = spinal cord injury, SD = standard deviation.
*Address all correspondence to David J. Reinkensmeyer, PhD; UC Irvine, Department of Mechanical and Aerospace Engineering, 4200 Engineering Gateway, Irvine, CA 92697-3975; 949-824-5218; fax: 949-824-8585.
Email: dreinken@uci.edu
DOI: 10.1682/JRRD.2005.04.0073
INTRODUCTION

Gait training with body-weight support (BWS) and manual assistance of the legs and pelvis is a promising rehabilitation technique that may enhance locomotor plasticity following neurological injuries such as spinal cord injury (SCI) and stroke [1-4]. However, the clinical application of this technique is limited because it is labor intensive; it requires two to three therapists to assist the patient's legs and torso during each training session. Another limiting factor is that we do not fully understand the neuroadaptive mechanisms that underlie the training, and thus, we are not sure how to optimize the training. Recognizing these limiting factors, several research groups and a major U.S. healthcare provider (HealthSouth, Birmingham, Alabama) have developed robotic devices for automating locomotor training in humans [5-7].

The focus of each of these devices has been to drive the legs through a stepping pattern in the parasagittal plane while a patient is walking on a treadmill. These devices are useful platforms for automating and studying locomotor training. However, they are relatively limited in the patterns of forces and motions that they can apply, which makes investigating the full range of possible training techniques difficult. At present, development of appropriate control strategies for these devices relies on time-consuming and labor-intensive clinical testing. Further, because of their limited degrees of freedom (DOF), these devices alter the normal walking pattern and thus the sensory input experienced during gait. For example, the Lokomat (Hocoma AG , Volketswil, Switzerland) [5] constrains the pelvis to move only vertically and does not allow internal-external rotation and abduction-adduction of the leg, which alters the muscle activity patterns generated by even nondisabled subjects [8]. The Gait Trainer (Reha-Stim, Berlin, Germany) attaches to the bottom of the foot and does not provide the abrupt transition in ground reaction force that normally occurs between swing and stance [6]. We want patients to practice walking with as normal a gait pattern as possible to facilitate transfer to a functional gait. In terms of neural plasticity, the specific pattern of sensory input provided to the injured nervous system clearly affects levels and timing of muscle activity and thus patterns of plasticity over time [3,9]. Providing as normal as possible sensory input may make locomotor training more efficient.

This article reviews our recent efforts to understand and improve robotic gait-training techniques. These efforts are focused on two goals. First, we are developing tools that can help provide insight into the mechanisms of locomotor adaptation during gait training. We are using these tools to test training strategies in rodent models, to quantify therapist patterns of assistance, to assess locomotor adaptation to novel perturbations, and to model possible computational mechanisms of locomotor training. Studies with these tools are helping guide the progress toward our second goal, i.e., the development of a robotic device and appropriate control algorithms for human gait training.

TOOLS FOR UNDERSTANDING ROBOTIC GAIT TRAINING
The Rat Stepper: A Small-Scale Test Bed

The use of animal models can potentially accelerate the testing of different gait-training strategies as well as facilitate insight into the mechanisms of locomotor plasticity through invasive neuroscience techniques. To further our understanding of injury and recovery in a frequently used animal model, we developed a small robotic system that can measure and manipulate hind-limb stepping on a treadmill by spinal-injured rats [10]. The rat stepper incorporates a motorized treadmill, a BWS system, and a pair of lightweight robot arms that attach to the animal's hind limbs (Figure 1). The robot arms can be operated in a passive mode, which allows for precise recording of hind-limb position, similar to optical motion capture systems with automatic digitization. The robot arms can also perturb or assist stepping. We use the BWS system to adjust the loading level on the hind limbs to within a few grams. A similar device has also been developed for mice [11].


Figure 1. (a) Rat stepper. Rat is placed in cloth harness and attached to end of motorized body-weight support (BWS) device.

We are using the rat stepper as a small-scale test bed to understand locomotor control and plasticity, with a view toward transferring promising training algorithms to human gait training. A recurring principle that we have found important during the development process is that the pattern of sensory input provided to the nervous system determines the quantity and quality of the elicited movement. We briefly review several results that illustrate this principle.

Our first approach to developing a rat robotic device was to use a commercially available haptic robot (PHANTOM 1.0, SensAble Technologies, Inc, Woburn, Massachusetts) to simulate a "virtual treadmill" [12]. We attached the robot to the bottom of the animal's paws and programmed it to provide a support force during stance and to move freely along with the hind limb during swing. We chose this approach because it eliminated the need for a treadmill and allowed for convenient measurement of the ground reaction forces using the robot.

Note that the rats stepped bipedally in experiments with this virtual treadmill, as well as in the other experiments described here. The main reason we have studied bipedal gait is that it allows us to precisely control the load on the hind limbs. If the forelimbs are touching the ground, determining how much load they are bearing relative to the hind limbs is difficult. Since load strongly modulates gait patterns following SCI [13-14], studying bipedal gait allows us to better control this confounding variable. Studies of rat bipedal gait may also lead to insight into the human bipedal gait [15]. At the least, both bipedal rats and humans must solve the same basic problem of coordinating two articulated legs while supporting the body's mass against gravity.

We tested the virtual treadmill concept using rats whose spinal cords were completely transected as neonates (n = 4). Neonatal-transected rats recover some weight-supported stepping ability even without supraspinal input because of plasticity in the locomotor neural circuits in the spinal cord. However, the rats rarely stepped when we placed them on the virtual treadmill. We hypothesized that the reason for the lack of stepping was that the robot arms were attached to the bottom of the paws during swing and that this abnormal sensory input inhibited the swing phase of gait. We then moved the robot attachment point back to the metatarsus, behind the toes. The rats took steps more frequently but still not nearly as often as on a physical treadmill. With the metatarsal attachment, the inappropriate stimulus to the toes was avoided during swing but an appropriate loading pattern was not provided to the toes during stance.

The results from the virtual treadmill experiment led us to switch strategies and attach the robot arms to the lower shanks of the rats as they walked on a physical treadmill. A physical treadmill provides a regular, abrupt transition in sensory input to the paws between load and no-load conditions. The shank attachment minimized obtrusive sensory input, while allowing measurement and manipulation of leg movement. We quantified how frequently the transected animals stepped in the virtual and physical treadmill environments using power spectral analysis of the stepping trajectories [12]. The spectra for the physical treadmill had clear peaks at a stepping frequency of about 1 Hz, whereas the spectra for the virtual treadmill had no discernible peaks, indicative of the sporadic nature of the stepping on the virtual treadmill. We subsequently improved the physical treadmill system by developing custom lightweight robot arms that further minimized the inertial and viscous load applied to the legs during stepping [10] (Figure 1).

In a second experiment highlighting the importance of somatosensory input for stepping, we tested the effect of a small change in load to the paw during stepping in neonatal-transected rats (n = 9 animals) [16]. We programmed the rat-robot arms to push downward on the lower shank during stance with a force proportional to the backward velocity of the paw. Forces of only 6 percent body weight caused a repeatable 7 percent decrease in stance duration. Applying the perturbation to only one leg decreased stance duration in that leg, and the spinal cord adjusted the swing time of the contralateral leg to maintain stepping symmetry. This experiment confirmed that the lumbosacral spinal cord "listens" to the detailed pattern of load information provided to the paw and makes subtle adjustments to the stepping pattern based on what it "hears."

We recently tested the rat stepper with spinally contused animals [17-18], an injury model that mimics the most common SCI pathology in humans. A key feature of spinal contusion models is that a variable amount of descending control over the spinal neurons that generate stepping is retained. Kinematic measurements of stepping are likely related to locomotor ability in these animals, yet establishing these relationships has been difficult. One problem we have encountered is that stepping can be difficult to elicit if the animals are too severely impaired. Moreover, animals that maintain enough descending control to choose not to step will often "allow" their hind limbs to drag on the treadmill. Since locomotor plasticity is use-dependent, training and evaluation would likely be more effective if stepping could be more reliably elicited across a range of contusion severities.

Sensory information related to hind-limb extension and unloading plays an important role in the initiation of swing in animals with SCI [19-20]. Further, we knew from the stance-loading experiment described previously that the spinal cord responds to load information delivered to the hind limb by the rat stepper. Therefore, we hypothesized that robot-assisted extension of the hind limbs of spinally contused rats would increase the probability that the animals would initiate the swing phase of gait [18]. We programmed the rat stepper arms to assist in pulling the hind limbs backward along with the treadmill. This robot action prevented the paw from plantar flexing onto its dorsal surface by resisting upward movement of the lower shank. Contused rats (6 mildly, 10 moderately, 8 severely injured animals, with contusions created by the Infinite Horizons Device [Precision Systems and Instrumentation, LLC, Lexington, Kentucky]) initiated swing with a significantly greater probability (75 16.9% standard deviation [SD], p < 0.001) and generated more steps in a 1-minute period (p = 0.03) when the rat stepper assisted in the extension of their hind limbs compared with the treadmill pulling their hind limbs into extension with no robotic assistance (38.9 16.6% SD) [18]. Thus, enhancing load- and/or position-related sensory information with robot-assisted extension dramatically enhanced swing probability across a range of contusion injuries.

To summarize, our results from the rat stepper highlight the importance of avoiding disruptive sensory input and providing appropriate load-related sensory input to elicit stepping in rats with SCI. Similar principles likely apply to humans with SCI [13,21-22]. The implication is that robotic gait trainers should be designed to maintain appropriate patterns of loading on the feet and legs and phasing between the legs to increase the quantity and quality of stepping activity.

Capturing Therapist Expertise

Experienced therapists have developed knowledge about what type of manual assistance works best for gait training, although much of this clinical knowledge has not been validated by controlled research studies. We have developed a sensor system for measuring the forces and movements applied by therapists [23]. Two 6-axis force-torque sensors (JR3 Inc, Woodland, California), which also contain 6 axes of accelerometers, are attached to a customized, lightweight orthosis. A passive instrumented linkage (MicroScribe, Immersion Corp, San Jose, California) measures the position and orientation of the orthosis. The orthosis is designed to allow the therapist to assist in movement with handhold locations that are as close as possible to the ones used in manual training. The weight of the entire system is 2.0 kg. The seven therapists who have used the system during training so far have remarked that providing manual assistance through it feels different and more limiting than when they directly touch the subject but felt that they could provide adequate assistance for stepping. Results from the system should thus be interpreted as reflecting the best a therapist can do with the particular interface that the system provides, an interface akin to one that might be used by a gait-training robot (i.e., rigid, with sensors interposed) rather than the usual, hand-on-skin, contact of a therapist.

To date, we have measured manually assisted stepping of four subjects with SCI using the orthosis [23]. The University of California Irvine Institutional Review Board approved this and all the other experiments with human subjects described in this article. Each subject underwent one session of locomotor training with three to four therapists taking part. A total of seven therapists have been tested during the four sessions. Preliminary results show that each experienced therapist assisted differently, both in terms of the forces provided and the resultant kinematics (Figure 2). For the same patient and during the same session, greater inter- than intratherapist variability was noted in the forces and kinematics during stepping.


Figure 2.(a) Instrumented orthosis for measuring forces and motion applied by human therapists during locomotor training.

Robotic gait-training devices will standardize the most effective locomotor training. Such standardization may represent a substantial improvement over the large intertherapist variability that is quantifiable even with highly experienced human therapists, although this possibility assumes that an optimal pattern of assistance exists and remains to be tested. Also, some level of intra- or intertherapist variability may be beneficial for training the nervous system, because it may help the nervous system learn to generalize across different training conditions and perhaps across real-world variances.

We plan on using the instrumented orthosis to possibly determine an optimal training technique by correlating short-term outcome measures (e.g., muscle activity and mechanical work) of the locomotor ability of patients with SCI with specific parameters that describe the mechanical assistance provided by different therapists (e.g., force magnitude and timing, stiffness).

Measuring Locomotor Adaptation

We have developed a lightweight robotic device for studying motor adaptation during stepping on a treadmill [24] (Figure 3). The robot, Ambulation-assisting Robotic Tool for Human Rehabilitation (ARTHuR) provides high-bandwidth force control at one attachment point in the parasagittal plane (i.e., bottom of foot, lower shank, or knee). It uses two moving coil forcers to drive either end of a two-bar linkage.


Figure 3. (a) Ambulation-assisting Robotic Tool for Human Rehabilitation (ARTHuR) leg robot. Device uses linear motor with 2 forcer coils and V-shaped linkage to accommodate and drive motion of apex in parasagittal plane.

We have used ARTHuR to implement a robotic protocol that has been very useful for understanding motor adaptation of the arm. This protocol uses a lightweight robotic device to apply a novel dynamic environment (or "force field") to the arm and measures the motor response to this force field during reaching or drawing movements [25]. The novel dynamic environment is often referred to as a "force field" because it is typically a vector field of forces that depend on the state (i.e., position and velocity) of the arm. The protocol has been used extensively for studying arm movement in nondisabled subjects, which demonstrates that internal models are widely used for motor adaptation and control [26]. An internal model is a neural transformation between sensory input and motor output that is used by the nervous system to predictively cancel perturbing forces. The nervous system rapidly builds and alters internal models to move the arms in a coordinated fashion.

Initial studies with ARTHuR demonstrated that the unimpaired motor system uses internal models to control the leg during the swing phase of gait [27-28]. In one experiment, we applied a viscous force field to the lower shank of nondisabled subjects as they stepped on a treadmill. The force field produced a vertical perturbing force proportional to the subject's horizontal shank velocity during swing. When the force field was initially turned on, the subjects stepped higher than normal because the force field pushed their leg upward (Figure 3). The subjects then adapted to the field after about 10 steps and returned toward their normal swing trajectory. When the field was unexpectedly turned off, they exhibited an aftereffect, stepping lower than normal. The aftereffect gradually disappeared after another 10 to 20 steps. The presence of the aftereffect indicates that the locomotor system adapted to the field by predictively canceling it, which is consistent with the use of an internal model.

We recently also found that the process of motor adaptation and internal model formation is well described by an error-based learning law with a "forgetting factor" [28-29]. That is, the time series of step-height errors in the force field was well predicted by the learning law

equation for step-height errors in the force field

where xi = the step height on the ith step, ui = the peak force from the leg on the ith step, xd = the desired step height, g = learning gain, and f = forgetting factor.

This learning law interacts with the spring-like model of the leg dynamics

equation for spring-like model of the leg dynamics

where K = leg stiffness and Fi = the peak vertical force from the robotic force field on the ith step. The parameters of this model can be identified with multiple linear regression of experimental data. The model explains 70 to 80 percent of the variance of step-height error during stepping in the force-field environment [28].

The two key features of this proposed neural learning law are error-based learning and forgetting. Error-based learning refers to a process that describes how the nervous system forms an internal model by adjusting its motor output in proportion to the experienced step-height error. Forgetting refers to a process that describes how the nervous system attempts to reduce the force when errors are small. The average forgetting factor, f, that we have identified for stepping was 0.77 0.1 SD for 10 subjects, a value significantly less than 1 (two-tailed t-test, p < 0.001) [28-29]. Thus, the nervous system acts as if it gradually "forgets" the motor command when trajectory errors are small. We have also recently shown that the learning law, Equation (1), implements an optimization procedure: it minimizes a cost function that is a weighted sum of error, force, and change in force [30]. The forgetting process is the mechanism used by the nervous system to minimize the force component of the cost function.

Understanding the computations that underlie motor adaptation will provide insight into how motor adaptation can be improved. For example, since motor adaptation is driven by kinematic error, providing a larger kinematic error should make motor adaptation occur faster. We recently demonstrated this phenomenon for nondisabled subjects as they adapted to a viscous force field applied by ARTHuR [28] in an adaptation experiment similar to the one described previously. We used ARTHuR to transiently amplify the viscous force field for one step by an amount determined by the computational model. Transiently amplifying the force field briefly increased the kinematic error and resulted in the subjects adapting to the force field in 25 percent less time.

To summarize, our experiments with ARTHuR indicate that a key computational process underlying locomotor adaptation is error-based internal-model formation with forgetting. This process is a form of effort-error optimization [30]. Individuals with SCI who have some preserved descending control are likely to make at least partial use of this optimization process as they learn to walk again. Following neurological injury, the altered response dynamics of the locomotor system might be viewed as a "virtual force field" that must be learned and compensated for to allow the subjects to walk again. If so, amplifying, rather than reducing, stepping errors may prove more effective for increasing the speed of learning in some rehabilitation situations. At the least, we want patients to experience some trajectory errors during gait training, rather than being rigidly driven through a stepping pattern.

Computational Models of Movement Training

Currently, a lack of mathematical models exists to provide a possible rationale for how locomotor training might work. If an effective robotic-training algorithm exists, it should be mathematically describable through reasonable models of motor adaptation. Such models are desirable because they make assumptions concrete and help generate testable hypotheses.

We previously developed an adaptive Markov model of sensorimotor control to study how different types of mechanical assistance would theoretically affect recovery [31]. The model assumes that (1) the central nervous system probabilistically interprets proprioceptive information in real time to generate motor output, (2) sensorimotor pathways become more reliable with repetitive activation in a sort of Hebbian learning, and (3) normal sensory input sometimes elicits abnormal motor output following neurological injury because of disrupted neural organization. The model predicts that the best movement recovery occurs when an external trainer intervenes to correct errant movements on an "as-needed" basis compared with no or continual assistance.

We also developed a model of locomotor training based on the learning law described in the previous section [32]. This model assumes that the nervous system recovers movement ability after injury by relearning the transformation between a desired movement and the required muscle activation patterns, i.e., by learning an "internal model" of its own altered neuromuscular dynamics.

More specifically, assuming that human motor adaptation following SCI is well modeled by the learning law in Equation (1), we asked the question, "How would such a process respond to mechanical assistance provided by a robotic device?" [32]. For these simulations, we assumed that the nervous system adapted according to Equation (1); i.e., we assumed that it iteratively formed an internal model of any perturbing forces. We modeled the robotic device as a position-controlled device that produced a force Ri proportional to the step-height error, with a gain G so that

eqution for step-height error with a gain

We also assumed that the limb behaved like a spring with stiffness, K, in response to muscle force ui and a perturbing force, F, all assumed to operate perpendicular to the swing trajectory so that

equation for swing trajectory

To simulate rehabilitation, we imagined F to be an internal "virtual" force field caused by damage to the neuromuscular system. For example, if a patient had difficulty lifting the foot during the swing phase of gait, this could be viewed as the effect of a virtual force that pushed the leg downward.

We made the simulated robotic training device adaptive by adjusting the position control gain Gi for each trial i based on the trajectory error in a process that mimicked the human controller so that

equation for adjusting the position control gain

where fR < 1. In other words, the robot device increased its stiffness if movement errors were large and decreased it if they were small, with the relative change controlled by the robotic forgetting factor fR and gain gR. Thus, the learning system contained two interacting adaptive controllers; i.e., the simulated neural controller that tried to model the virtual force F to reduce trajectory errors and the simulated robot controller that increased the "firmness" of assistance when trajectory errors were large. We simulated their interaction for different values of f, g, fR, and gR, until the error converged (500 iterations).

Figure 4 shows typical behavior as a function of the robot forgetting factor, fR, where the other parameters were f = 0.9, g = 0.5, gR = 0.5, K = 1, G1 = 100, and xd = 0. These specific values were chosen arbitrarily; the general behavior described here is robust to a wide range of specific values, with the relative relationship between the robot forgetting factor and the human forgetting factor playing the key role in determining the overall type of behavior. Increasing fR decreased the maximum trajectory error experienced throughout training (Figure 4(a)), because the robot assisted more when it slacked less. However, the simulated neural system relied more on the robot's assistance when the robot did not forget (fR = 1) and never learned to model the force-field perturbation (Figure 4(b)).


Figure 4. Simulation results from computational model of adaptive robot trainer coupled to adaptive neural controller.

The critical determinant of whether the neural system learned to model the force field was the relative value of the robot and neural forgetting factors. If the neural system forgot more quickly than the robot, then the neural system began to rely on the robot and did not learn. However, if the robot forgot too quickly, then the neural system experienced large trajectory errors.

In gait training, large errors from a lack of mechanical assistance could lead to falls, which could prevent movement practice, cause injury, and cause a loss of confidence in one's abilities or in the efficacy of the training. The implication of the simulation is that an optimal robot-training algorithm will forget just fast enough to "out-forget" the nervous system but not fast enough that falls become a possibility. The principle is similar to that of the use of training wheels for learning to ride a bicycle, with special care given to continuously adjusting the height of the training wheels to the rider's ability.

We have recently shown that this principle can also be derived by assuming that the robotic movement trainer minimizes a weighted sum of kinematic error (i.e., it limits movement error) and force (i.e., it assists with the least force possible) [33]. That is, the assist-as-needed principle can be expressed as an optimization problem. The solution to this problem is a robotic controller that systematically "backs off" on the level of assistance it provides, similar to the controller in Equation (5). We recently experimentally tested the controller with a nondisabled subject [33]. The controller helped the subject learn an internal model, while creating smaller than normal kinematic errors for the task of walking on a treadmill in the presence of a novel dynamic environment (i.e., the viscous force-field environment described previously). Thus, this work demonstrates the feasibility of deriving robotic controllers for motor training based on mathematical models of learning and optimization principles.

DEVELOPMENT OF A GAIT-TRAINING ROBOT
Working Hypotheses: Normalize Sensory Input and Assist as Needed

Our efforts to develop a gait-training robot for humans are driven by two main hypotheses that are derived from research with the tools described in the previous section. First, we hypothesize that a robotic device that generates as normal as possible sensory input will increase locomotor activity and lead to better locomotor recovery. We are, therefore, developing a robotic device that interfaces with the legs on flexor surfaces during flexion only and extensor surfaces during extension only (Figure 5). The device also allows naturalistic pelvic and leg motion as well as arm swing, which facilitates the normal dynamics of walking.


Figure 5. Pneumatic gait training robot: pelvic assist manipulator (PAM) and pneumatically operated gait orthosis (POGO).

Second, we hypothesize that assisting in movement only as needed will be more effective than providing a fixed amount of assistance, in the sense that such assistance will produce faster and/or better recovery. The rationale here is that the nervous system must eventually experience the dynamics of unassisted walking to learn to control those dynamics. This rationale is supported by subjective reports from experienced therapists, which we hope to verify quantifiably using the instrumented orthosis described previously. It is also supported by the computational models reviewed in the previous section, which model locomotor adaptation as a Hebbian-like learning process or, alternatively, as a process of learning an altered sensorimotor transformation while limiting errors. To allow for the possibility of assistance-as-needed, we are developing a robotic device that can apply a wide range of forces, from fully assisting to "just going along for the ride." Assistance-as-needed is then achieved in part by using a compliant position-controller that only applies forces when the stepping pattern deviates from the normal pattern. The goal of assistance-as-needed is also aided by a control algorithm that synchronizes the assistance of the robot to the stepping pattern of the subject, thereby ensuring that appropriate forces are applied at appropriate times.

POGO: Pneumatically Operated Gait Orthosis

The leg robot ARTHuR has the potential to be used as a gait-training device, but it is relatively expensive and cannot assist at both the knee and the lower shank as experienced therapists commonly do. We are therefore exploring an alternate design for a leg robot, called the pneumatically operated gait orthosis (POGO). POGO is composed of two pneumatic cylinders that are fixed to a telescoping rail system that extends from a hip belt to a molded cuff around the lower shank (Figure 5). One cylinder pushes and pulls between the hip belt and the rail, thus flexing and extending the hip. A second cylinder pushes and pulls between the middle of the rail and a strap wrapped just below the knee, thus providing knee flexion and extension.

This design has several advantages. First, pneumatic cylinders are inexpensive and relatively lightweight for the amount of force they can deliver. Using the nonlinear control laws described here, we can control their force, position, and compliance. Second, since the knee cylinder is referenced off a rail that extends from hip to shank, it cannot hyperextend the knee. Third, the entire system is referenced to the hip belt and therefore accommodates pelvic swivel and tilt. Fourth, the leg attachments are positioned to mimic the handholds used by experienced therapists, pushing on the flexor tendons (hamstring and tibialis anterior tendons) when assisting in leg flexion and the extensor tendons (achilles and patellar tendons) when assisting in leg extension. We are currently performing initial testing of POGO in subjects with SCI.

PAM: Pelvic Assist Manipulator

An important consideration in generating normal sensory input during gait training is pelvic motion. Existing robotic gait trainers restrict pelvic motion to 1 DOF [5-6]. However, the pelvis undergoes three translational displacements and three angular displacements during normal locomotion. These displacements are tightly coupled to step rate and stride-length parameters [34]. Lateral pelvic motion is especially important for shifting load between legs. As found with the rat robot [14], and for humans in other studies [3], load-related afferent input is a powerful input into the locomotor controller that drives spinal plasticity. Pelvic rotation contributes to the swing phase of gait and is carefully coordinated by experienced human gait therapists. Optimal control theory suggests that pelvic motions can be manipulated to assist in the swing motion of the leg [35].

We have therefore developed a device that can accommodate and control naturalistic pelvic motion [36-38] (Figure 5). The pelvic assist manipulator (PAM) consists of two, 3 DOF pneumatic robots that attach to the back of a rigid belt worn by the subject. The PAM attaches to the back of the belt so that hand swing is not impeded. The PAM has 5 DOF that provide control of three translations and two rotations, i.e., pelvic rotation and pelvic obliquity. Pelvic tilt cannot be controlled. A separate force-controlled overhead BWS system (Robomedica Inc, Irvine, California) unloads the patient as needed. The PAM's hip belt is also the base of support for the POGO.

Control Strategies

To control the pneumatic cylinders in the POGO and PAM, we model and cancel the nonlinear compressible air flow dynamics for each cylinder and servomechanism valve and use pressure sensors on both sides of the pistons for force feedback [37,39]. This technique allows us to achieve good force or position control. In force-control mode, we can command the PAM or POGO to avoid applying forces to the subject, thus allowing the robots to move along naturally with the patient- (or therapist-) driven movements while measuring those motions. In position-control mode, we can command the PAM or POGO to move the subject's pelvis and legs in a desired pattern with a proportional position controller. If the actual stepping pattern matches the desired pattern, then the robot applies no force, thus contributing toward the assistance-as-needed goal.

We have demonstrated the PAM's ability as a teach-and-replay device [37]. In this experiment, a nondisabled subject walked on the treadmill at different speeds while the PAM operated in a back-drivable mode (zero-force control). The PAM recorded motion capture data for 50 s. We then identified step cycles based on threshold crossings and took an average of the 5 measured DOF of step data over all cycles, which produced a mean pelvic trajectory at each treadmill speed. We then replayed this trajectory against a 45 kg, bungee-suspended punching bag with a stiff proportional-derivative controller that used the nonlinear force control method. The root-mean-square tracking error was less than 1 cm for the left-right component of the desired pelvic trajectory for one period of the gait cycle at three different speeds up to 1.5 m/s.

When we applied this teach-and-replay technique to a nondisabled subject, the subject had difficulty synchronizing his movements with the gait pattern reproduced by the PAM, even though that gait pattern had been recorded from the subject himself. This difficulty was because small variations in gait timing existed, even during normal walking. The PAM allows these variations because it is compliant and allows for the possibility that the subject will move out-of-phase from the robot.

To solve this synchronization problem, we introduced foot switches to detect the gait timing in real time and developed a feedback control algorithm that adjusts the playback speed of the gait pattern [38]. The feedback algorithm measures the error between the desired and actual stepping period and limb phasing and makes small adjustments to the playback speed of the desired gait pattern to compensate for these errors. The algorithm is effective in synchronizing the robotic assistance, even when the nondisabled subjects change their step size and period. Good synchronization substantially reduces the power delivered by the PAM to nondisabled subjects during normal stepping [38]. Thus, feedback-based synchronization appears to contribute to implementing the assist-as-needed principle; i.e., it ensures that the robot intervenes only when the overall stepping pattern is poor, rather than when the timing of an otherwise good pattern is slightly different from the averaged stepping pattern.

We are currently working on other strategies for implementing the assist-as-needed principle, which could be used along with feedback-based synchronization. One possibility is to modulate the impedance of the robot based on the sensed trajectory tracking error, with the adaptive law given in Equation (5). However, the robot cannot be made to act very stiffly because of the natural compressibility of the air used to power it. Another strategy is to modulate the level of force the robot applies based on the trajectory tracking error. We have implemented an interface that achieves this force modulation by allowing the therapist to spatially scale the desired position trajectories of the PAM and POGO. Enlarging the desired trajectories causes the robots to apply more force and create bigger steps but still maintain an appropriate phasing and a "soft" feeling because of the robot's compliance. We plan to explore the possibility of automatically scaling the trajectories based on tracking error with an adaptive law similar to Equation (5).

DISCUSSION AND CONCLUSIONS

We are developing a toolbox of technologies to understand and optimize robotic gait training. Our results with the rat stepper highlight the importance of maintaining appropriate somatosensory input to elicit stepping following SCI. These results are consistent with recent clinical results in patients with SCI [13,21-22]. Initial work with computational models of motor training following SCI suggests that assisting in movement only as needed will be more effective than providing a fixed amount of assistance, because assistance-as-needed can limit stepping errors, while still allowing learning of a novel sensorimotor transformation. If larger stepping errors are tolerable, then amplifying errors may accelerate learning, as reviewed here for stepping experiments with nondisabled subjects [28] and elsewhere for reaching movements by nondisabled and stroke subjects [40-41].

The rationale for providing assistance-as-needed and naturalistic sensory input during gait rehabilitation in individuals with SCI still remains to be validated experimentally. A useful tool for testing this rationale would be a robotic system that can be moved by the patient with a small resistance but that can also provide large forces when needed to create naturalistic walking movements. Preliminary control performance testing with the PAM and POGO robots reviewed here suggests that these devices meet these criteria. Our initial work toward implementing assistance-as-needed with these robots has included using a compliant proportional position controller and automatically synchronizing the robot to the subject based on real-time measurement of foot falls so that the robot "goes along for the ride" if the subject is stepping with a normal pattern, even if the pattern has small variations in timing. To conclude, we discuss several key goals for future research.

We will seek to identify optimal training algorithms using the rat stepper. We are currently testing the hypothesis that assisting only as needed produces better outcomes than fixed assistance. Experimental results with robotic training of spinal-transected mice support this hypothesis [42].

Using the instrumented orthosis that measures the forces and motion applied by human therapists, we will identify the impedance control and timing features associated with eliciting better stepping. Our goal is to codify these features into control software for the PAM and POGO. The PAM and POGO are amenable to human-like control algorithms because they are strong yet inherently compliant because of their use of air power.

For the motor adaptation and modeling studies, we intend to assess to what extent individuals with varying severities of SCI adapt using error-based internal-model formation. Another key question is whether error-amplification techniques can enhance motor adaptation. We will also refine our computational models to provide theoretical guidance for developing training algorithms and to make our assumptions and hypotheses mathematically concrete.

Finally, we have begun preliminary testing of the PAM and POGO in subjects with SCI. Initial results with five subjects with a range of injury severities suggest that the PAM and POGO can adequately replay a pattern of stepping measured with the passive robot as therapists manually stepped the subjects. During replay, no human therapist was required at the pelvis, and the therapists at the legs assisted with greatly reduced effort, only guiding foot placement and preventing foot drop. Video of the robot with a subject with SCI can be found at http://www.eng.uci.edu/~dreinken/first_clinical_testing_of_gait_robots.htm. We are working now to determine whether the pattern of muscle activity that the PAM and POGO induce is similar to that elicited by experienced human therapists. We will then determine whether providing naturalistic gait mechanics, appropriate sensory input, and adjustable levels of well-timed assistance produces better training outcomes than devices that do not have these features, as quantified by the rate and extent of recovery.

ACKNOWLEDGMENTS

Susan J. Harkema is now with the Department of Neurological Surgery, University of Louisville, Frazier Rehabilitation Institute, Louisville, Kentucky.

This material was based on work supported by the National Institute of Standards and Technology, Advanced Technology Program (grant ATP 00-00-4906); The National Institute on Disability and Rehabilitation Research (grant H133E020724); the National Institutes of Health (NIH) (grants R01NS42951 and NO1NS3235); and the NIH National Center for Research Resources (grant M01RR00827).

The authors have declared that no competing interests exist.

REFERENCES
1. Barbeau H. Locomotor training in neurorehabilitation: Emerging rehabilitation concepts. Neurorehabil Neural Repair. 2003;17(1):3-11. [PMID: 12645440]
2. Hesse S, Werner C. Partial body weight supported treadmill training for gait recovery following stroke. Adv Neurol. 2003;92:423-28. [PMID: 12760209]
3. Dietz V, Harkema SJ. Locomotor activity in spinal cord-injured persons. J Appl Physiol. 2004;96(5):1954-60.
[PMID: 15075315]
4. Edgerton VR, De Leon RD, Harkema SJ, Hodgson JA, London N, Reinkensmeyer DJ, Roy RR, Talmadge RJ, Tillakaratne NJ, Timoszyk W, Tobin A. Retraining the injured spinal cord. J Physiol. 2001;533(Pt 1):15-22.
[PMID: 11351008]
5. Colombo G, Joerg M, Schreier R, Dietz V. Treadmill training of paraplegic patients using a robotic orthosis. J Rehabil Res Dev. 2000;37(6):693-700. [PMID: 11321005]
6. Hesse S, Uhlenbrock D. A mechanized gait trainer for restoration of gait. J Rehabil Res Dev. 2000;37(6):701-8.
[PMID: 11321006]
7. Galvez JA, Reinkensmeyer DJ. Robotics for gait training after spinal cord injury. Top Spinal Cord Inj Rehabil. 2006; 11(2):18-33.
8. Hidler JM, Wall AE. Alterations in muscle activation patterns during robotic-assisted walking. Clin Biomech (Bristol, Avon). 2005;20(2):184-93. [PMID: 15621324]
9. Edgerton VR, Tillakaratne NJ, Bigbee AJ, De Leon RD, Roy RR. Plasticity of the spinal neural circuitry after injury. Annu Rev Neurosci. 2004;27:145-67. [PMID: 15217329]
10. Nessler JA, Timoszyk W, Merlo M, Emken JL, Minakata K, Roy RR, De Leon RD, Edgerton VR, Reinkensmeyer DJ. A robotic device for studying rodent locomotion after spinal cord injury. IEEE Trans Neural Syst Rehabil Eng. 2005;13(4):497-506. [PMID: 16425832]
11. Fong AJ, Cai LL, Otoshi CK, Reinkensmeyer DJ, Burdick JW, Roy RR, Edgerton VR. Spinal cord-transected mice learn to step in response to quipazine treatment and robotic training. J Neurosci. 2005;25(50):11738-47.
[PMID: 16354932]
12. Timoszyk WK, De Leon RD, London N, Joynes R, Minakata K, Edgerton VR, Reinkensmeyer DJ. Comparison of virtual and physical treadmill environments for training stepping after spinal cord injury. Robotica. 2003;21(1):25-32.
13. Harkema SJ, Hurley SJ, Patel UK, Requejo PS, Dobkin BH, Edgerton VR. Human lumbosacral spinal cord interprets loading during stepping. J Neurophysiol. 1997;77(2): 797-811. [PMID: 9065851]
14. Timoszyk WK, Nessler JA, Acosta C, Roy RR, Edgerton VR, Reinkensmeyer DJ, De Leon RD. Hindlimb loading determines stepping quantity and quality following spinal cord transection. Brain Res. 2005;1050(1-2):180-89.
[PMID: 15979592]
15. Dietz V. Do human bipeds use quadrupedal coordination? Trends Neurosci. 2002;25(9):462-67. [PMID: 12183207]
16. Timoszyk WK, De Leon RD, London N, Roy RR, Edgerton VR, Reinkensmeyer DJ. The rat lumbosacral spinal cord adapts to robotic loading applied during stance. J Neurophysiol. 2002;88(6):3108-17. [PMID: 12466434]
17. Nessler JA, Reinkensmeyer DJ, Sharp K, Kwak E, Minakata K, De Leon RD. Robotic assessment of locomotor recovery in spinal contused rats. In: Proceedings of the 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society; 2004 Sep 1-5; San Francisco, CA. New York: IEEE; 2004. p. 2687-90.
18. Nessler JA, Minakata K, Sharp K, Reinkensmeyer DJ. Gait activity depends on limb extension and phasing in spinal cord contused rodents: Implications for robotic gait training and assessment. In: Proceedings of the IEEE 9th International Conference on Rehabilitation Robotics: Frontiers of the Human-Machine Interface; 2005 Jun 28-Jul 1; Chicago, IL. New York: IEEE. 2005. p. 556-60.
19. Duysens J, Clarac F, Cruse H. Load-regulating mechanisms in gait and posture: Comparative aspects. Physiol Rev. 2000;80(1):83-133. [PMID: 10617766]
20. Pearson KG , Misiaszek JE, Fouad K. Enhancement and resetting of locomotor activity by muscle afferents. Ann N Y Acad Sci. 1998;860:203-15. [PMID: 9928313]
21. Kawashima N, Nozaki D, Abe MO, Akai M, Nakazawa K. Alternate leg movement amplifies locomotor-like muscle activity in spinal cord injured persons. J Neurophysiol. 2005;93(2):777-85. [PMID: 15385590]
22. Ferris DP, Gordon KE, Beres-Jones JA, Harkema SJ. Muscle activation during unilateral stepping occurs in the nonstepping limb of humans with clinically complete spinal cord injury. Spinal Cord. 2004;42(1):14-23.
[PMID: 14713939]
23. Galvez JA, Kerdanyan G, Maneekobkunwong S, Weber R, Scott M, Harkema SJ, Reinkensmeyer DJ. Measuring human trainers' skill for the design of better robot control algorithms for gait training after spinal cord injury. In: Proceedings of the IEEE 9th International Conference on Rehabilitation Robotics: Frontiers of the Human-Machine Interface; 2005 Jun 28-Jul 1; Chicago, IL. New York: IEEE; 2005. p. 231-34.
24. Emken J, Wynne J, Harkema SJ, Reinkensmeyer DJ. A robotic device for manipulating human stepping. IEEE Trans Rob. 2006;22(1):185-89.
25. Shadmehr R, Mussa-Ivaldi FA. Adaptive representation of dynamics during learning of a motor task. J Neurosci. 1994;14(5 Pt 2):3208-24. [PMID: 8182467]
26. Reinkensmeyer DJ, Emken JL, Cramer SC. Robotics, motor learning, and neurologic recovery. Annu Rev Biomed Eng. 2004;6:497-525. [PMID: 15255778]
27. Emken JL, Reinkensmeyer DJ. Evidence for an internal model dedicated to locomotor control. In: Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society; 2003 Sep 17-21; Cancun, Mexico. New York: IEEE; 2003. p. 1503-6.
28. Emken JL, Reinkensmeyer DJ. Robot-enhanced motor learning: Accelerating internal model formation during locomotion by transient dynamic amplification. IEEE Trans Neural Syst Rehabil Eng. 2005;13(1):33-39.
[PMID: 15813404]
29. Liu J, Reinkensmeyer DJ. Motor adaptation as an optimal combination of computational strategies. In: Proceedings of the 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society; 2004 Sep 1-5; San Francisco, CA. New York: IEEE; 2005. p. 4025-28.
30. Reinkensmeyer DJ, Liu J, Emken JL, Bobrow JE. The nervous system appears to minimize a weighted sum of kinematic error, force, and change in force when adapting to viscous environments during reaching and stepping. In: Advances in Computational Motor Control III Symposium; 2004 Oct 22; San Diego, CA. Washington (DC): Society for Neuroscience; 2004.
31. Reinkensmeyer DJ. How to retrain movement after neurologic injury: A computational rationale for incorporating robot (or therapist) assistance. In: Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society; 2003 Sep 17-21; Cancun, Mexico. New York: IEEE; 2003. p. 1479-82.
32. Reinkensmeyer DJ, Aoyagi D, Emken JL, Galvez J, Ichinose W, Kerdanyan G, Nessler J, Maneekobkunwong S, Timoszyk B, Vallance K, Weber R, De Leon RD, Bobrow JE, Harkema SJ, Wynne J, Edgerton V. Robotic gait training: Toward more natural movements and optimal training algorithms. In: Proceedings of the 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society; 2004 Sep 1-5; San Francisco, CA. New York: IEEE; 2004. p. 4818-21.
33. Emken JL, Bobrow JE, Reinkensmeyer DJ. Robotic movement training as an optimization problem: Designing a controller that assists only as needed. In: Proceedings of the IEEE 9th International Conference on Rehabilitation Robotics: Frontiers of the Human-Machine Interface; 2005 Jun 28-Jul 1; Chicago, IL. New York: IEEE; 2005. p. 307-12.
34. Inman VT, Ralston HJ, Todd F. Human walking. Baltimore (MD): Williams & Wilkins; 1981. p. 41-55.
35. Wang CY, Bobrow JE, Reinkensmeyer DJ. Dynamic motion planning for the design of robotic gait rehabilitation. J Biomech Eng. 2005;127(4):672-79. [PMID: 16121538]
36. Ichinose WE, Reinkensmeyer DJ, Aoyagi D, Lin JT, Ngai K, Edgerton VR, Harkema SJ, Bobrow JE. A robotic device for measuring and controlling pelvic motion during locomotor rehabilitation. In: Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society; 2003 Sep 17-21; Cancun, Mexico. New York: IEEE; 2003. p. 1690-93.
37. Aoyagi D, Ichinose W, Reinkensmeyer DJ, Bobrow JE. Human step rehabilitation using a robot attached to the pelvis. In: Proceedings of the 2004 American Society of Mechanical Engineers (ASME) International Mechanical Engineering Congress and Research Development and Design Exposition; 2004 Nov 13-19; Anaheim, CA. New York: ASME; 2004. p. IMECE2004-59472.
38. Aoyagi D, Ichinose WE, Harkema SJ, Reinkensmeyer DJ, Bobrow JE. An assistive robotic device that can synchronize to the pelvic motion during human gait training. In: Proceedings of the IEEE 9th International Conference on Rehabilitation Robotics: Frontiers of the Human-Machine Interface; 2005 Jun 28-Jul 1; Chicago, IL. New York: IEEE; 2005. p. 565-68.
39. Bobrow JE, McDonell BW. Modeling, identification, and control of a pneumatically actuated, force controllable robot. IEEE Trans Rob Autom. 1998;14(5):732-42.
40. Patton JL, Stoykov ME, Kovic M, Mussa-Ivaldi FA. Evaluation of robotic training forces that either enhance or reduce error in chronic hemiparetic stroke survivors. Exp Brain Res. 2005;168(3):368-83. [PMID: 16249912]
41. Wei Y, Bajaj P, Scheidt R, Patton JL. Visual error augmentation for enhancing motor learning and rehabilitative relearning. In: Proceedings of the IEEE 9th International Conference on Rehabilitation Robotics: Frontiers of the Human-Machine Interface; 2005 Jun 28-Jul 1; Chicago, IL. New York: IEEE; 2005. p. 505-10.
42. Cai LL, Fong AJ, Otoshi CK, Liang Y, Burdick JW, Roy RR, Edgerton VR. Implications of assist-as-needed robotic step training after a complete spinal cord injury on intrinsic strategies of motor learning. J Neurosci. 2006;26(41): 10564-68. [PMID: 17035542]
Submitted for publication April 21, 2005. Accepted in revised form December 19, 2005.

Go to TOP  

Go to the Contents of Vol. 43 No. 5

Last Reviewed or Updated  Tuesday, May 29, 2007 11:49 AM