VA Research and Development LOGO

Logo for the Journal of Rehab R&D
Vol. 37 No. 6, November/December 2000


User involvement in the design and evaluation of a smart mobility aid

Gerard Lacey, BAI, MA, PhD and Shane MacNamara, BA

Trinity College, Department of Computer Science, Dublin 2, Ireland; Vartry Research Ltd., Wicklow Enterprise Park, Wicklow Town, Ireland

Abstract — This paper describes the design and evaluation of an innovative smart mobility aid for the frail visually impaired. The Personal Adaptive Mobility AID (PAM-AID) was developed to address the difficulties in personal mobility of the frail and elderly visually impaired. The paper provides an overview of the PAM-AID research at Trinity College and describes the evolutionary nature of the design process. Because there were no existing systems to guide its development, a series of prototypes was constructed and they were regularly evaluated in the field. This approach views potential users as vital contributing members of the design team and led to rapid and hopefully useful improvements in the design.

Key words: elderly, mobility aid, robot mobility aids, visual impairment.


This material is based on work supported by the National Rehabilitation Board, the European Union Telematics Applications Project DE3210, and the Trinity Foundation.
Address all correspondence and requests for reprints to: Gerard Lacey, Vartry Research Ltd., Wicklow Enterprise Centre, Wicklow Town, Ireland; email: gerard.lacey@vartry.com

INTRODUCTION

   The opportunity to be independently mobile is a key factor in the quality of life of everyone. For the visually impaired person, mobility may be improved by using a long cane or a guide dog. However, if the person is frail then these mobility aids can be very difficult or dangerous to use because of the increased risk of falls and collisions. Without a mobility aid the person can be forced into a lifestyle that is both sedentary and heavily dependent on others. The lack of exercise and independence adversely affects physical and mental health. The combination of visual impairment and frailty occurs most often among the elderly, who make up 75 percent of the blind population.

   The Personal Adaptive Mobility AID (PAM-AID) project began in 1994 with the aim of developing a mobility aid to specifically address the needs of the frail blind. Current mobility aids for the visually impaired do not provide both physical support and environmental information to the user. These deficiencies force the frail elderly into a sedentary lifestyle with all the deleterious consequences that ensue. The aim of the PAM-AID project was to maintain the ability of the frail visually impaired to take exercise safely and independently.

   In the United States visual impairment affects 18.1 percent, or 3.6 million people aged 70 and older (1). The visually impaired elderly are twice as likely to report associated difficulty walking (43.3 percent versus 20.2 percent), to have experienced falls in the previous 12 months (31.2 percent versus 19.2 percent), and to have broken a hip (7.1 percent versus 4.2 percent; reference 1). They also report significant additional difficulties in ADL tasks over the sighted elderly (1). In addition, the elderly visually impaired are more likely to suffer hypertension (53.7 percent versus 43.1 percent), heart disease (30.2 percent versus 19.7 percent), stroke (17.4 percent versus 7.3 percent), and depression and anxiety (13.3 percent versus 7 percent; reference 1).

   In Europe at least 65 percent of all visually impaired people are aged over 70 (2); some studies estimate this to be as high as 90 percent (3) due to underreporting by the elderly. A similar range is quoted for the United States (4). Demographic trends show that 25 percent of the European population will be aged over 60 by 2020. In the United States, the percentage of those in the population over age 65 will increase from its current level of 12.5 percent to 20 percent by 2030 (5). These surveys also show that the largest increase will be in the number of people aged over 75, in whom disability is most common.

   To summarize, the elderly represent the majority of visually impaired people and show a significant deterioration in their activity levels and independence. The direct correlation of visual impairment with frailty has been noted by Rubin and Salive (6) and suggests that both sense of balance and judgment of moving obstacles undergo a progressive deterioration with age.

   Vision loss in later life can be particularly disabling for those in long-term care. Psychological problems associated with lack of motivation and lessened expectations make mobility training difficult (4). This difficulty is compounded by memory loss, the need for support during walking, and an increased fear of falling. If a cane is used for both support and mobility it can be quite heavy and its use can lead to fatigue. Using a long cane in tandem with a walking aid results in both hands being occupied and thus increases the fear of falling. In long-term care facilities practical concerns discourage independent mobility for the aged visually impaired. Many of the visually impaired in residential care will have lost their sight within the previous few years. Consequently, they have had little experience in using canes as mobility aids, and as a result they often swing the cane with too much force and at dangerous heights, causing them to be viewed as a danger to other residents.

   The difficulties of providing the elderly visually impaired with independent mobility can result in their being confined to beds or to chairs, supposedly for their own safety. In this sedentary state a rapid deterioration in the cardiopulmonary systems occurs. The link between inactivity and the deterioration of health in older persons has been noted (7). The psychological effect of increased dependence also has an adverse effect on the person's quality of life. Even limited independent mobility can greatly increase the quality of life of the elderly.

THE PAM-AID PROJECT

   The consequences of visual impairment and frailty provided the motivation for the development of the PAM-AID. The PAM-AID project began in 1994 in the Department of Computer Science, Trinity College, Dublin, Ireland. From the outset of the project potential users were consulted regularly through their representative organization, the National Council for the Blind of Ireland (NCBI). The first phase of the project involved an assessment of user needs by interviewing potential users, mobility trainers, and caregivers. During the interviews the proposed solution of "a robot walking frame" was described along with a range of potential functionality. Interviewees were encouraged to comment on the features of the design and the limitations they could foresee, and to propose alternative designs. During 1995 the Irish government, through the National Rehabilitation Board, sponsored the PAM-AID research. Between January 1997 and June 1999 PAM-AID became a European Union (EU) project with six partners. The partners and their roles are listed in Table 1. In July 1999 one of the PAM-AID prototypes constructed at Trinity College won the Paralyzed Veterans of America (PVA) Student Design award at RESNA.


Table 1.
Partners in the EU project 1996-1999.
EU Project Partner
Role
Trinity College, Dublin, Ireland
System Design and
Construction
Zenon SA, Greece
Project Management
Euroflex, Sweden
Mechanical Construction
National Council for the Blind of Ireland
User Representatives
Hertfordshire University, United Kingdom
System Evaluation
Chalmers University of Technology,
Sweden
System Evaluation

   The PAM-AID research is continuing in a new startup company, Vartry Research, formed by the authors. A new research project to measure the clinical outcomes of PAM-AID has been undertaken between Vartry Research and the VA centers in Pittsburgh and Atlanta.

   Over the course of the PAM-AID project, a variety of prototypes were constructed and evaluated. At all times the designers tried to involve the users within the design process. The remainder of this paper will provide an overview of how the design of PAM-AID evolved and the role of the users in shaping that design.

User Involvement in Design
   Key to the success of the PAM-AID project has been the involvement of potential users at all stages of the development of the device, and the structure of the project team. The EU project team was evenly divided into independent teams: a human factors team and an engineering team. The human factors team managed user trials and conducted the majority of the interviews with the users. The independence of the teams facilitated the unbiased evaluation of the PAM-AID devices within the project and ensured that the design remained focused on the user's needs.

   Communication between the engineering team, the human factors team, and the users was a critical issue throughout the project. Early in the project it was clear that all participants in the project would have to revisit the definition of the smart mobility aid regularly. Initially, the users and the human factors team found the PAM-AID concept difficult. However, as time progressed the concept became clearer, helped particularly by the production of prototypes. The regular revisiting of the definition of PAM-AID greatly improved the engineering team's grasp of both the user's needs and the constraints imposed by the user's environment. At the beginning of each meeting (held every 4 to 6 months) the engineering team updated the human factors team on the latest changes and improvements to the design. These design features were then communicated to the users by the human factors team during field trials.

   The engineering team was present during all the field trials to provide technical support and to observe the performance of the devices. Due to their presence at the field trials the engineering team gained an appreciation for several important factors of the design:

At the end of field trials the engineering team was able to discuss new design ideas with users and with the human factors team. These proved to be a very useful source of design ideas, with ideas being contributed by all participants.

Determination of User Requirements
   The aim of the user requirements study was to provide the users and caregivers with a means of contributing to the PAM-AID design process. Analysis of the user needs, the user profile, and understanding the limitations of the technology produced the initial specification of the device. Many of the comments from potential users were given to the first author at the pre-prototype stage and were at times self-contradictory because of the difficulty potential users had in imagining the proposed solution. The interviews were conducted in a free-form fashion with a list of open-ended questions. The aim was to encourage potential users and their caregivers to provide the researchers with their priority specifications for the device design. During the course of the interview it became clear that many of the potential users had imagined many different things based on their (often quite limited) experience of computer technology. One notable example was when a subject reported surprise that the device was not "a tin man," as expected.

   Care was taken to design the device to meet the needs of a wider population rather than just those of the user panel initially interviewed. This was achieved by consulting professionals that provide mobility and medical services to the elderly and blind and by subsequent user interviews at the prototype stage.

   The majority of the user's interest was focused on the user interface. The users wanted something intuitive such as switch input or a voice interface. The problem of arthritis preventing the use of simple switches was very prominent in the user needs survey. Ideally what was required was a compliant interface similar to that provided by a guide dog augmented by a small number of voice commands. To facilitate training and ease of use the cognitive load must be kept as low as possible. Additionally, to keep stress to a minimum, the user should feel in control of the robot at all times.

   The user interview phase of the project gathered much valuable information on the general design and specification of the PAM-AID device. The general outline was for a device that the users could direct easily and that would provide them feedback on surrounding landmarks. Additional information was also gathered on the size, speed, and features that users desired to see in the device.

System Evaluations
   There is currently no widely accepted theory of system design that will predict the acceptability of a complex mechatronic system in a real world environment. The most reliable method at present is the construction and evaluation of prototypes. Where possible the field trials should be conducted in the target environment to ensure that all assumptions are tested. Engelhardt and Edwards advocate the use of an Interactive Evaluation strategy in the development of assistive technology (8). By this they mean that potential users should be consulted early and often during the design process. This research has endeavored to adopt such a strategy; however, user trials and surveys were carried out only when there was a need for new information, e.g., the testing of new interface configurations. In addition, due to the frailty of the user population, it was not possible to run exhaustive tests in which one user could compare multiple configurations of the device. Instead different configurations were tried with different users; this rule was only relaxed when users themselves requested an alternative configuration. The user trials in all cases were carried out with the approval of the Ethics Committee of the University of Hertfordshire.

   The number of subjects necessary to identify the usability problems is a topic of great importance. Several studies (9,10) show that 80 percent of usability problems will be identified by three to five subjects, with each additional participant adding less new information. In the course of these evaluations the number of subjects varied as a result of subject availability at the time of user trials. The number of participants in each trial is summarized in Table 2.


Table 2.
Trials of the PAM-AID smart mobility aid.
Trial Date Location No. Users Purpose
January 1997 Dublin,
Ireland
4 Evaluation of the Concept
Prototype
June 1997 Hatfield,
England
8 Trials of the Rapid Prototype
November 1997 Dublin,
Ireland
16 Trials of the Active
Demonstrator
February 1998 Hatfield,
England
7 Trials of the Active
Demonstrator with
Parkinson's Patients
September 1998 Dublin,
Ireland
8 Trials of the Passive
Demonstrator
February 1999 Gothenberg,
Sweden
  Trials of the Passive
Demonstrator
April 1999 Dublin,
Ireland
5 Trials of the Active and
Passive Demonstrators

   Each field trial evaluated a new configuration of the PAM-AID device or involved users having different sensory or physical disabilities. The evaluations undertaken during this research have focused primarily on different forms of user acceptability. Alternative performance measures as in Simpson (11), such as average speed through an obstacle course, average time taken, et cetera, were not suitable, as these tests focus on the outcomes of device use rather than the acceptability of device features. Users were asked to walk with the devices, and their opinions were sought on the acceptability and performance of various device features. The responses were ranked on a five-point Likert scale (16).

Concept Prototype
   The first device to be constructed was the Concept Prototype. Domain experts at Trinity College carried out the evaluation of the Concept Prototype, as it was felt that the device was too large to undergo a full field trail in a residential home. This evaluation allowed the potential users and the human factors team to get "hands-on" experience with the device. It also allowed many technical aspects of the design to be explored. The PAM-AID Concept Prototype was completed and evaluated in the laboratory by user representatives in October 1996 and again in January 1997 (12). The system, shown in Figure 1, consisted of a LabmateTM robot base to which a handrail was fitted. The user input was captured by means of a joystick and two switches. The user feedback was by means of recorded voice messages and tones. Sonar and infrared sensors were used to capture information required for robot navigation. The control system had two modes: The first mode allowed the user to control the robot direction but provided warnings about potential collisions. In the second mode the robot performed automatic wall following. The user selected between modes using a switch, and the device stopped if no input of any kind was received from the user.


Figure 1. A photograph of the PAM-AID prototype
Figure 1.
Concept prototype.

   The evaluation resulted in the recommendation of the following design changes:

The Rapid Prototype
   The comments from the domain experts were incorporated into the design of the second device, the Rapid Prototype (13). The main design change was in mechanical design; a conventional walker or rollator was fitted with motors, sensors, and a controller, as was used as a test bed for the design. The Rapid Prototype is shown in Figure 2. Two user-input options were developed to replace the joystick. The first was instrumented handles that detected small movements in the handles. They could detect if they were being pushed or pulled, or were at rest. The second interface was composed of simple switches, one each for forward, backward, left, and right. The sensing and control architecture was largely identical to that of the previous prototype. There were two user feedback options, namely speech messages and musical tones.


Figure 2. A photograph of the PAM-AID rapid prototype
Figure 2.
Rapid prototype.

   The evaluation was carried out with the help of eight visually impaired people, who ranged in age from 76 to 90 years old with a mean age of 85. The trials took place in two nursing homes in Hertfordshire, United Kingdom. The results of the trials were a set of 31 recommendations, 22 of which related to the design of the device. The Sensory Disability Research Unit (SDRU) of Hertfordshire University oversaw the evaluation of the Rapid Prototype. To prevent fatigue among the subjects it was decided to test two input-output configurations of the user interface. The configurations chosen were instrumented handles with speech feedback and switch input with tonal feedback. The procedure for the trials was as follows: PAM-AID and its basic functionality were explained to each subject. Then each subject walked with the device, first in automatic mode and then in manual mode. After using the device, subjects were asked questions from two questionnaires: a device evaluation questionnaire and a technology acceptance questionnaire.

   The evaluation highlighted some deficiencies in the rapid prototype design and enabled the users to contribute design ideas. During the user requirements phase researchers from the SDRU had interviewed the subjects. However, until they had direct experience with the device, the PAM-AID concept was unclear and interpreted subjectively. Such confusion made the early contributions to the user requirements difficult to interpret reliably. This can be contrasted sharply with the user contributions during the trials when the users made several important design suggestions--a bicycle handlebar input device, new switch design, et cetera. In addition, they stressed that transfer from chair to PAM-AID was an important issue for them.

   After the field trials of the Rapid Prototype a number of design options for the smart mobility aid were identified. These options relate to whether the device is self powered, as in an electric wheelchair, or user powered, as in a manual wheelchair. The options also relate to whether the device is providing only information to the user or is steering around objects independent of the user. These options are summarized in Table 3.


Table 3.
Design options for PAM-AID.
  Traction Control User Interface  
1 User Pushed Warning only This is a walker fitted with the
sensor system and user interface.
Warnings are given about the
proximity and location of obstacles.
2 User Pushed Warning, plus
automatic
braking
Similar to the above option except
that the system is able to brake
before a collision occurs. The user
remains in complete control over
direction and speed.
3 User Pushed Obstacle
Avoidance and
Warning system
In this configuration steered
wheel(s) sets the device direction.
The user remains in complete
control over speed however
direction control is shared with the
robot controller.
4 Self Propelled Warning, plus
automatic
braking
In this configuration motors
compensate for the weight of the
device and can gently pull the user
along if needed. The user has
complete control over direction and
motion of the robot, communicated
via the user interface. However, if
there is a danger of a collision the
robot will stop.
5 Self Propelled Obstacle
Avoidance and
Warning system
This is similar to the configuration
described above except that the user
shares control over direction with
the robot controller.

   The next phase in the development of PAM-AID pursued the two options, a self-propelled version and a user-pushed version. The self-propelled option built on the development of the Rapid Prototype and was called the Active Demonstrator. The user-pushed version required a complete mechanical redesign in a version called the Passive Demonstrator. During their development, a number of user trials were undertaken to validate various aspects of the design. To aid readability and discussion, the results of the evaluations will be amalgamated for each robot.

The Active Demonstrator
   Following the outcome of the Rapid Prototype evaluations, many of the recommendations were either incorporated into the subsequent versions of the PAM-AID or were investigated and subsequently rejected. For example, the force-sensing handles were built but ultimately rejected due to the their complexity both mechanically and as a reliable user-input modality. The bicycle handlebars idea was developed and integrated into the user-pushed version of the PAM-AID robot. The buttons on the robot were changed to high-profile, tactile buttons and the spatial layout was changed to that suggested by the users: left and right on one hand and forward and reverse on the other. During the user trial the cognitive load on users was noted and judged to be too high. Users were being asked to evaluate too many different features of the user interface and the robot in general. It was decided that for the next set of trials the number of features in the evaluations would be reduced.

   The self-propelled version was investigated for a number of reasons: The powered traction compensated for the weight of the device itself, thus very frail individuals could use it easily. As the device is self powered it can also operate autonomously, stowing itself while not in use, driving between a number of users who share it, or operating a fetch-and-carry service within a residential care setting. The powered traction does present safety issues in that care must be taken not to upset the user's balance and the user must at all times feel in control of the device. The device was operated at all times in a fail-safe mode; the device would only move if the user pushed a direction switch and would immediately stop if the switch were released. However, this did not prevent the user from making an input error, e.g., confusing left and right or turning too far when negotiating a junction. To solve these problems a context-sensitive user-input system was developed (14). This system ensured that the user input was valid for the current environment before passing it to the device controller. The self-propelled system was field tested several times during its development to ensure that the design remained true to the users' needs and to facilitate the users' involvement in the design process.

Active Demonstrator Evaluations
   The Active Demonstrator (Figure 3) has had three field trials to date, November 1997, February 1998, and April 1999. The first trial was undertaken in two locations in Dublin. Sixteen people, ranging in age from 55 to 94 years old with a mean age of 77, tested the device. In this first trial, major elements of the rapid prototype were directly transferred to the Active Prototype. The trials aimed to investigate the new switch design, the hoist handles, a new mechanical design, and to involve a new set of users in the design process.


Figure 3. A photograph of the Active Demonstrator
Figure 3.
Active Demonstrator.

   The robot was evaluated in the corridors and rooms of two different nursing homes. The user interface was configured with switch input. The audio feedback was disabled in an effort to reduce the cognitive load on the users. The trials were conducted using two modes, manual user control and automatic wall following. In this trial the users made comments that were condensed into a set of general recommendations.

The users also commented on the things they liked about the PAM-AID.

   The second user trials were carried out in the United Kingdom with seven patients with Parkinsons disease. The main aim of the trial was to assess the potential use of the PAM-AID with Parkinsons disease patients. An automatic voice recognition input device was also evaluated. The trial was carried out under the supervision of the SDRU using the PAM-AID active prototype as described above with the addition of a voice interface. Voice output messages provided warnings of objects to the left, right or directly ahead. Voice input was available in the form of commands forward, left, right, backward, stop, and automatic.

   The obstacle-avoidance functionality of the device was of limited use to the majority of the subjects in this trial as most were not visually impaired and collisions did not appear to be a major cause of concern. The users' interest was focused on the potential for PAM-AID to assist them in recovering from freezing caused by their Parkinsons. The aim was to gently pull them forward and initiate a balance reflex that would restart a normal walking pattern.

   In addition to switch input the users also evaluated a voice-input system. The voice-input device used was a Verbex Speech Commander, which needs to be trained on the voice of the person using it. The training procedure involved the users being prompted to say words as they appeared on the screen. Training times were typically between 20 and 45 minutes. Difficulties arose due to the difficulty some elderly people have in maintaining consistent speech quality.

   The subjects were trained in the use of voice commands for the control of the robot. The users then walked up and down the corridor using voice input. The trials resulted in the following set of recommendations.

   The voice-input system exhibited the classic types of error found in voice recognition, i.e., deletion, substitution, insertion, and rejection. The training and recognition of the voice interface had used strict matching criteria for safety. However, this resulted in a large number of deletion and rejection errors. These errors are consistent with no input, in which case the device simply stopped. These errors were primarily due to the difficulty in maintaining consistent tone and volume when speaking the commands.

   Despite the problems experienced by the recognition and the sometimes long training periods, the subjects strongly preferred the voice interaction to switch interaction. This was despite the fact that they were required to wear a headphone and microphone. It was noted, however, that the time between input commands in voice interaction was of the order of five seconds. It was felt that this high rate of interaction would have to be significantly reduced to make the device acceptable over a longer period. With regard to assisting the user in initiating motion, it was found that it was not as simple as expected. The users tried to resist the motion of the robot. More trials are necessary to understand the interaction to any extent. Further work with this particular user group was beyond the scope of the project.

   The final field trial in April 1999 evaluated new control strategies in the Active Demonstrator. The sensors were changed to include a laser range finder, with sonar acting as a backup. The accuracy of the laser allowed the development of a feature-extraction system that determined the corridor type. This new information was used in two ways: It produced voice messages for the landmarks, and it was used in the context-sensitive reasoning system.

   The final user trials were conducted in Dublin in April 1999. Five subjects, all female residents, tested the active prototype by navigating the corridor circuit in which residents take exercise. The switch user interface was used throughout this trial. The Active Demonstrator was used in two configurations, fixed and adaptive shared control. The subjects tested the device in each mode before answering a questionnaire. The order in which the operating modes were presented was alternated between subjects. The voice messages describing the features of the environment were presented to the subjects throughout the trials.

   The subjects were aged 67 to 95 with a mean age of 82. All subjects were partially sighted; the severity of vision loss ranged from good residual vision in one eye to very poor vision. The mobility of the subjects ranged from good independent mobility to normally using a wheelchair. The severity of both visual impairment and mobility impairment was correlated roughly with the ages of the subjects. The subjects navigated corridors that involved several turns and obstacle-avoidance maneuvers, thus ensuring that the functions of the interface were evaluated thoroughly. The questionnaire was administered after subjects had tested the device in both modes and had performed a circuit of the exercise area.

   The evaluation aimed to test two forms of shared control, Fixed Shared Control and Adaptive Shared Control. The fixed shared control system used a combination of the user input and the range data from a laser scanner to guide the user on the safe path. This guidance took no account of the type of corridor, et cetera, to change the way in which the device operated or to validate the user inputs. The adaptive shared control system performed two functions; it checked to see that the user input was "in context" and it provided different types of assistance based on the situation. Out-of-context input could be, for example, if the user pressed the right turn switch in a corridor where no right turn was possible. The device could respond by ignoring this potentially erroneous command and issuing a warning voice message. The adaptive shared control system also helped the user negotiate junctions by turning the device only until they were pointing straight down the corridor and then issuing a voice message. This behavior was achieved without the use of maps and purely by analyzing the features in the environment around the robot.

   In the fixed shared control system the switch interface allowed subjects to set the goal points for the robot guidance system. The switches used were forward, left, and right. The reverse switch was disabled due to the stress this had caused subjects in earlier evaluations. The fixed shared control system required users to determine the heading for the robot based on their residual vision and on the information provided via the voice messages. The mean ratings on a scale of 0 to 5 (0 is poor, 5 is excellent) are summarized in Table 4.


Table 4.
Ratings for fixed shared control.
 
Mean Rating
Standard Deviation
Overall Usability
3.8
0.4
Physical Support
5
0
Learnability
4.2
0.74
Switch Usability
4.3
0.4
Ease of Remembering
4.2
0.4

   In addition to the prepared questionnaire, caregivers and subjects gave comments and recommendations to the experimenter. These were particularly useful as a means of user involvement in the design process. Some comments also highlight the factors that influence the users' stated preferences. For example, the oldest subject (95 years old) said she was very nervous at the beginning, but after a short period of time she calmed down and began walking with a very natural gait. The senior nurse noted that this subject was walking with a much-improved gait. The younger subjects, 67 and 81 years old, respectively, were more mobile and had better residual vision. They suggested that the switches be more brightly colored and have different shapes. The robot sometimes took corners too wide for their liking and they used short bursts of the left/right turn switches to adjust their trajectory. They primarily used their residual vision to navigate the robot. The final subject had significant arthritis and found keeping the switches pressed difficult.

   All the subjects used the adaptive shared control mode. The experimenter began by explaining the function of the mode and the meaning of the warning/explanation voice messages. If the user input was deemed to be out of context by the control system, the robot issued a warning message and stopped. An example of "out-of-context" input was if the Left or Right switch was pressed while in a straight corridor, or if the Forward switch was pressed while at a T-junction.

   To compare the assistive mode with the manual mode, subjects were asked if they found it helpful when the device noticed erroneous input. Subjects gave a positive mean rating of 4, (i.e., quite helpful) with a standard deviation of 0.6. This question prompted a lot of discussion. The subjects with good partial vision found it less useful than those with poor vision. The subjects with good residual vision commented that the robot did not take the "shortest path," i.e., did not cut wide corners as they would have done themselves and prevented them from doing so manually by disabling the switches. Subjects with poor vision found that the assistive mode was particularly useful when turning at the end of corridors. Subjects with poor vision lose their bearings very quickly when turning. The assistive mode stopped the robot from turning when it was pointing straight down the new corridor, thereby preventing overshoot. In the words of one subject there was "no chance of turning too far" while using the assistive mode.

   Subjects were also asked if they felt safer using the device in adaptive mode over the fixed mode. They gave a mean rating of 3.3, (i.e., marginally safer in adaptive) with a standard deviation of 0.5. Many of the comments were that both modes were safe. Those subjects with severe visual impairment commented that they found the assistive mode safer. Some subjects said that in assistive mode they paid more attention to the voice messages, as they now contained more information. This attention was most noticeable when the robot issued a warning and stopped until a safe input was given. A frequent comment was that while subjects found the robot safer in assistive mode, they did not like giving up more control to the robot. Some stated that this was due to nervousness and that they would get used to it over time. Others said they did not need the help and therefore did not want it.

   Voice messages were used as the primary mode of feedback. Feature messages describing the landmarks were produced every six seconds while the landmark remained unchanged. However, if a new landmark appeared the appropriate message was produced immediately. Table 5 gives the mean ratings and standard deviations of questions directed at assessing the utility, content, and comprehension of the voice messages.


Table 5.
Voice message mean ratings.
 
Mean Rating
Standard Deviation
Utility
3.8
0.4
Content
5
0
Comprehension
3.67
0.7

   The most frequent comment was that the messages were too quiet. The corridors were periodically very noisy; radios playing loudly, metal trolleys being pushed around, and loud conversations sometimes drowned out the voice messages. The more visually impaired subjects also mentioned that they would like the naming of specific places such as the dining room, smoking room, et cetera. The caregivers and the NCBI representatives who attended the trials echoed this last point.

   The subjects were asked a number of general questions in order to rate the overall acceptability and usefulness of the device. They were also asked to make suggestions regarding the long-term use of the device in the home. Table 6 summarizes the results of questions regarding the overall safety, usability, the utility of the device in general, and the utility of the device to them personally.


Table 6.
Ratings given to general features.
 
Mean Rating
Standard Deviation
Safety
4.5
0.6
Usability
4.6
0.5
General Utility
4.2
0.4
Personal Utility
4, (4.5)
1.1, (0.57)

   The device was regarded as quite safe and very easy to use. Subjects felt that is would be quite useful in general for other people in the nursing home. When asked if they would use the device themselves the average statement given was that they would find it quite useful. However, if the youngest and most mobile subject is eliminated the mean statement rose to very useful. This is because the youngest subject felt that she did not need a mobility aid and could not foresee herself needing one. Her response may reflect the fact that she was the youngest and most mobile of the subjects, but also she alone among the subjects was partially blind since birth and had developed a range of strategies to compensate for her disability. The other subjects had lost their vision relatively recently (median 3.4 years) and did not have a lifetime's experience of coping with visual impairment.

   Subjects made a range of suggestions relating to the overall use of the device. The most common suggestion was that the device should identify specific places in the nursing home: dining room, smoking room, bedrooms, et cetera. Some subjects wanted the device to tell them where to go and what button to press. The frailest subjects wanted a seat on the device so that they could rest if they got tired. One subject wanted the functionality of the device built into her chair so that she could come and go as she pleased without having to walk. This subject had severe arthritis and tired easily while walking.

   A voice interface was mentioned as a desirable interaction method. In particular, this was mentioned in the context of requesting the device to navigate from point to point in the nursing home. Also, many features of the building were mentioned as obstacles to personal mobility, such as the weight of fire doors, the problem of following handrails while using a walker, and frequent gaps in the handrail due to junctions, doors, and windows.

The Passive Demonstrator
   The second of the systems identified in Table 3 was the user-propelled Passive Demonstrator (Figure 4). Four goals were set for the design of the Passive Demonstrator:

  1. The user should have complete control over the speed of the device in a very intuitive manner.
  2. The system should be intrinsically safe.
  3. The device must be highly maneuverable.
  4. The power requirements and the overall weight of the device were to be reduced compared to the Active Demonstrator.

Having the user push the device and having motorized steering met these goals. The device was made more compact with the user standing inside the frame to a greater extent than before, thus allowing for greater support and a smaller device. The user interface was completely redesigned to incorporate the handlebar design highlighted in the trials of the Rapid Prototype. In the manual mode of operation, the handlebar rotation is converted to a steering angle and the device can be used in the same way as a conventional walker. In automatic mode the input from the handlebars was used as user input to the device navigation system.


Figure 4. A photograph of the Passive Demonstrator
Figure 4.
The Passive Demonstrator.

Passive Demonstrator Evaluations
   The passive demonstrator was evaluated in three separate field trials. The first field trial had 7 male participants with an average age of 82 years. The main aim of this trial was concept evaluation. The two modes of operation--autonomous mode and manual mode--were evaluated. The autonomous mode of the device was simulated in a Wizard-of-Oz style--a member of the evaluation team controlled the steering remotely. It was invaluable to carry out a trial at this early stage as the device's shortcomings could be seen early on in the development. Users enjoyed the increase in maneuverability over the active demonstrator but did request that even more maneuverability would be beneficial in certain circumstances. The caregivers, who had some difficulty positioning the device in front of a user, echoed this. The maneuverability of the device was improved by the addition of a button that, when pressed, would flip the wheels into an orientation where the user could rotate the device on the spot. It was thus possible to get out of very tight situations with the PAM-AID. The different wheel alignments are illustrated in Figures 5a and b.


Figure 5. A diagram of the plan view of the Passive PAM-AID
Figure 5.
Plan view of the Passive PAM-AID illustrating different possible wheel orientations.

   Other comments on the device reinforced our earlier findings. In this particular trial, the device was not yet equipped with audio feedback on the location of features in the environment. Consequently, users were constantly taking one hand off the handlebar to feel for recognizable features. This behavior emphasized the need for good feature recognition and the proper scheduling of voice messages. A summary of user responses on their initial use of the Passive Demonstrator is given in Table 7.


Table 7.
Summary of results from long-term trial.
Category
Mean Rating
Ease of learning
3.8
Ease of remembering how to use
device
4.2
Feeling of safety
4.4
Ease of use
3.8

   Further user trials were carried out in Sweden and Ireland. The field trial in Ireland was a week in duration and was designed such that the user would be exposed to the device for a longer period of time on a daily basis. Prior to this evaluation, users had typically tested the device for about 15 minutes (limited by user stamina). This was sufficient to obtain feedback on their initial impressions but not enough for them to get particularly practiced at using the device. Twelve visually impaired female participants, with an average age of 79 years, took part in this Irish trial. They were all resident in a home for visually impaired persons. They were all aware of the PAM-AID project and some had taken part in the User Requirements Study and previous trials of the Active Demonstrator.


Figure 6a. A photograph of an early model of the Passive Demonstrator Figure 6b. A photograph of another early model of the Passive Demonstrator
Figure 6.

Early developments in the Passive Demonstrator.

   In this long-term trial, users did point out that they thought the device was a little heavy and found it difficult to push for extended periods of time. This was largely due to the frame being made of steel. Early versions in the development of the Passive Demonstrator are shown in Figure 6. Subsequent versions of the device will be constructed from aluminum. Users complained about intermittent jerkiness as a result of unreliable sonar sensor readings. The system would, for example, mistake smooth-surfaced doors for openings and only at the last minute detect the obstacle and move away. Also, obstacle recognition with sonar is very difficult and unreliable. Users complained of lack of accuracy of the feature detector and also that they were notified of particular features too late. Subsequently the laser-based navigation system was transferred from the Active Demonstrator and provided a more accurate picture of the environment, and as a result, safer, smoother motion was achieved. Some users found it difficult to locate the mode-changing switch and the "turn-on-the-spot" switch. These were subsequently repositioned so that no errors could be made in finding the appropriate switches. The user trials in Sweden highlighted the same issues. It was also emphasized that to be useful to the Swedish potential users, who were much more independently mobile, the device must be able to function outdoors. A summary of overall user impressions is provided in Table 8.


Table 8.
Summary of results from long-term trial.
Category Mean rating
Ease of pushing device 3.1
Ease of steering device 3.8
Ease of switching modes 3.6
Ease of maneuvering on the spot 4.0
Usefulness of voice messages 4.4
Safety 4.4
Overall usefulness 3.8
Personal interest in using device 2.89

DISCUSSION

   Both the Active and Passive Demonstrators were evaluated in the St. Mary's Nursing Home for the Visually Impaired, Dublin, Ireland, for five days (April 12 to 16, 1999). The residents, caregivers, and NCBI staff discussed the devices at length for the week, resulting in three major recommendations in the areas of device customization, integration with the building, and integration with the daily living routine.

Device Customization
   The first major recommendation was that the device should be customized for individuals. The customization should include the type and position of input device. Customizations should accommodate arthritis and mild hemiplegia common among the elderly. The subjects' responses also indicated a need for customization of the control functions of the device based on their visual impairment. The adaptive assistance mode was useful for the severely visually impaired but less so for the partially sighted. The most frail of the participants preferred the Active Demonstrator, as there was no requirement to push the device. During the trials, the utility measures in the context-sensitive user interface were not customized for each user and were set at an "interventionist" level to highlight the difference between the two modes. However, reducing the level and type of intervention would provide significant customization. The caregivers see the device as being useful as a shared resource between several residents. Each user would have his or her own handles that a caregiver could change as needed. The caregiver interface should allow them to easily set and modify user preferences during changeover.

Integration with the Building
   Many of the subjects and caregivers mentioned the need for naming particular places within the building, such as the dining room and chapel, while the device was moving along. Additionally, some subjects recommended point-to-point navigation as a desirable feature of the device. Others requested that the device give them specific directions rather than lead them from point to point. This level of integration could be achieved in the first instance by using maps of the building and performing accurate robot localization. To date this approach as been avoided due to the feeling that the device should be able to operate in any residential setting with a minimum of customization. In the longer term, the recognition of places within the residential home or hospital is an important goal.

Integration with Daily Living Routine
   The final user trial took place over a five-day period and allowed users and caregivers to explore in more depth the concept of having an intelligent mobility aid. Life in a nursing home is highly structured with all activities following a set routine. By the end of the week, structured mobility had become part of the routine of the residents, with each of them taking turns on the devices. Previously, the use of the aid had been seen as an "on-request" service provided to residents. During the trial, residents queued up to use the device and guarded their time on the device jealously. Part of this behavior was due to the novelty of the device and the attention accorded to the user. Regardless of their initial motivation, the device quickly became "part of the day" for residents. This social dynamic may wane if the device becomes commonplace or may work to reject the device if not introduced sensitively. Viewed positively, the device allowed the development of a structured exercise regime that did not require constant attendance by a caregiver.

CONCLUSIONS AND FUTURE WORK

   At this point in the development of PAM-AID, both systems have reached a level of maturity in their overall design. The users in the PAM-AID trials have reported that a smart mobility aid will be potentially useful to them and that the current designs of PAM-AID are acceptable. Research is continuing on the features requested by users, such as integration with the building and user customization. However, further research is required to measure the effect of longer-term use of the PAM-AID. The questions to be answered by such research are: Will it increase users' activity levels? Will it improve their quality of life? Will it free the time of caregivers for other tasks in a residential care facility? The authors recently have begun this type of quantitative research with the Pittsburgh and Atlanta VA centers. The results of this research will appear in subsequent papers in this journal. Reports on the work in progress will be available on the Internet at www.vartry.com.

ACKNOWLEDGMENTS

   The authors acknowledge the contribution of the partners in the EU project PAM-AID, Heather Hunter, Blaithín Gallagher, Helen Petrie, Anne-Marie O'Neill, Marianne Karlsson, Magnus Frost, and Nikos Katevas.

REFERENCES
  1. Campbell Vincent A, Crews John E, Moriarty David G, Zack Matthew M, Blackman Donald K. Surveillance for sensory impairment, activity limitation, and health related quality of life among older adults--United States, 1993-1997. In: CDC Surveillance summaries, December 17, 1999. MMWR 1999;48(No. SS-8):131-54.
  2. Richards Bernard. Results of a survey carried out by the Hampshire Association for the Care of the Blind. Hampshire Association for the Care of the Blind, 1993, Hampshire, SO5 6BL, United Kingdom.
  3. Bruce I, McKennell A, Walker E. Blind and partially sighted adults in Britain: The RNIB Survey. Her Majesty's Stationary Office, 1991, Vol. 1, London.
  4. Welsh RL, Blasch BB. Foundations of orientations and mobility. American Foundation for the Blind, 1987.
  5. Unites States Census Bureau. Sixty-five plus in the United States [pamphlet]. Economics and Statistics Administration, U.S. Department of Commerce; May 1995.
  6. Rubin Gary S, Salive Marcel E. Vision and hearing. In: Guralnik JM, Fried LP, Simonsick EM, Kasper JD, Lafferty ME, editors. The women's health and ageing study: health and social characteristics of older women with disability. Bethesda, MD: National Institute on Ageing, NIH Pub. No. 95-4009; 1995
  7. Spirduso Waneen W, Gilliam-MacRae P. Physical activity and quality of life in the frail elderly. In: Birren James E, Lubben James E, Rowe Janice C, Deutchman Donna E, editors. The concept and measurement of quality of life in the frail elderly. New York: Academic Press; 1991.
  8. Engelhardt KG, Edwards RA. Human-robot integration for service robots. In: Rahimi Mansour, Karwowski Waldemar, editors. Human-robot interaction. London: Taylor and Francis; 1992. p. 315-46.
  9. Virzi Robert A. Refining the test phase of usability evaluation: how many subjects is enough? Human Factors 1992 Aug;34,(4):457-68.
  10. Nielsen Jakob. Guerrilla HCI: using discount usability engineering to penetrate the intimidation barrier. In: Bias Randolph, Mayhew Deborah, editors. Cost-justifying usability. New York: Academic Press; 1994.
  11. Simpson Richard. Improved automatic adaptation through the combination of multiple information sources, PhD Thesis, University of Michigan, 1997.
  12. Lacey Gerard, Mac Namara Shane, Dawson-Howe Kenneth M. Personal adaptive mobility aid for the frail and elderly blind. In: Mittal Vibhu, Yanco Holly, Aronis John, Simpson Richard, editors. Lecture notes in AI: assistive technology and artificial intelligence. Elsevier Science; 1998.
  13. Lacey Gerard, Dawson-Howe Kenneth M. The application of robotics to a mobility aid for the blind. Robotics and Autonomous Systems 1998 July;23(4):245-52.
  14. Lacey Gerard. A Bayesian Network to control action selection for a robot mobility aid. Proceedings of the Field and Service Robotics Conference; 1999 August; CMU, Pittsburgh.
  15. Lacey Gerard, Humann Wolfram, Katevas Nikos. User interfaces for robot mobility aids. Proceedings of the Field and Service Robotics Conference; 1997 December; Canberra.
  16. Lindgaard Gitte. Usability testing and system evaluation: a guide for designing useful computer systems. London: Chapman & Hall; 1994.

View Contents

  Last Updated Monday, November 10, 2008 11:21 AM

Back to Top