Journal of Rehabilitation Research & Development (JRRD)

Quick Links

  • Health Programs
  • Protect your health
  • Learn more: A-Z Health
Veterans Crisis Line Badge
 

Volume 53 Number 3, 2016
   Pages 321 — 334

Usability evaluation of low-cost virtual reality hand and arm rehabilitation games

Na Jin Seo, PhD;1* Jayashree Arun Kumar, MS;2 Pilwon Hur, PhD;3 Vincent Crocher, PhD;4 Binal Motawar, PT, MS;5 Kishor Lakshminarayanan, MS6

1Division of Occupational Therapy, Department of Health Professions, Department of Health Sciences and Research, Medical University of South Carolina, Charleston, SC; 2Larsen & Toubro Infotech Ltd, Edison, NJ; 3Department of Mechanical Engineering, Texas A&M University, College Station, TX; 4Department of Electrical and Electronic Engineering, University of Melbourne, Victoria, Australia; Departments of 5Occupational Science and Technology and 6Industrial and Manufacturing Engineering, University of Wisconsin-Milwaukee, Milwaukee, WI

Abstract — The emergence of lower-cost motion tracking devices enables home-based virtual reality rehabilitation activities and increased accessibility to patients. Currently, little documentation on patients' expectations for virtual reality rehabilitation is available. This study surveyed 10 people with stroke for their expectations of virtual reality rehabilitation games. This study also evaluated the usability of three lower-cost virtual reality rehabilitation games using a survey and House of Quality analysis. The games (kitchen, archery, and puzzle) were developed in the laboratory to encourage coordinated finger and arm movements. Lower-cost motion tracking devices, the P5 Glove and Microsoft Kinect, were used to record the movements. People with stroke were found to desire motivating and easy-to-use games with clinical insights and encouragement from therapists. The House of Quality analysis revealed that the games should be improved by obtaining evidence for clinical effectiveness, including clinical feedback regarding improving functional abilities, adapting the games to the user's changing functional ability, and improving usability of the motion-tracking devices. This study reports the expectations of people with stroke for rehabilitation games and usability analysis that can help guide development of future games.

Key words: finger, game, hand, Kinect, motion, movement, P5 Glove, rehabilitation, stroke, virtual reality.

Abbreviations:3D = three-dimensional, NIH = National Institutes of Health.
*Address all correspondence to Na Jin Seo, PhD; 151B Rutledge Ave, Charleston SC 29425; 843-792-0084; fax: 843-792-0710. Email: seon@musc.edu
http://dx.doi.org/10.1682/JRRD.2015.03.0045
INTRODUCTION

The goal of this study was to improve our understanding of the usability of lower-cost virtual reality games for upper-limb rehabilitation. Specifically, the first objective was to obtain preliminary data on the expectations and preferences of virtual reality rehabilitation games for people with stroke. The second objective was to evaluate the usability of the lower-cost rehabilitation games for finger and arm movement that we have developed.

Community-dwelling people with chronic stroke were the target users of rehabilitation games for this study because stroke is one of the primary causes of long-term disability in adults in the United States [1]. There are more than 6 million people with stroke in the United States [1], many of whom experience long-term disability, especially in the hand and arm [2–8]. Recovery of upper-limb function continues to occur in the chronic stage after patients with stroke are discharged from conventional rehabilitation programs [7,9]. The need for continuous goal-directed physical activity across all stages of recovery is clear [10–11]. However, most people with chronic stroke are not engaged in rehabilitation activity due to the high cost of therapy, lack of insurance coverage, and tendency to drop out of home-based exercise routines [12].

Home-based rehabilitation games have been suggested to facilitate continuous physical rehabilitation at home by providing interesting game activity [13]. Computer-based virtual rehabilitation has shown preliminary efficacy [14–15]. Unlike high-cost systems [16–18], lower-cost games have been enabled by the emergence of lower-cost motion tracking devices such as the Kinect (Microsoft Corporation; Redmond, Washington) and P5 Glove (Essential Reality Inc; New York, New York). While the Kinect has been used for rehabilitation games that involve gross arm movements only [19–20] and the P5 Glove has been used for rehabilitation games for hand and finger movements only [21], there is currently no lower-cost rehabilitation game that involves coordinated arm and finger movement at the same time. Finger and arm coordination is critical for upper-limb function in daily activities such as reaching to grasp and transferring an object to a shelf. Practicing coordinated arm and finger movement is suggested to increase rehabilitation outcomes [18]. Therefore, the need exists for lower-cost rehabilitation games targeting coordinated movement of both hand (with fingers) and arm. To address this need, we have developed lower-cost rehabilitation games targeting coordinated finger and arm movements by using the Kinect and P5 Glove simultaneously and using free open-source three-dimensional (3D) graphics software (Blender, Stichting Blender Foundation; Amsterdam, the Netherlands).

These developed games were evaluated for usability by people with chronic stroke, because developed games may be useless unless they are liked and thus used by target users. In addition, general expectations and preferences for virtual reality rehabilitation games were obtained using a survey to guide future development of rehabilitation games. In addition to surveys using questionnaires, House of Quality analysis was used for usability evaluation of the developed games. The House of Quality is a design-management method to enable product design based on customers' desires from product conception [22]. The House of Quality has been successfully used by many major developers, including Hewlett-Packard, AT&T, Ford, General Motors, and Toyota [22], as well as for customized wheelchair seat design [23]. The House of Quality prioritizes development needs to most effectively satisfy customer expectations. In this study, the House of Quality was used to identify the most imminent technical improvement needs that address user dissatisfaction and increase user satisfaction. This user-centered approach was deemed important because many rehabilitation games are developed by researchers with little input from patients. The target users' liking of any rehabilitation product is expected to directly affect adherence to and motivation for a rehabilitation program and, subsequently, its outcomes.

METHODS
Subjects

Ten community-dwelling people with stroke participated in the usability evaluation, since they are the target users of home-based rehabilitation games (5 males and 5 females, aged 43–76 yr [mean = 63 yr]). They were recruited via flyers posted around the community. They were not videogame users and were not previously involved in virtual reality or gaming studies. Time since stroke ranged from 3 to 13 yr (mean = 8 yr). Upper-limb function was assessed by Fugl-Meyer Assessment [24], and scores ranged from 2 to 66 out of 66 (mean = 42). No subjects had obvious cognitive impairment.

Procedure

Subjects first completed a survey about their expectations for virtual reality rehabilitation games. The subjects then experienced three games developed in the laboratory while being observed. The subjects completed another survey to evaluate the games. The survey results were further used for the House of Quality analysis.

Pregame Survey

The subjects rated the importance of 10 criteria about rehabilitation games on a Likert scale of 1 to 5 (where 1 = not important and 5 = very important) (Figure 1). They were further asked to express preference between a game that is easy to install and play, but without proven clinical effectiveness, and a game that is complicated to install and play, but with proven clinical effectiveness. They were also asked whether they would prefer game score tracking versus clinical functional score tracking.


Figure 1. Ratings of 10 criteria in regards to their importance for virtual reality rehabilitation games were obtained in pregame survey. Criteria were ease of understanding game goals and rules; ease of use; interest in games; motivation of games; adequate level of difficulty for user’s functional ability; graphics quality; variety of game activity and scenes; game score tracking; clinical feedback, including assessment of clinical functional scores; and proven clinical effectiveness. Subjects rated seven criteria as important or very important (green bars) and three criteria as less than important (blue bars). Median with first and third quartile values from 10 subjects is shown.

Figure 1.

Ratings of 10 criteria in regards to their importance for virtual reality rehabilitation games were obtained in pregame survey. Criteria were ease of understanding game goals and rules; ease of use; interest in games; motivation of games; adequate level of difficulty for user's functional ability; graphics quality; variety of game activity and scenes; game score tracking; clinical feedback, including assessment of clinical functional scores; and proven clinical effectiveness. Subjects rated seven criteria as important or very important (green bars) and three criteria as less than important (blue bars). Median with first and third quartile values from 10 subjects is shown.

Click Image to Enlarge. View as PowerPoint Slide

In addition, the subjects' expectations on the contents of rehabilitation games were surveyed. Specifically, the subjects were asked to rank the most motivating and suitable content of games in the categories of sports, fictional, hobby, and activities of daily living (Figure 2). The subjects were also asked whether rehabilitation games should focus on the hand, arm, or both.


Figure 2. Ranking of four different game contents (a) from most to least motivating rehabilitation games and (b) from most to least suitable rehabilitation games. Median values with first and third quartiles obtained from 10 subjects in pregame survey are shown.

Figure 2.

Ranking of four different game contents (a) from most to least motivating rehabilitation games and (b) from most to least suitable rehabilitation games. Median values with first and third quartiles obtained from 10 subjects in pregame survey are shown.

Click Image to Enlarge. View as PowerPoint Slide

Furthermore, expected usage was surveyed, including whether subjects would use the game, the time duration they were willing to spend on setup, preferred frequency and length of gameplay, whether they would buy the game, and preferred game cost (Figure 3). Last, subjects were asked whether they would prefer therapist guidance during gameplay and whether they would like a therapist to access gameplay records to provide encouragement. Subjects were asked to provide a rationale for their choices when possible.


Figure 3. Responses for (a) likelihood of playing rehabilitation game at home, (b) acceptable setup time before each play, (c) preferred frequency of game play, (d) preferred duration of each play, (e) likelihood of buying rehabilitation game, and (f) reasonable price for game system. No. = number.

Figure 3.

Responses for (a) likelihood of playing rehabilitation game at home, (b) acceptable setup time before each play, (c) preferred frequency of game play, (d) preferred duration of each play, (e) likelihood of buying rehabilitation game, and (f) reasonable price for game system. No. = number.

Click Image to Enlarge. View as PowerPoint Slide

Game Experience

Subjects experienced three lower-cost virtual rehabilitation games developed in the laboratory: a kitchen game (Figure 4), archery game (Figure 5), and puzzle game (Figure 6). The games were tested in the following order: puzzle game, archery game, and kitchen game. All three games targeted coordinated finger and arm movement since most daily living activities involving the upper limb require coordination of both the gross and fine motor movements of the arm and fingers, such as reaching for and grasping and releasing of objects. The games used lower-cost motion tracking devices, the P5 Glove and Kinect. The P5 Glove is a wearable glove with bending sensors that detect finger closing and opening and an infrared light-emitting diode sensor that detects hand position. The Kinect is a markerless sensor for detecting the whole body posture. The total hardware cost was approximately $160 (~$120 for the Kinect and ~$40 for the P5 Glove).


Figure 4. (a–d) Kitchen game involved four functional tasks. Game showed (b) score and (d) time left at top of screen. (e) Virtual arm mimicked subject’s arm and hand movements detected by Kinect (Microsoft Corporation; Redmond, Washington) and P5 Glove (Essential Reality Inc; New York, New York) sensors. Figure 4(a–d) was previously published in Arun Kumar [25] and reprinted with permission.

Figure 4.

(a–d) Kitchen game involved four functional tasks. Game showed (b) score and (d) time left at top of screen. (e) Virtual arm mimicked subject's arm and hand movements detected by Kinect (Microsoft Corporation; Redmond, Washington) and P5 Glove (Essential Reality Inc; New York, New York) sensors. Figure 4(a–d) was previously published in Arun Kumar [25] and reprinted with permission.

Click Image to Enlarge. View as PowerPoint Slide


Figure 5. Archery game involved orienting bow and firing arrow by pointing hand at target and opening fingers. Game screen showed number of hits and time elapsed. Subject’s arm and finger movements were detected by Kinect (Microsoft Corporation; Redmond, Washington) and P5 Glove (Essential Reality Inc; New York, New York) sensors, respectively.

Figure 5.

Archery game involved orienting bow and firing arrow by pointing hand at target and opening fingers. Game screen showed number of hits and time elapsed. Subject's arm and finger movements were detected by Kinect (Microsoft Corporation; Redmond, Washington) and P5 Glove (Essential Reality Inc; New York, New York) sensors, respectively.

Click Image to Enlarge. View as PowerPoint Slide


Figure 6. Puzzle game involved grabbing puzzle piece by closing fingers, moving piece by moving hand, and placing piece by opening fingers. Hand and finger movements were detected by P5 Glove (Essential Reality Inc; New York, New York).

Figure 6.

Puzzle game involved grabbing puzzle piece by closing fingers, moving piece by moving hand, and placing piece by opening fingers. Hand and finger movements were detected by P5 Glove (Essential Reality Inc; New York, New York).

Click Image to Enlarge. View as PowerPoint Slide

During the gameplay experience, subjects were given written instructions to ensure proper placement of the Kinect and P5 Glove sensors, how to wear the P5 Glove on their paretic hand, how to turn on the sensors, and how to click an icon on the desktop computer to start the game. Subjects were asked to play all three games for a total of approximately half an hour. All subjects experienced all activities of the games. Only the affected upper limb was used in the gameplay. Subjects were asked to set up and play the games on their own as much as possible, but they were assisted when they had difficulty with instruments and could not solve the issue on their own. Each of the three games is described in more detail later.

The kitchen game simulated functional tasks, including moving cups and plates between a counter and an overhead compartment (Figure 4(a–b)), grasping a teapot and pouring water into a cup (Figure 4(c)), and opening and closing a drawer and moving utensils (Figure 4(d)). These movements represent multiple upper-limb joint coordination, especially hand-arm coordination, and include upward reaching (Figure 4(a–b)), forward reaching (Figure 4(d)), shoulder rotation (Figure 4(c)), and finger opening and closing (Figure 4(a–d)). These upper-limb movements were chosen because they are typically assessed to track rehabilitation progress in people with stroke [24]. Task instructions were given on the game screen (Figure 4(c–d)), along with information on the current game score and time left for a given task (Figure 4(b),(d)). Applause was provided upon successful completion of a task as an in-game audio cue.

The archery game simulated a sporting activity. The subject oriented the bow by pointing the hand to one of the targets, drew the arrow by closing the fingers, and fired the arrow by opening the fingers (Figure 5), facilitating coordination of the hand and arm movement. In addition, the targets were placed in different game screen locations to promote various arm postures outside the abnormal flexion and extension synergy patterns poststroke [26]. The game screen displayed the number of hits on the targets and elapsed time.

Last, for the puzzle game, the subject grabbed a virtual puzzle piece that resembled the shape of individual states on a United States map by closing the fingers, moved and rotated the piece by moving the hand up and down and side to side as well as rotating the forearm or shoulder, and placed the piece at the appropriate location by opening the fingers until the map is complete (Figure 6). The required movements of shoulder flexion or abduction, forearm rotation, and finger flexion or extension were drawn from the upper-limb Fugl-Meyer Assessment [24]. A circular cursor on the game screen reflected the real-time hand position in the frontal plane. The game screen displayed the number of states completed at the top-left corner.

These games were realized technically in the following ways. For the kitchen and archery games, the subject's upper-limb position and finger bending data were obtained from the Kinect and P5 Glove, respectively (Figure 4(e)), using C# programming with software development kits. By combining information from both the P5 Glove (tracking finger motion) and Kinect (tracking gross upper-limb motion), complete motion tracking for the whole upper limb, including fingers, became possible. These real-time complete upper-limb motion data were fed to Blender, where upper-limb joint angles were computed and interaction between the hand and movable objects (e.g., cups, dishes, teapot, and silverware) were determined using the Python language. The virtual arm and virtual objects were updated real-time using the Blender game engine and displayed via the computer monitor and speaker. In the kitchen game, the virtual arm was custom-designed using the armature bones and meshes in Blender and mimicked the upper-limb movements performed by the user in real time (Figure 4). The kitchen environment and kitchen objects were also custom-created in Blender. In the archery game, the 3D models were adapted from the Blender tutorial Web site (Figure 5). The puzzle game uses similar architecture to the other two games, except that only the P5 Glove was used to track the hand position, orientation, and finger bending, and the custom programming was done in C++ and OpenGL.

Postgame Survey

In the postgame survey, subjects rated their satisfaction for each game per 10 criteria on a Likert scale of 1 to 5 (where 1 = terrible, 2 = unsatisfactory, 3 = okay, 4 = satisfactory, and 5 = exceeded expectation). In addition, they were asked to provide any comments about the games.

House of Quality Analysis

To identify the priority of technical improvements needed to increase user satisfaction with the games, House of Quality analysis was performed [23]. The House of Quality matrix was constructed in the following way. Technical improvement needs (technical characteristics), user expectation criteria, and the interrelationship matrix (Figure 7(a)) were determined by the authors via consensus building according to the literature [22]. The technical characteristics (Figure 7(a)) included the usefulness of written instructions for game setup, usefulness of game instructions that appeared on the game screen during gameplay, game contents, device reliability (performing required function and producing same results on repeated trials), game reliability (providing scores or feedback in a consistent manner), game adaptation (changing contents according to user functional ability and improvement over time), number of difficulty levels available in the game, game realism (immersive environment with realistic displays), 3D display (as opposed to two-dimensional display), inclusion of assessment of clinical functional scores such as the Fugl-Meyer Assessment using the motion tracking devices and computer programming, and clinical evidence based on clinical trials. The interrelationship matrix (Figure 7(a)) noted the strong, moderate, weak, or no relationship between individual technical characteristics and user expectation ­criteria.


Figure 7. (a) House of Quality matrix determined priority weights and percentages as outcomes based on user expectation weights for individual criteria, evaluation ratings for each game, technical characteristics, and interrelationship matrix. User expectation weights and evaluation ratings were median ratings obtained in pre- and postgame surveys, respectively. (b) Priority percentages (normalized priority weights) for individual technical characteristics. Top four priority needs for improvement were evidence of clinical effectiveness, inclusion of clinical functional assessment, game adaptation for changing functional level, and device reliability. 3D = three-dimensional.

Figure 7.

(a) House of Quality matrix determined priority weights and percentages as outcomes based on user expectation weights for individual criteria, evaluation ratings for each game, technical characteristics, and interrelationship matrix. User expectation weights and evaluation ratings were median ratings obtained in pre- and postgame surveys, respectively. (b) Priority percentages (normalized priority weights) for individual technical characteristics. Top four priority needs for improvement were evidence of clinical effectiveness, inclusion of clinical functional assessment, game adaptation for changing functional level, and device reliability. 3D = three-dimensional.

Click Image to Enlarge. View as PowerPoint Slide

The user expectation weight was the median of the subjects' pregame survey rating for each criterion, while the evaluation rating for each game was the median of the subjects' postgame ratings for each criterion for each game, following the literature (Figure 7(a)) [27–28]. Then, the priority weight (Figure 7(a)) was computed as follows for each technical characteristic for each game. Priority weight for a technical characteristic was—

Equation

The conventional interrelationship indices of 9, 3, 1, and 0 for the strong, moderate, weak, and no relationship, respectively, were used [27,29–30]. For example, the priority weight for the first technical characteristic (installation manual) for the kitchen game was computed as 5.0 × 9 × (5 – 3.0) ≈ 90. The priority weights were converted to priority percentages within each game to express the relative priorities across the technical characteristics (Figure 7). The technical characteristics with the high priority weights or percentages are regarded as the priority for technical improvements.

RESULTS
Pregame Survey Results
Rating of 10 Criteria

The rating of importance showed that people with stroke regarded seven criteria as important for virtual reality rehabilitation games (with median ratings ≥4) (Figure 1). The important criteria were ease of understanding game goals and rules, ease of use, interesting games, motivating games, adequate level of difficulty for individual users, clinical feedback, and proven clinical effectiveness. The other three criteria were rated as somewhat important.

Seven out of ten subjects responded that they would prefer a game with proven clinical effectiveness even if the games were complicated to install and play, whereas the other three subjects responded that they would prefer an easy-to-use game even without proven clinical effectiveness. Nine out of ten subjects responded that clinical score tracking was preferred to game score tracking.

Game Content Expectation

The subjects regarded sports and fictional games as more motivating than games about activities of daily living and hobbies, though they regarded the sports and fictional games as less suitable for rehabilitation (Figure 2). All subjects responded that rehabilitation games should focus on both the hand and arm as opposed to only the hand or only the arm.

Expected Game Use and Cost

The majority of subjects responded that they would likely play the game once or twice a day, with each play lasting 40 to 60 min with 1 to 5 min for setup time (Figure 3(a–d)). Positive respondents noted "highly motivated" and "Some patients want to work as much as possible on their own. And it is very hard to generate ideas." The one negative respondent noted the reason for not likely using games at home is that "home board games and cards reinforced physical therapy as an outpatient" (thus, virtual reality games are not needed). Most subjects also responded that they would likely buy games between $21 and $50 (Figure 3(e–f)).

Therapist Involvement

There was a split among subjects (5 vs 5) regarding their preference in having therapist guidance during gameplay. Subjects who prefer to have therapist guidance during gameplay noted, "So I know what I am trying to do" and "Only at the beginning stage." Subjects who did not prefer to have therapist guidance noted, "One should be able to think and figure out how to become more advanced," "Sometimes you learn more on your own," and "I am self-motivated and would want to start and proceed on my own." All respondents said that they prefer having a therapist access their game records such as play frequency, duration, and scores, because "Therapists need to know results to prescribe better rehab," "Patients may not have good judgment for their needs," and "Therapists' suggestion helps to improve patients' progress." All except one respondent said they would like to have a therapist's encouragement to play the game based on the game records. They noted, "It's their job," "To make sure patients do it [play the game]," "To better understand what I am doing," "Continuous support is needed for all patients, emotional as well as physical," "With additional practice data, therapists could easily assume a coaching role. Coaching and encouragement are very important in rehabilitation," and "Therapists should know which games will lead to improvement in rehab."

Postgame Survey Results
Rating of Satisfaction

Figure 7(a) shows the satisfaction ratings obtained for the ten criteria for each game. The least satisfactory criterion was found to be proven clinical effectiveness, followed by clinical feedback. The most satisfactory criterion was the motivating aspect of the games.

Comments

A word cloud shows the main comments the subjects provided about the games (Figure 8). Six subjects commented that the games required "challenging movements." Specifically, one subject commented, "The arm and shoulder movement required is very good movement to recover, i.e., reaching from side to side. Hand and finger are hard movement, perfect for grasp and release."

Nine subjects commented that the games were "interesting," "fun," and/or "engaging" and/or they "liked" the game. Specifically, one subject commented, "I like that [the archery game]. [It is the] sport I have not been able to play since my stroke. You had a purpose to aim at the target. [I] had to think about trajectory of the arrow to aim high or low depending on the target." Another subject commented about the puzzle game: "Very good for those with visual field cuts" (visual neglect).

Four subjects commented that the game rules and goals were "easy to understand" and/or "intuitive." However, difficulty with instruments, including device donning, glove fit, and being captured by the motion tracking devices, was raised by a total of seven subjects: three subjects commented that the written instruction for sensor setup was "difficult to understand," five subjects commented that the P5 Glove was "difficult to put on" or did not "fit" well, and one subject commented that the P5 Glove was easy to put on and had a good fit and size. Specifically, one subject commented, "Help [was] required to get fingers in the glove, but otherwise [the glove was] not as difficult as some hand gadgets in the market." Five subjects commented on the sensor malfunctioning or frustration with motion tracking. Specifically, one subject commented, "Staying in the sensor's range was difficult. The range issues added another level of complexity. It would be better if I didn't have to account for the sensor position. Feedback on the sensor range would be good, too."


Figure 8. Word cloud display of subjects’ comments about games.

Figure 8.

Word cloud display of subjects' comments about games.

Click Image to Enlarge. View as PowerPoint Slide

House of Quality Analysis

The House of Quality matrix was completed by entering the median ratings from the pregame survey for the user expectation weights and the median postgame survey ratings for the evaluation ratings and computing priority weights and percentages (Figure 7(a)). Figure 7(b) visually plots the priority percentages. The top four priority needs for improvement were found to be evidence of clinical effectiveness, inclusion of clinical functional assessment, game adaptation to accommodate the user's changing functional ability, and device reliability.

DISCUSSION
User Expectation for Virtual Reality Rehabilitation Games

The user expectation results may be useful in guiding future rehabilitation game development. Specifically, the rating of importance suggests that development efforts be focused on the seven important criteria (Figure 1). The important criteria included ease of understanding and ease of use, suggesting the need to involve ergonomics experts in game development. The other important criteria of games being motivating and interesting suggest that behavioral or educational psychologists and personal training be involved to develop motivating and interesting games for target users. Increased interest and motivation to play games are expected to result in adherence to rehabilitation programs that are critical for rehabilitation success [12,31–32].

Clinical feedback and proven clinical effectiveness were the other important criteria for the subjects with stroke (Figure 1). Additionally, the subjects preferred games with proven clinical effectiveness that tracked their clinical scores rather than game scores. This finding is interesting given that most people with stroke do not know their clinical functional scores. Taken together with the comments, it appears that people with stroke value clinical evidence and want to be more informed about their clinical status. The subjects were found to value therapist knowledge and insights and welcome therapist help in this process of becoming better informed.

As for game content, the ranking from the most to least "motivating" is reversed for the ranking from the most to least "suitable" (Figure 2). If the goal of a rehabilitation game is to motivate people to move their affected limb, sports or fictional racing or shooting games may be better than those based on activities of daily living, such as a kitchen or grocery store setting. Regardless of the game theme, upper-limb rehabilitation games may focus on both hand and arm movement together, suggested by subject unanimous demand.

The subjects had positive views on rehabilitation games, with most showing interest in buying and playing games at home (Figure 3(a),(e)). This positive view is consistent with the previous literature [33]. Future developers may tailor their games to support daily gameplay of 20 to 60 min, with a short setup time, for a price between $20 and $50 (Figure 3). The information contained in the survey results is expected to inform future development of rehabilitation devices using similar technology and concept so as to minimize potential waste of labor and development cost due to technology abandonment.

Evaluation of Lower-Cost Rehabilitation Games

Subjects rated the kitchen, archery, and puzzle games favorably overall (Figure 7(a)). It is possible that this favorable evaluation may have been biased by the subject characteristics, because people with stroke who participate in research studies may be more adventurous and willing to explore new activity. However, it is encouraging that these relatively simple games developed in a laboratory, not by game specialists, were considered to be satisfactory or okay in most criteria.

In particular, the highest satisfaction rating was obtained for the criterion of the games being motivating. Part of the reason may be that games are inherently perceived as more enjoyable than other therapy programs (see "Postgame Survey Results" section) and that the potential for playing the games at home away from clinical settings is comforting to some people with stroke [34].

To effectively improve subject game evaluation, the unsatisfactory criteria may be addressed by using the House of Quality analysis results. In particular, the criterion with which subjects were least satisfied was proven clinical effectiveness. This dissatisfaction resulted in the number one priority of obtaining clinical evidence (Figure 7(b)). Certainly, clinical effectiveness of these three specific games in the current versions is unknown. For a broader range of various motion-tracking rehabilitation games, many positive results have been reported in pilot studies, but the key active ingredients for increasing rehabilitation outcomes are unknown [35]. This lack of understanding of key ingredients presents a challenge in further developing effective rehabilitation games. Systematic investigation is needed to elucidate the effect of game content on the extent of game usage for individual persons, the effect of adjusting difficulty on outcomes, and dose-response relationships, in order to develop effective rehabilitation games and a clinical trial. It should be noted, however, that clinical effectiveness alone does not supersede other important criteria such as ease of use and games being interesting [36].

The second unsatisfactory criterion was clinical feedback. This unsatisfactory clinical feedback may be improved by addressing the three priority needs for improvement identified in the House of Quality—clinical assessment, game adaptation, and device reliability (Figure 7). Specifically, the former two priority needs for improvement may be addressed by including clinical assessments such as the Fugl-Meyer Assessment within a game; directly linking clinical scores to qualitative descriptions of a user's motor ability, such as by using a key form recovery map [37]; providing the user with the qualitative descriptions as feedback; and adapting the subsequent game content based on the clinical assessment results [38].

The last priority need for improvement, device reliability, is a basic requirement not only for clinical feedback but also for gameplay. However, major issues with the motion tracking devices were raised (see "Postgame Survey Results" section). We observed that many subjects did not understand the concept of the capture volume of motion tracking devices. Specifically, the subjects were often outside of the capture volume by being too close to, too far away from, or too far to the side of the sensor, which resulted in malfunctioning motion tracking. In addition, two motion tracking devices add complexity when the subject may be within the capture volume of the Kinect but his or her hand may be outside the capture volume of the P5 Glove sensor, in which case hand grasping and releasing is not detected and the subject cannot accomplish the game activity.

We also observed that many subjects had difficulty putting on the P5 Glove. Wearable devices for the paretic hand are typically difficult to put on using the contralateral hand only. Specifically for the P5 Glove, all individual paretic fingers had to be placed in rings, which was difficult for the paretic hand. Granted, the P5 Glove is a gaming device not designed for people with paresis. A more patient-friendly motion tracking device for fingers may be explored or developed. In addition, as one subject suggested, the game may provide intuitive, easy-to-understand graphical instructions that visualize the capture volumes of each motion tracking device for users. The games may ask the users to be within the sensors' capture volumes whenever the person moves outside of the capture volume range.

Even though the kitchen and archery games offer a considerable practical advantage with free software and motion tracking devices costing approximately $160 compared with other laboratory devices that typically cost more than $70,000, $160 was still more than subjects were willing to pay (Figure 3(f)). The puzzle game with the P5 Glove only is within the preferred price range (Figure 3(f)).

Future Work

The survey results for the evaluation of the three games were obtained after 30 min of gameplay experience. Thus, only the most critical points could have been captured and other detailed feedback could have been missed, which is deemed acceptable for the purpose of identifying the priority needs for improvement at an early stage of development. At the later stage of development, however, games may be used at home for an extended period of time to allow more in-depth feedback, such as continued usage information.

This study describes perception and preference of rehabilitation games by people with stroke as opposed to clinicians. The rationale is that people with stroke are the end users, and thus, their enjoyment and liking of games will determine whether they would actually use games and comply with therapeutic regimens at home or abandon them as with many existing home rehabilitation programs. In addition, a majority of people with chronic stroke do not see therapists any further, yet have a potential to gain recovery. As such, while the data presented in this article may be used to help increase interest and motivation for people with stroke, the data do not directly represent clinical effectiveness. Future work may aim to obtain expert opinions from therapists to provide additional information on development directions for rehabilitation games.

CONCLUSIONS

This study surveyed the expectations and preferences on virtual reality rehabilitation games and evaluated the usability by people with stroke of three lower-cost rehabilitation games that we have developed to target coordinated finger and arm movements using the Kinect and P5 Glove motion tracking devices. Rehabilitation games show a potential to provide motivating, challenging, interesting, easy-to-use, and easy-to-understand rehabilitation activity that many people with stroke are willing to try. The priority needs for improvement were identified for the developed games using the House of Quality analysis. They were to obtain evidence for clinical effectiveness, to include clinical functional assessment within the games, to adapt the games per changing user functional ability, and to improve usability of motion tracking devices. This study demonstrated user-centered data collection and analysis that can help guide future rehabilitation development to maximize target user satisfaction and successful device deployment.

ACKNOWLEDGMENTS
Author Contributions:
Study concept and design: N. J. Seo, J. Arun Kumar, P. Hur,
V. Crocher, B. Motawar, K. Lakshminarayanan.
Game development: J. Arun Kumar, P. Hur, V. Crocher,
B. Motawar.
Participant recruitment and clinical assessment: B. Motawar.
Usability evaluations: J. Arun Kumar, K. Lakshminarayanan.
Analysis of data: N. J. Seo, J. Arun Kumar, B. Motawar, V. Crocher.
Financial Disclosures: The authors have declared that no competing interests exist.
Funding/Support: This material was based on work supported by the Boston Rehabilitation Outcomes Center, Eunice Kennedy Shriver National Institute of Child Health and Human Development as part of the Medical Rehabilitation Research Infrastructure Network (grant 5R24HD065688-05); the American Heart Association; and the National Center for Advancing Translational Sciences, National Institutes of Health (NIH) (grant 8UL1TR000055).
Additional Contributions: The authors thank Alek Brandenburg and Jerome Scott for their initial contribution to game programming and Anthony Rick for proofreading.
Institutional Review: All subjects signed a consent form approved by the University of Wisconsin-Milwaukee Institutional Review Board.
Participant Follow-up: The authors do not plan to inform participants of the publication of this study.
Disclaimer: The contents of this article are solely the responsibility of the authors and do not necessarily represent the official views of NIH.
REFERENCES
1.
Go AS, Mozaffarian D, Roger VL, Benjamin EJ, Berry JD, Borden WB, Bravata DM, Dai S, Ford ES, Fox CS, Franco S, Fullerton HJ, Gillespie C, Hailpern SM, Heit JA, Howard VJ, Huffman MD, Kissela BM, Kittner SJ, Lackland DT, Lichtman JH, Lisabeth LD, Magid D, Marcus GM, Marelli A, Matchar DB, McGuire DK, Mohler ER, Moy CS, Mussolino ME, Nichol G, Paynter NP, Schreiner PJ, Sorlie PD, Stein J, Turan TN, Virani SS, Wong ND, Woo D, Turner MB; American Heart Association Statistics Committee and Stroke Statistics Subcommittee. Heart disease and stroke statistics—2013 update: A report from the American Heart Association. Circulation. 2013;127(1):e6–245. [PMID:23239837]
http://dx.doi.org/10.1161/CIR.0b013e31828124ad
2.
Trombly CA. Stroke. In: Radomski MV, Latham CA, editors. Occupational therapy for physical dysfunction. 6th ed. Philadelphia (PA): Wolters Kluwer Health/Lippincott Williams & Wilkins; 1989. p. 454–71.
3.
Gray CS, French JM, Bates D, Cartlidge NE, James OF, Venables G. Motor recovery following acute stroke. Age Ageing. 1990;19(3):179–84. [PMID:2363378]
http://dx.doi.org/10.1093/ageing/19.3.179
4.
Nakayama H, Jørgensen HS, Raaschou HO, Olsen TS. Recovery of upper extremity function in stroke patients: The Copenhagen Stroke Study. Arch Phys Med Rehabil. 1994;75(4):394–98. [PMID:8172497]
http://dx.doi.org/10.1016/0003-9993(94)90161-9
5.
Parker VM, Wade DT, Langton Hewer R. Loss of arm function after stroke: Measurement, frequency, and recovery. Int Rehabil Med. 1986;8(2):69–73. [PMID:3804600] http://dx.doi.org/10.3109/03790798609166178
6.
Wade DT, Langton-Hewer R, Wood VA, Skilbeck CE, Ismail HM. The hemiplegic arm after stroke: Measurement and recovery. J Neurol Neurosurg Psychiatry. 1983;46(6): 521–24. [PMID:6875585]
http://dx.doi.org/10.1136/jnnp.46.6.521
7.
Broeks JG, Lankhorst GJ, Rumping K, Prevo AJ. The long-term outcome of arm function after stroke: Results of a follow-up study. Disabil Rehabil. 1999;21(8):357–64.
[PMID:10503976]
http://dx.doi.org/10.1080/096382899297459
8.
Fischer HC, Stubblefield K, Kline T, Luo X, Kenyon RV, Kamper DG. Hand rehabilitation following stroke: A pilot study of assisted finger extension training in a virtual environment. Top Stroke Rehabil. 2007;14(1):1–12.
[PMID:17311785]
http://dx.doi.org/10.1310/tsr1401-1
9.
Page SJ, Sisto S, Levine P, McGrath RE. Efficacy of modified constraint-induced movement therapy in chronic stroke: A single-blinded randomized controlled trial. Arch Phys Med Rehabil. 2004;85(1):14–18. [PMID:14970962]
http://dx.doi.org/10.1016/S0003-9993(03)00481-7
11.
Billinger SA, Arena R, Bernhardt J, Eng JJ, Franklin BA, Johnson CM, MacKay-Lyons M, Macko RF, Mead GE, Roth EJ, Shaughnessy M, Tang A; American Heart Association Stroke Council; Council on Cardiovascular and Stroke Nursing; Council on Lifestyle and Cardiometabolic Health; Council on Epidemiology and Prevention; Council on Clinical Cardiology. Physical activity and exercise recommendations for stroke survivors: A statement for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. 2014;45(8):2532–53. [PMID:24846875]
http://dx.doi.org/10.1161/STR.0000000000000022
12.
Jurkiewicz MT, Marzolini S, Oh P. Adherence to a home-based exercise program for individuals after stroke. Top Stroke Rehabil. 2011;18(3):277–84. [PMID:21642065]
http://dx.doi.org/10.1310/tsr1803-277
13.
Crosbie JH, Lennon S, McNeill MD, McDonough SM. Virtual reality in the rehabilitation of the upper limb after stroke: The user's perspective. Cyberpsychol Behav. 2006; 9(2):137–41. [PMID:16640466]
http://dx.doi.org/10.1089/cpb.2006.9.137
14.
Henderson A, Korner-Bitensky N, Levin M. Virtual reality in stroke rehabilitation: A systematic review of its effectiveness for upper limb motor recovery. Top Stroke Rehabil. 2007;14(2):52–61. [PMID:17517575]
http://dx.doi.org/10.1310/tsr1402-52
15.
16.
Subramanian S, Knaut LA, Beaudoin C, McFadyen BJ, Feldman AG, Levin MF. Virtual reality environments for post-stroke arm rehabilitation. J Neuroeng Rehabil. 2007; 4:20. [PMID:17587441]
http://dx.doi.org/10.1186/1743-0003-4-20
17.
Duff M, Chen Y, Attygalle S, Herman J, Sundaram H, Qian G, He J, Rikakis T. An adaptive mixed reality training system for stroke rehabilitation. IEEE Trans Neural Syst Rehabil Eng. 2010;18(5):531–41. [PMID:20934938]
http://dx.doi.org/10.1109/TNSRE.2010.2055061
18.
Merians AS, Fluet GG, Qiu Q, Saleh S, Lafond I, Davidow A, Adamovich SV. Robotically facilitated virtual rehabilitation of arm transport integrated with finger movement in persons with hemiparesis. J Neuroeng Rehabil. 2011;8:27. [PMID:21575185]
http://dx.doi.org/10.1186/1743-0003-8-27
19.
Brokaw EB, Eckel E, Brewer BR. Usability evaluation of a kinematics focused Kinect therapy program for individuals with stroke. Technol Health Care. 2015;23(2):143–51.
[PMID:25425584]
20.
Roy AK, Soni Y, Dubey S. Enhancing effectiveness of motor rehabilitation using Kinect motion sensing technology. Proceedings of the 2013 IEEE Global Humanitarian Technology Conference: South Asia Satellite; 2013 Aug 23–24; Trivandrum, India. p. 298–304.
21.
Morrow K, Docan C, Burdea G, Merians A. Low-cost virtual rehabilitation of the hand for patients post-stroke. Proceedings of the 2006 International Workshop on Virtual Rehabilitation; 2006; New York, NY. p. 6–10.
23.
Logan GD, Radcliffe DF. Potential for use of a House of Quality matrix technique in rehabilitation engineering. IEEE Trans Rehabil Eng. 1997;5(1):106–15.
[PMID:9086391]
http://dx.doi.org/10.1109/86.559355
24.
Fugl-Meyer AR, Jääskö L, Leyman I, Olsson S, Steglind S. The post-stroke hemiplegic patient. 1. A method for evaluation of physical performance. Scand J Rehabil Med. 1975;7(1):13–31. [PMID:1135616]
25.
Arun Kumar J. Development and usability evaluation of low-cost virtual reality rehabilitation games for patients with upper limb impairment [thesis]. Milwaukee (WI): Department of Engineering, University of Milwaukee-Wisconsin; 2014.
26.
Brunnstrom S. Movement therapy in hemiplegia: A neurophysiological approach. 1st ed. New York (NY): Medical Department, Harper & Row; 1970.
27.
Park SH, Ham S, Lee MA. How to improve the promotion of Korean beef barbecue, bulgogi, for international customers. An application of quality function deployment. Appetite. 2012;59(2):324–32. [PMID:22634190]
http://dx.doi.org/10.1016/j.appet.2012.05.008
28.
Wang RT. Improving service quality using quality function deployment: The air cargo sector of China Airlines. J Air Transp Manage. 2007;13(4):221–28.
http://dx.doi.org/10.1016/j.jairtraman.2007.03.005
29.
Adiano C, Roth AV. Beyond the House of Quality: Dynamic QFD. Benchmark Qual Manage Technol. 1994; 1(1):25–37. http://dx.doi.org/10.1108/14635779410056868
30.
Park T, Kim KJ. Determination of an optimal set of design requirements using House of Quality. J Oper Manage. 1998;16(5):569–81.
http://dx.doi.org/10.1016/S0272-6963(97)00029-6
31.
Maclean N, Pound P, Wolfe C, Rudd A. Qualitative analysis of stroke patients' motivation for rehabilitation. BMJ. 2000;321(7268):1051–54. [PMID:11053175]
http://dx.doi.org/10.1136/bmj.321.7268.1051
32.
Kwakkel G, Wagenaar RC, Koelman TW, Lankhorst GJ, Koetsier JC. Effects of intensity of rehabilitation after stroke. A research synthesis. Stroke. 1997;28(8):1550–56.
[PMID:9259747]
http://dx.doi.org/10.1161/01.STR.28.8.1550
33.
Chen MH, Huang LL, Lee CF, Hsieh CL, Lin YC, Liu H, Chen MI, Lu WS. A controlled pilot trial of two commercial video games for rehabilitation of arm function after stroke. Clin Rehabil. 2015;29(7):674–82.
[PMID:25322868]
http://dx.doi.org/10.1177/0269215514554115
34.
Poltawski L, Boddy K, Forster A, Goodwin VA, Pavey AC, Dean S. Motivators for uptake and maintenance of exercise: perceptions of long-term stroke survivors and implications for design of exercise programmes. Disabil Rehabil. 2015;37(9):795–801. [PMID:25082472]
http://dx.doi.org/10.3109/09638288.2014.946154
35.
Proffitt R, Lange B. Considerations in the efficacy and effectiveness of virtual reality interventions for stroke rehabilitation: Moving the field forward. Phys Ther. 2015; 95(3):441–48. [PMID:25343960]
http://dx.doi.org/10.2522/ptj.20130571
36.
Page SJ, Levine P, Sisto S, Bond Q, Johnston MV. Stroke patients' and therapists' opinions of constraint-induced movement therapy. Clin Rehabil. 2002;16(1):55–60.
[PMID:11837526]
http://dx.doi.org/10.1191/0269215502cr473oa
37.
Velozo CA, Woodbury ML. Translating measurement findings into rehabilitation practice: An example using Fugl-Meyer Assessment-Upper Extremity with patients following stroke. J Rehabil Res Dev. 2011;48(10):1211–22.
[PMID:22234665]
http://dx.doi.org/10.1682/JRRD.2010.10.0203
38.
Metzger JC, Lambercy O, Califfi A, Dinacci D, Petrillo C, Rossi P, Conti FM, Gassert R. Assessment-driven selection and adaptation of exercise difficulty in robot-assisted therapy: A pilot study with a hand rehabilitation robot. J Neuroeng Rehabil. 2014;11:154. [PMID:25399249]
http://dx.doi.org/10.1186/1743-0003-11-154
This article and any supplementary material should be cited as follows:
Seo NJ, Arun Kumar J, Hur P, Crocher V, Motawar B, Lakshminarayanan K. Usability evaluation of low-cost virtual reality hand and arm rehabilitation games. J Rehabil Res Dev. 2016;53(3):321–34.
http://dx.doi.org/10.1682/JRRD.2015.03.0045
iThenticateCrossref

Go to TOP

Last Reviewed or Updated  Tuesday, May 10, 2016 1:55 PM

Valid HTML 4.01 Transitional