Journal of Rehabilitation Research & Development (JRRD)

Quick Links

  • Health Programs
  • Protect your health
  • Learn more: A-Z Health
Veterans Crisis Line Badge
 

Volume 50 Number 10, 2013
   Pages vii — xiii

Guest Editorial

William Kearns, PhD; Jan M. Jasiewicz, PhD; James L. Fozard, PhD; Paul Webster, PhD; Steven Scott, DO; Jeffrey Craighead, PhD; Mary E. Bowen, PhD; Joseph McCarthy, PT, MS

Temporo-spacial prompting for persons with cognitive impairment using smart wrist-worn interface

INTRODUCTION

In this editorial, we report on the development of a smartwatch designed to enhance the Smart Home at the Department of Veterans Affairs (VA) hospital in Tampa, a project that uses advanced tracking technology to help Veteran patients with traumatic brain injury (TBI) independently plan, organize, and complete everyday activities [1]. The literature is replete with examples of proposed technologies designed to influence or control behavior, the most recent incarnation being –persuasive technologies– [2–3]. While handheld smartphones have been the pervasive device that readers are most likely to have experience with, new devices targeted at the consumer market now include Google Glass and Samsung's Galaxy Gear smartwatch. Both devices attempt to make the interaction with wireless computing infrastructures a ubiquitous experience. Similarly, many new persuasive applications have been developed to run on common devices such as smartphones, PDAs, and standard computer systems. These applications are often designed to prompt for desired behaviors contingent on time of day to provide basic reminder functions. Very few of these applications use global positioning system (GPS) to derive location and far fewer still are based on precise indoor location [4]. A handful of technologies combine time of day with coarse indoor location to provide targeted prompts to individuals, as for example Takeuchi and Sugimoto, who developed a smartphone application that guides and makes purchasing suggestions to shoppers in Japan [5].

The application of precise location-aware technologies to indoor navigation has been difficult to implement because of poor in-building GPS performance and the high cost of constructing indoor reference grids to duplicate the ranging functions performed outdoors by GPS satellites [6]. For certain applications, ultrawideband (UWB) technologies have shown considerable promise, although the implementation costs can be significant [7]. In a prior Journal of Rehabilitation Research and Development editorial, we reported on the James A. Haley Veterans– Hospital Polytrauma Transitional Rehabilitation Program (PTRP) Smart Home [1] that employs a UWB tracking system from Ubisense that is capable of resolving locations to around 20 cm (vs GPS's 3–5 m accuracy) anywhere in the facility. The system's Smart Home technologies permit selective medication prompting, automated check-in and check-out from the facility, and guidance to therapeutic sessions by presenting prompts on wall-mounted, touch-sensitive screens. To date, 30 PTRP patients have firsthand experience with the Smart Home tracking technology.

While the Smart Home results have been generally positive, we encountered a human-factors issue while deploying this technology in that Veterans would sometimes miss or ignore a prompting message even though the screen delivering the message was very close to the Veteran. In an effort to address this issue, a metric was added to track the changes in Veterans– location. Confirmation that a Veteran saw and responded to the prompt was presumed when the change in the Veteran's location was to that of the prompted destination. Preliminary examination of the data indicated a nominal prompt compliance rate of 75 percent. While seemingly quite high, this can and should be increased. Providing auditory cues when a message appears on the screen may improve the response rate, but providing multiple auditory cues throughout the facility would be an annoying distraction to others. A more personalized approach might be for the Veteran to wear a wireless prompting device with the capability of providing vibrotactile, auditory, and/or visual alerts based on location context.

It is axiomatic that poor compliance with a prompt will occur if the item to be acted on is a large distance away. For example, if a Veteran is prompted to take medication when the pill bottle is not in the immediate vicinity, the Veteran may say, –I must remember to take my medication when I get home.– But short-term memory impairment may undo his or her best intents, with the most likely outcome being a continuation of previous activity. Integration of location-driven notifications on a wearable device is desirable, not only for the convenience of not having to hold and keep track of the assistive device, but in light of the well-documented psychological principle discovered by Pavlov that states repeated performance of a behavior in a specific setting may result in a return to that setting triggering repetition of the behavior. Hence, the idea of augmenting the PTRP system with a location-driven smartwatch was born.

In 2010, work began gathering requirements for a wrist-mounted unit integrated with the PTRP Smart Home tracking system that could provide context-sensitive ??signals using sound, vibration, short text strings, and flashing display. The resultant device, known as the Ubiwatch, was designed so that the watch could be activated remotely or manually. Remote activation would link an individual's measured location with his or her daily schedule. Having an individual activate the watch manually would allow the wearer to tag his or her current location, thus providing a context for the interaction. Two of the authors (WK and JLF) conceptualized the tracking watch-tag during earlier pilot projects to monitor persons with disabilities using UWB in assisted-living facilities (ALFs). They theorized greater acceptance of such a tracking tag if presented with extra functionality such as keeping time. Early conceptual designs were developed in a 2009 University of South Florida design course taught by Franco Lodato, a noted industrial designer, along with one of the authors (WK).

The Ubiwatch development was guided by the Technology Acceptance Model (TAM) [8] and the Unified Theory of Acceptance and Use of Technology Theory (UTAUT) [9]. The major factor predicting successful implementation is the degree to which the user/consumer/stakeholder is involved in the design process. TAM and UTAUT stipulate that users are more likely to adopt devices that are perceived to meet their needs [10–11]. We adopted a collaborative consumer-orientated approach in which all stakeholders were involved from the outset. Our team included prospective users, Ubisense designers and engineers, clinicians, ALF staff, psychologists, and gerontechnologists. We understood early on that each stakeholder possessed unique expertise and insights and their contribution to the design specifications was critical. For example, Veteran stakeholders who experienced polytrauma may have co-occurring sensory loss and may require specialized prompts keyed to remaining intact senses; those with visual impairment benefit from a larger font size and better contrast and may desire vibrotactile signals to draw their attention to the display when required. Therefore, a key part of the design process of the Ubiwatch involved gathering end-user input with respect to optimizing signaling methods, weight, and size. Our Smart Home development team interviewed six Veterans with TBI and five nonveteran ALF residents, with the two groups providing input on the typeface and size of the fonts, the physical dimensions and weight, and the desirability of the signaling interface. An important factor was also the device's perceived size since aesthetics would be an influence in promoting compliance to wear the device. As the process advanced, each stakeholder began to appreciate that the design exercise involved a series of compromises and constraints. More importantly, they learned that the greatest chance for success came from keeping expectations of the users and stakeholders within realistic bounds [10].

Based on the experiences of the Ubisense team and their pioneering work in ubiquitous computing [12], a significant deviation from other similar pervasive devices was that the Ubiwatch would provide limited interaction capabilities and rely on the richer processing and user interface available from the wall-mounted displays deployed in the Smart Home for all specific two-way interaction with the wearer. Figure 1 shows an example of one of these displays. This decision was made so that the Ubiwatch could have a battery life measured in months rather than the daily recharge cycle required by smartphones or similar devices.


Figure 1. One of many wall-mounted message screens in Polytrauma Transtitional Rehabilitation Program that provides customized prompts or reminders to Veteran whose location is revealed by signals transmitted from wrist-worn Ubiwatch shown to ultra-wideband sensor network in facility. Large area on left is where prompt or reminder is displayed; smaller area on right shows name of Veteran for whom message is intended. Ubiwatch also provides customized signal that alerts Veteran that message is on display nearest him/her.

Figure 1.

One of many wall-mounted message screens in Polytrauma Transtitional Rehabilitation Program that provides customized prompts or reminders to Veteran whose location is revealed by signals transmitted from wrist-worn Ubiwatch shown to ultra-wideband sensor network in facility. Large area on left is where prompt or reminder is displayed; smaller area on right shows name of Veteran for whom message is intended. Ubiwatch also provides customized signal that alerts Veteran that message is on display nearest him/her.

Click Image to Enlarge. View as PowerPoint Slide

The generic input modalities of Ubiwatch are touch and movement: a rear-mounted proximity sensor provides an indication that the device is being worn, as opposed to sitting on a nightstand, while a single front-mounted touch sensor allows basic user interaction. In addition, a mechanical motion switch indicates whether the tag, and therefore the wearer, is stationary or in motion. The response parameters resulting from these interactions can be controlled through software.

Output capabilities are slightly more comprehensive. Visual feedback is provided via a custom oversized LCD display that provides four seven-segment displays with the capability to display time, day of the week, and separate icons to indicate that the Ubiwatch is paired with the UWB location system and transmitting (Figure 2). A five-segment display is arranged in one row on the bottom edge of the LCD display. Each segment can be selectively turned on or off under remote software control. Each segment is labeled by a user-defined printed overlay so that when a segment is activated it conveys a specific meaning. For example, if a segment above an email icon becomes active the wearer will know that he or she received an email message. The wearer can retrieve the email content on a nearby screen. If the segment above a calendar icon becomes active, the user will know he or she has a pending appointment and, as before, the details of the appointment can be retrieved from a nearby screen. The same is true if a segment above the medication icon appears, indicating to the wearer that he or she needs to take medication.


Figure 2. Ubiwatch display with example overlay. Ubiwatch ordinarily displays digital display of time and day of week. Programmable display can show four-character strings such as PILL or APPT as needed. Soft buttons activated by pressing display face adjust lighting display, etc.

Figure 2.

Ubiwatch display with example overlay. Ubiwatch ordinarily displays digital display of time and day of week. Programmable display can show four-character strings such as PILL or APPT as needed. Soft buttons activated by pressing display face adjust lighting display, etc.

Click Image to Enlarge. View as PowerPoint Slide

While a four-element, seven-segment display cannot represent all possible letters, it can still provide four-character informative messages such as PILL, APPt, CALL, HELP, bAtt, PANL, etc. The LCD display unit has a novel backlight driven by multiple RGB LEDs that allows the entire display to illuminate in many different colors. This can be used to tailor the contrast of the Ubiwatch display for Veterans with low vision, or it can be used as a prompting mechanism with different colors signifying the severity of a message. Tactile signaling is provided using a microminiature vibration motor (useful for low vision and/or hearing loss), and audio signaling is accomplished via a small piezoelectric buzzer.

A two-way radio, operating on the global 2.4 GHz band, allows remote communication with the device, while a one-way UWB radio, based on the standard Ubisense tag technology, provides the precise location tracking capability. The Ubisense UWB location technology is primarily deployed in industrial applications, in which it is used to monitor, control, and optimize the manufacturing of high-value assets, usually in the automotive and aerospace sector. Therefore, one of the first design changes was to adapt the existing tag design for personnel tracking, including utilizing an alternative antenna design that allowed for a planar design that is more suited for the limited space afforded by the watch form factor.

Fitting all the desired features into a compact form factor required a number of novel techniques to be combined for the manufacture and assembly of the device. The UWB radio required the use of specialist high-??frequency printed circuit board (PCB) dielectrics, which were combined with a flex-rigid design that allowed a two-dimensional PCB to fold into a compact three-dimensional structure within the enclosure, shown in Figure 3.


Figure 3. Diagram displaying how circuitry fits into form factor of Ubiwatch. Ultrawideband radio required use of specialist high-frequency printed circuit board (PCB) dielectrics, which were combined with flex-rigid design that allowed two-dimensional PCB to fold into compact three-dimensional structure within watch enclosure.

Figure 3.

Diagram displaying how circuitry fits into form factor of Ubiwatch. Ultrawideband radio required use of specialist high-frequency printed circuit board (PCB) dielectrics, which were combined with flex-rigid design that allowed two-dimensional PCB to fold into compact three-dimensional structure within watch enclosure.

Click Image to Enlarge. View as PowerPoint Slide

The enclosure design is based on a consumer GPS sports watch and does not appear conspicuous. A number of techniques were used to keep the size to a minimum, the most important being contouring the PCB providing the UWB location beacon, which is at 45??, around the wrist and into the strap area, thus improving –visibility– of the radio antenna during sitting, walking, and eating. In order to feed the rigid PCB into the strap area, the hinges between the straps and the body of the watch were –flipped– so the strap fits around the body of the watch, which also provides additional space in the enclosure for the audio buzzer. Figure 4 shows that final design of the Ubiwatch.


Figure 4. Picture of wrist-worn Ubiwatch. In order to feed rigid printed cir-cuit board into strap area, hinges between straps and body of watch are –flipped– so strap fits around body of watch, which also provides additional space in enclosure for audio buzzer.

Figure 4.

Picture of wrist-worn Ubiwatch. In order to feed rigid printed circuit board into strap area, hinges between straps and body of watch are "flipped" so strap fits around body of watch, which also provides additional space in enclosure for audio buzzer.

Click Image to Enlarge. View as PowerPoint Slide

The enclosure is manufactured using a twin-shot molding technique, with the body structure first being molded in a polycarbonate derivative (similar to the material used for enclosure of some smartphones) for strength and then a soft rubber material molded on top to provide a contrasting finish that reduces the perceived thickness of the unit, as well as to form a water-resistant gasket. The straps are implemented using an over-molding technique that allows a much softer, nonallergenic strap material to be applied over the rigid hinge elements. Two strap lengths are provided to support different wrist diameters.

Finally, a custom nonrechargeable power cell was designed using the same manufacturing and packaging used for the lithium polymer cells used in most modern phones. The resulting 2 Ah unit is only 4 mm and is expected to power the unit continuously for an average of 6 mo.

The current Ubiwatch firmware provides a number of useful built-in features suited for clinical applications:

When the proximity sensor indicates it is being worn, it can periodically beacon on the 2.4 GHz radio, allowing the wearer to be located with a handheld tracker, even beyond the range of the UWB infrastructure. This negates the requirement for a costly ubiquitous infrastructure, particularly in locations where privacy is a concern.
The Ubiwatch's front touch sensor triggers the backlight of the display to allow it to be read easily in low light. An optional mode allows the touch sensor, when depressed for a few seconds, to trigger a context-sensitive CALL feature to request assistance at the wearer's location. Furthermore, a sustained press on the touch sensor escalates the CALL notification to an urgent request for HELP. The Ubiwatch provides altitude data so an unconscious person on the floor may also be located if he or she cannot send a distress signal.

In addition, the 2.4 GHz radio channel allows the Smart Home software to trigger alerts on the Ubiwatch using any of the audiovisual or vibrotactile methods described, either alone or in combination, making the Ubiwatch a highly flexible test bed for a variety of different interaction models for cognitively impaired Veterans. Nor do the potential benefits of the Ubiwatch end with prompting; Kearns et al. have found that the Ubiwatch's movement tracking technology reveals increasingly disordered walking patterns in ALF residents with cognitive impairment that can predict impending falls [13]. The fall risk index, known as fractal dimension (Fractal D), is gathered from Veterans with TBI as their locations are tracked within the Smart Home, and preliminary evidence strongly suggests that the index decreases as Veterans recover from TBI. Fractal D may one day serve as a real-time measure of clinical change for Veterans with TBI. A short Video clip of the Ubiwatch can be viewed online.

We are developing adaptations of the PTRP system of prompts and reminders to install in individual patient homes, adaptations that may serve as formal care substitutes for patients who cannot be accommodated in the PTRP setting or as a means to provide continuing care for PTRP patients who still need some prompting services in their own homes. We will also determine whether home treatment using an adaptation of our technology will be useful in the treatment of other persons with cognitive limitations, e.g., persons with early-stage dementia. The application of the Ubiwatch capabilities to home settings will be very useful, particularly the capacity of the Ubiwatch to signal persons outside the immediate range of the sensor infrastructure.

Cognitive prosthetics–timely prompts and reminders to perform everyday tasks such as taking medications and keeping appointments–constitute the core of interventions used for treating persons with TBI and other cognitive impairments such as early-stage dementia. There are a number of technologies, including specialized wrist-worn devices, available to deliver timely prompts based on preprogrammed calendar-linked events such as specific times of the day, dates, or days of the week. The motivation behind the development of the Ubiwatch device is that the effectiveness of calendar-linked prompts will be more effective if they are also linked to the location of the person relative to the task or activity being prompted. In the case of the Smart Home, where the locations of both the patient's living area and the treatment areas are fixed, an important use of the addition of the spatial information to the temporal cue is to increase the probability that a prompt delivered on a screen near the location of the Veteran will be responded to properly. In the adaptation of the technology to the home setting currently under development, the delivery of the prompt will be made contingent on the location of the person relative to the resource required for prompted activity, e.g., medications located in the bathroom medicine cabinet. While specific scientific evaluation of the combined time-space linked prompt is not yet available, research on operant conditioning–both basic and applied'shows how specific locations become the controlling stimuli for learned behaviors.

By design, the Ubiwatch is limited in functionality to increase reliability, battery life, and technology acceptance and to decrease the need of the wearer to divide attention between two simultaneously changing environments–the virtual one presented on the watch and the physical one in which he or she is moving. Depending on the application, the vibratory and auditory stimuli can provide the prompt directly or determine when it is necessary for the user to inspect the visual display of words on the watch face. The words themselves are limited to four characters to limit the complexity of the message. Auditory signals are limited to simple sounds; as in other wrist-worn devices, speech capability is usually unreliable. The simplicity of the Ubisense design is in contrast to many other devices that may provide a variety of soft and hard buttons or levers and in some cases scrolling functions. Many of these complex functions are designed to overcome the space limitations of the watch with respect to keyboard and other input modalities, but this fails to consider that these watches are carried with other devices, such as smartphones or computers, that can increase the computing power behind the watch functions.

The Ubiwatch contains an active radio transponder designed to work with Ubisense sensors to derive coordinate data from within a predefined monitored indoor space. Currently, this technology provides the most reliable location information (20 cm in all three planes) within a general indoor monitored area. The 2.4 GHz radio feature allows the wearer to be located when he or she is outside, beyond the range of the UWB infrastructure and allows the wearer to send a message such as a distress signal. Passive sensor systems are less accurate and do not uniquely identify a wearer in congregate settings. The higher cost of the UWB system is readily justified in settings with multiple users but may be relatively more expensive than alternatives when deployed in individual homes. In conclusion, the Ubiwatch provides a way to deliver time-based prompts and reminders contingent on the location of the wearer within the range of the location-aware Ubisense infrastructure. The delivery system represents a personalized and effective message delivery system that does not require the wearer to consult fixed location message display systems. The device provides for determining the wearer's location outside the range of the Ubisense sensor system and allows the wearer to send limited messages.

This project was supported by a T-21 grant awarded to Steven Scott and Jan M. Jasiewicz from the Office of Patient Centered Care and Cultural Transformation, and a T-21 Non-Institutional Long-Term Care grant awarded to Jan M. Jasiewicz from the VA Office of Geriatric and Extended Care.

The authors would like to acknowledge the significant contribution toward the design and implementation of the Ubiwatch: Franco Lodato, Fred Steier, Colin Crompton, Frederica Guzzetta, Christian Almazan, Joe Newman, Edgar Cooper, and others involved in the Smart Home project. Susan Kloss from the Office of Geriatric and Extended Care kindly provided valuable editorial feedback. Finally, we would like to warmly acknowledge Ardis Hanson's contribution to this editorial by conducting a thorough literature review on this topic.

William Kearns, PhD;1–2* Jan M. Jasiewicz, PhD;1 James L. Fozard, PhD;1,3 Paul Webster, PhD;4 Steven Scott, DO;1,5 Jeffrey Craighead, PhD;1 Mary E. Bowen, PhD;6–7 Joseph McCarthy, PT, MS4

1Center of Innovation on Disability and Rehabilitation Research, Veterans Integrated Service Network 8, James A. Haley Veterans Hospital, Tampa, FL; Department of Rehabilitation and Mental Health Counseling, College of Behavioral and Community Sciences, University of South Florida, Tampa, FL; 3School of Aging Studies, College of Behavioral and Community Sciences, University of South Florida, Tampa, FL; 4Ubisense, Cambridge, United Kingdom; 5Physical Medicine & Rehabilitation Services, James A. Haley Veterans Hospital, Tampa, FL; 6Department of Health, West Chester University, West Chester, PA; 7Philadelphia VA Medical Center, Center for Health Equity, Research and Promotion, Philadelphia, PA
*Email: kearns@usf.edu
1.
Jasiewicz J, Kearns W, Craighead J, Fozard JL, Scott S, McCarthy J Jr. Smart rehabilitation for the 21st century: The Tampa Smart Home for veterans with traumatic brain injury. J Rehabil Res Dev. 2011;48(8):vii–xviii.
[PMID:22068378]
http://dx.doi.org/10.1682/JRRD.2011.07.0129
2.
Fogg BJ. Persuasive technology: Using computers to change what we think and do. Boston (MA): Morgan Kaufmann; 2003.
3.
Fogg BJ. Creating persuasive technologies: an eight-step design process. Proceedings of the 4th International Conference on Persuasive Technology; 2009 Apr 26–29; Claremont, CA.
4.
Ha YG, Byun C. A ubiquitous homecare service system using a wearable user interface device. Proceedings of the IEEE/ACIS 11th International Conference on Computer and Information Science.; 2012 May 30–Jun 1; Shanghai, China.
5.
Takeuchi Y, Sugimoto M. CityVoyager: An outdoor recommendation system based on user location history. In: Ma J, Jin H, Yang L, Tsai JP, editors. Ubiquitous intelligence and computing. Heidelberg (Germany): Springer-Verlag; 2006. p. 625–36.
6.
Moreira A, Mautz R. Indoor positioning and navigation. Part III: Navigation systems. J Location Based Services. 2013;7(1):1–2.
http://dx.doi.org/10.1080/17489725.2012.737120
7.
De Angelis A, Nilsson J, Skog I, Handel P, Carbone P. Indoor positioning by ultrawide band radio aided inertial navigation. Metrology Measurement Systems. 2010; XVII(3):447–60.
8.
Davis FD, Bagozzi RP, Warshaw PR. User acceptance of computer technology: A comparison of two theoretical models. Manage Sci. 1989;35(8):982–1003.
http://dx.doi.org/10.1287/mnsc.35.8.982
9.
Venkatesh V, Davis FD. A theoretical extension of the technology acceptance model: Four longitudinal field studies. Manage Sci. 2000;46(2):186–204.
http://dx.doi.org/10.1287/mnsc.46.2.186.11926
11.
Scherer MJ, Lane JP. Assessing consumer profiles of –ideal– assistive technologies in ten categories: An integration of quantitative and qualitative methods. Disabil Rehabil. 1997;19(12):528–35. [PMID:9442991]
http://dx.doi.org/10.3109/09638289709166046
12.
Harter A, Hopper A, Steggles P, Ward A, Webster P. The anatomy of a context-aware application. Wireless Networks. 2002;8(2/3):187–97.
13.
Kearns WD, Fozard, JL, Becker M, Jasiewicz JM, Craighead JD, Holtsclaw L, Dion C. Path tortuosity in everyday movements of elderly persons increases fall prediction beyond knowledge of fall history, medication use, and standardized gait and balance assessments. J Am Med Dir Assoc. 2012;13(7):665.e7–665.e13. [PMID22884091]
This article and any supplementary material should be cited as follows:
Kearns W, Jasiewicz JM, Fozard JL, Webster P, Scott S, Craighead J, Bowen ME, McCarthy J. Temporo-spatial prompting for persons with cognitive impairment using smart wrist-worn interface. J Rehabil Res Dev;50(10): XX–XX.
http://dx.doi.org/10.1682/JRRD.2013.12.0261
iThenticateCrossref

Go to TOP

Last Reviewed or Updated  Tuesday, March 11, 2014 11:28 AM

Valid HTML 4.01 Transitional