VA LOGO

Logo for the Journal of Rehab R&D
Journal of Rehabilitation Research and Development
Vol. 38 No. 2, March/April 2001


Vibratory coded directional analysis: evaluation of a three-microphone/four-vibrator DSP system

Erik Borg, MD, PhD; Jerker Rönnberg, PhD; Lennart Neovius, MSc

Ahlsén Research Institute, Örebro Medical Centre Hospital, S-701 85 Örebro, Sweden; Department of Physiology and Pharmacology, Karolinska Institute, S-171 77 Stockholm, Sweden; Department of Behavioural Sciences, Linköping University, S-581 83 Linköping, Sweden; Hitech Development AB Åldermansvägen 19-21, 171 48 Solna, Sweden


This material is based upon work supported by The Swedish Council for Social Research (Grant #94-0053:2C).
Address all correspondence and requests for reprints to: Erik Borg, MD, PhD, Ahlsén Research Institute, Örebro Medical Centre Hospital, S-701 85, Örebro, Sweden; email: anita.dandenell@orebroll.se.

Abstract — A sound localization aid based on eyeglasses with three microphones and four vibrators was tested in a sound-treated acoustic test room and in an ordinary office. A digital signal-processing algorithm provided a determination of the source angle, which was transformed into eight vibrator codes each corresponding to a 45° sector. The instrument was tested on nine deaf and three deaf-blind individuals. The results show an average hit rate of about 80% in a sound-treated room with 100% for the front 135° sector. The results in a realistic communication situation in an ordinary office room were 70% correct based on single presentations and 95% correct when more realistic criteria for an adequate reaction were used. Ten of the twelve subjects were interested in participating in field tests using a planned miniaturized version.

Key words: digital signal processing, directional aid, sound localization.

INTRODUCTION

   Many individuals with hearing impairment of various degrees, particularly asymmetric losses, totally deaf subjects, and deaf-blind subjects have deteriorated sound-localization ability and thereby disablement with regard to monitoring of environmental information and events. Hearing-impaired subjects lose part of their ability to improve speech perception by using audiovisual speech reading because they cannot quickly localize the speaker. This may be particularly deleterious in a group conversation. Deaf people using sign language often find no potential use of hearing. Still they might benefit from rapid localization of the signing person and an improved ability to localize events in the environment, e.g. approaching cars, persons, dogs, the ringing telephone, and other signals. An improved directional and event-localizing ability might have the greatest benefit for deaf-blind individuals whose contact with the environment is drastically reduced both visually and auditorily. Even though a vibratory device may provide only limited help for language communication, users will gain more awareness of events in the environment and therefore will not be surprised or startled by persons approaching them. Feelings of security may therefore be improved.

   The importance of environmental sound perception for deaf individuals only recently has been emphasized by Söderlund (1). Little attention has been given to this issue in the past, and the focus has consistently been on speech and language communication (2-4). Proctor (5) is one of the few who has looked at the importance of environmental sounds in rehabilitation of severely hard-of-hearing persons. Particularly, Wallin (6) points out the limitation of cochlear implant strategies (with unilateral implants) in terms of the lack of directional information.

   Over the years, there has been some research interest in sound localization using the vibratory sense, for instance see Békésy (7), Gescheider (8), Frost and Richardson (9), and Weisenberger (10). The relevant literature has been recently reviewed in Borg (11). Although the laboratory tests have been promising, no portable equipment has been designed for this purpose so far. In an ongoing project, we have constructed and analyzed the capability of a real-time Digital Signal Processing (DSP) program utilizing signals from three microphones to determine the direction to sound sources with high precision in all directions (reference 12, companion manuscript). The algorithm is fairly robust in noisy environments, and it has been used to control vibrators placed on eyeglasses in order to produce directional information according to a simple code.

   The purpose of the present study is to determine the precision of the sound directional-analysis ability for persons using the described program and the coded vibratory signal, and to assess the individuals' perceived benefit from the system and any problems with the equipment. The tests were performed in two environments with different ecological validity, a) in a sound environmental chamber (13) enabling well-controlled tests with directional sound sources, and b) in an ordinary office room with live voice testing. Two groups of subjects were used, a) deaf subjects and b) multi-handicapped subjects with deaf-blindness.

METHODS

Subjects
   Nine socially deaf subjects in the age range 26-48 years were used for the tests in the environmental chamber, and one of them also in the office room. In order to prevent contamination of audible signals from the vibrators, all chosen test subjects were deaf. Three subjects had a pure tone average >100 dBHL in the best ear, and one was totally bilaterally deaf after head trauma with skull fracture. The best hearing ear in the subject group had a 0.5-, 1.0-, 2.0-Hz average of 60 dBHL. Four of the subjects normally used hearing aids, but not in the test situation. None reported hearing the test sounds or the vibrators. In addition, 3 deaf-blind persons, 23, 33, and 82 years of age, took part in the study. All three had hearing aids, but did not use them in the test situation. Their pure tone averages were 97, 85, and 83 dBHL in the best ear, respectively. The tests in the office room were performed with a totally bilaterally deaf subject who was blindfolded.

Instrument
   Figure 1 shows the three-microphone/four-vibrator device used for the present tests. The signals were correlated with an algorithm described by Borg et al. (reference 12, companion manuscript). The program provided directional information with small errors and a fair degree of noise resistance (signal-to-noise ratio +8 to +10 dB for speech and down to 0 dB for noise bursts). The vibrators were activated during 2 seconds and 2 modes were tested: alternating presentation and simultaneous presentation. The activation was controlled by a threshold and an adaptation function, i.e., after an activation the threshold was temporally raised to prevent continuous activation in constant high sound level surroundings.


A drawing of eyeglasses with three microphones and four vibrators
Figure 1. Drawing of eyeglasses with three microphones and four vibrators.

Vibrator Presentation Mode
   Two stimulation modes were tested in a pilot study on a separate group of nine totally deaf young adults. In the simultaneous mode, the vibrators were activated at the same time for 2 seconds. In the alternating mode, short bursts were presented alternately to 2 vibrators for 2 seconds. The vibration frequency was 125 Hz, identical for the two modes. There was a slightly better result for the alternating mode (4.3 percent error) than for the simultaneous mode (5.5 percent error), and most subjects found the alternating mode easier to perceive.

Direction Codes
   Eight directions were coded (Figure 2). When the two front vibrators were activated, the direction indicated was straight ahead. When 1 vibrator was activated, the direction was shifted by 45 degrees, and sector lines show the limits for the different codes. The code is simple and all subjects learned the code virtually without training.


An illustration of the directional code
Figure 2. The directional code. One or two of the vibrators were active in a simple scheme for each 45-degree sector.

Test Environment/Test Material
   The tests were performed in a sound environmental chamber and an office room, i.e., two environments with different acoustic conditions and ecological validity. In the first case, the subject was seated in the center of a circle (diameter 3 m). Twelve loudspeakers were placed on the periphery of the circle with a 30° separation. In the second case, an ordinary office room was used and 12 marks were placed on the floor, 30 degrees apart, where the experimenter could present live voice stimuli (corresponding to the 12 loudspeakers). The reverberation time of the environmental chamber was 0.2 seconds and that of the office room was 0.4 seconds.

   The test material consisted of monosyllables (approximately 0.8 seconds duration) presented pseudo-randomly from the 12 loudspeakers or by the experimenter in the office room. A relatively short test stimulus was used to create a realistic and fairly difficult situation--an approaching person saying "hello," or the sound of a door opening. The subject responded by pointing with the whole hand in one of the eight directions. A correct response, or "hit," was obtained when the response sector (45°) covered the stimulus direction (45°, 135°, 215°, and 305° azimuth), whereas the response sector 0°, 90°, 180°, and 270° corresponded exactly to the loudspeaker placed at the corresponding angle. Because the loudspeakers in the sound environmental room were fixed and separated by 30 degrees, certain response sectors covered two loudspeakers. For instance, speaker 30° and speaker 60° both fell within the response sector 22.5°-67.5°. There was, on the other hand, only 1 loudspeaker (90°) in the response sector 67.5°-112.5° azimuth. This design was dictated by the difference in construction of the environmental test chamber (12 loudspeakers) and the vibrator code (4 vibrators indicating eight directions). This arrangement was to some extent disadvantageous since the relation between the sound source and the vibratory response angle was not the same for all directions. On the other hand, it also gave an advantage. We had both simple (90-degrees stimuli) and more difficult conditions within the same test session, thereby gaining realism.

Structured Interview
   Two interview forms were established and all deaf subjects were interviewed (with interpreter) before and after testing the equipment. The deaf-blind subjects were interviewed (with interpreter) only after the test, since it was too demanding on these subject to have two interviews and one long test. The questions posed before the test concerned unwarned approaches of cars and persons, the subjects' compensatory strategies, and evaluation of their difficulties. The interview after the test focused on subjects' reactions to the equipment, the possible perceived difficulties, suggestions for alternative design, and attitudes towards a possible personal use of a miniaturized portable version of the equipment. The questions and the response distribution are presented in the Results section.

RESULTS

Tests in Sound Environmental Chamber
   The directional results were evaluated on the basis of a kind of polar diagram showing the indicated response direction. Figure 3 shows the total number of indicated directions in relation to the stimulus directions at 0° (Figure 3A) and 90° azimuth (Figure 3B) for the 9 deaf subjects. The errors in the individuals' interpretations are caused both by errors in the analysis program and by false interpretations of the vibrator code. Four of the errors are caused by the program and 13 by the subject in the 0° azimuth test. It is seen that the number of correct responses is about 80 percent in the 0° azimuth direction; however, all responses but 1 are within the front 135°.


An illustration of examples of the distribution of the direction indicated by the nine deaf subjects for two source directions
Figure 3. Examples of the distribution of the direction indicated by the nine deaf subjects for two source directions.

   Figure 4 is an extension of Figure 3 and shows the distribution of both indicated directions and program calculations for all of the 9 deaf subjects for the 12 sound-source directions. The filled bars show the result of the program calculations and the open bars show direction as indicated by the subject. The relations between the loudspeakers (30° separation) and response alternatives (45° sector) are such that a miscalculation of 16° may cause an erroneous vibrator code. This is most evident for the 135° and 225° response directions where the corresponding loudspeakers are close to the boundaries for the vibrator sector. The rate of correct responses was down to about 60 percent. At 0° azimuth there were 80 percent correct responses.


An illustration of the histogram showing the number (in percent) of responses in each 45-degree sector upon monosyllable stimulation from 12 loudspeakers
Figure 4. Histogram showing the number (in percent) of responses in each 45-degree sector upon monosyllable stimulation from 12 loudspeakers (loudspeaker 12=0° azimuth, 1 graph for each loudspeaker). The filled bars indicate the angle calculated by the computer. The open bars show the angle indicated by the subject.

   By accepting a response sector of ±45° (a total sector of 135°), the hit rate is close to 100 percent for the monosyllables. Such a wide response sector can still be assumed to be useful for a deaf-blind person who then may search with the hand in the appropriate sector, and also for a hearing-impaired sighted person who can turn his head and direct his visual field in the appropriate direction.

Tests in an Ordinary Room
   The word-localization test was accomplished with one deaf subject in an office room, with the experimenter as the speaker. The results showed a very good precision (no errors) in all directions except one (240° azimuth), where the median error was 45°. "No errors" means that all responses (45° sectors) included the stimulus direction. Different echo suppressions were tested and stable results were found for a time constant of 50 ms.

   An everyday communication situation was simulated in the office using a mannequin head supplied with the vibrator system. The experimenter approached the mannequin from behind, creating a situation with three successive events:

  1. opening a door to the room
  2. entering and saying "hello"
  3. approaching 2 meters and asking "how are you?"

   After each event, the mannequin head was turned to the position indicated by the vibrator code and the next stimulus was presented. The results showed that the mannequin faced the speaker correctly within 45° (one sector) for at least 1 of the 3 events in the situation in 17 out of 20 test series and in all series but 1 (95 percent correct) if 1 additional sector was allowed (a total of 135°).

   The echo suppression values used in the office room were 0.5 ms, 5 ms, 50 ms, and 1.0 sec. The results did not differ markedly, but the best values were obtained at 50 ms (10 fully correct responses out of 15 stimuli and the median of the calculated direction values deviated 3° or less from the presentation directions).

Deaf-blind Subjects
   There were some difficulties in the instruction of the deaf-blind subjects (even with experienced deaf-blind interpreters); one of the three had difficulties in maintaining a stable position due to a motor disorder and another made intermittent noises that activated the vibrator system. Figure 5 shows the results from one subject in the sound environmental chamber. It is seen that the hit rate is high. Also, the other two subjects showed good results but the intermittent disturbances made a strict quantitative evaluation difficult. Self-generated sounds that the subject was unaware of could activate the system and give false information of an external event. However, since the vibrations could make the subjects aware of the self-produced sounds, the device could lead to a better control of such tics. This has to be further investigated.


An illustration of directional information obtained for one deaf-blind subject
Figure 5. Directional information obtained for one deaf-blind subject.

Structured Interview
   Each deaf subject was interviewed before and after using the equipment. All but one used sign language as their first language and four used hearing aids (but not in the test situation). Four subjects were fairly often surprised by approaching cars. Four were often or fairly often surprised by approaching subjects. All nine used vibration or touch to notice events that they could not see.

   The nine deaf subjects were presented with a few questions after the test with the vibratory eyeglasses. Eight of them thought it was easy to perceive the vibratory signals. Five suggested that it would be good to have the vibration applied to the wrist. One suggested that there should be different vibratory patterns to the left and the right sides. In that way, front-back and left-right could be easier to distinguish.

   Five of the deaf subjects thought that the vibratory aid could give great help to localize other individuals, seven thought that it could give great help to localize an approaching car, and four thought it would be helpful in detecting an animal in the surrounding area. Seven participants expressed interest in testing a similar aid if it were miniaturized and easy to handle.

   The three deaf-blind subjects took part in a corresponding interview. Our results show that two deaf-blind individuals thought the vibratory signals were easy to perceive, and one additional subject thought they were relatively easy. Two subjects believed that directional vibratory information would be of considerable help in identifying and locating another human or a car (one subject: relatively little benefit). Two of them wanted to try a portable version of the equipment, the third said, "yes, maybe."

DISCUSSION

   The present results show that the percentage of correct responses decreased about 10 percentage points during the process of interpreting the vibrator code. Some of the errors made by the subjects are probably due to suboptimal fitting of the eyeglasses to the head. We have just one pair of eyeglasses, and the vibrator contact could differ among individuals, which in turn could be a cause for the individual variability. These errors should, of course, be possible to minimize with individual fitting of the instrument. This is supported by the observation that some individuals make virtually no vibrator interpretation errors, whereas others make a larger number of mistakes.

   The only comparable study is that of Weisenberg et al. (14). They found a hit rate of 60 percent at the best test frequency for their ear-level two-channel system. Their test conditions were somewhat different, only covering the front 180 degrees. The 80-percent hit rate of the present instrument at 0° azimuth presentation is promising. In addition, when the accepted sector was increased to 135°, there was nearly a 100-percent correct rate.

   What sector size and what precision of directional estimation are relevant for communication and for distant monitoring of the surroundings? We do not know the exactness of the directional information needed for different aspects of communication, orientation, and environmental control. A related phenomenon--turning blindfolded toward a requested clock direction--has a mean error of 9° with a standard deviation up to 15° (Gunnar Jansson, PhD, Department of Psychology, Stockholm University, personal communication).

   For deaf-blind subjects, whether they have a small functional visual field or not (Usher I subjects), an indication of the four main directions front, back, right, left [or even 135° sectors] may be sufficient . Such wide sectors probably allow convenient inspection within the remaining visual field using slow ocular movements. In addition, most environmental processes are made up of a sequence of single events, such as in the presently described test in the office room. In that test, three distinct acoustic signals each activated the directional analysis system. It is probably enough if only one of them leads to a correct orientation. The correct response leads to the establishment of contact with the other subject or the environmental process. The experience with the office room showed that there is a 95-percent chance of establishing contact within a sequence of three events. Extended field studies are required to further evaluate these aspects.

   The importance of directional information for deaf and deaf-blind subjects has seldom been acknowledged. In a recent interview with 13 deaf-blind persons, the role of monitoring of the acoustic surroundings has been highlighted (15). Wallin (6) also underlines the limitation of conventional cochlear implants in terms of the lack of capacity to transmit directional information.

   The experience with the three deaf-blind subjects indicates some difficulty regarding instruction, and an educational program has to be developed in the future. Another aspect of the system, however, became apparent. The presence of spontaneous sounds and movements is a source of error in the localization results. One the other hand, the self-induced vibratory responses may indicate another application of the instrument: as a means to reduce a person's own unintended activities and sounds.

CONCLUSION

   The evaluation in laboratory and conventional acoustic conditions indicates that the system has promising features. The use of individually built vibrator setups may further reduce the number of errors, and field studies are needed in order to determine whether the capacity or precision of the instrument is sufficient to fulfill the needs of hard-of-hearing, deaf, and deaf-blind persons with respect to detection and localization of individuals and events in the nearby surroundings.

ACKNOWLEDGMENTS

   We are grateful to Kapser Marklund, Mats Wilson, Anders Hjälm, David Josefsson, Eva Samuelsson, Yvonne Behrenth, and Anita Dandenell for technical support; Inga-Stina Olsson, Birgitta Andersson-Olsson, and Olle Söderlind for interviews, and Arne Risberg, Karl-Erik Spens, Birger Roos, Gunnar Jansson, and Olle Söderlind for valuable discussions.

REFERENCES
  1. Söderlund G. Tactiling and tactile aids: a user's viewpoint. In: Plant G, Spens KE, editors. Profound deafness and speech communication. London: Whurr Publishers; 1995. p. 25-39.
  2. Plant GL. The use of tactile supplements in rehabilitation of the deafened: a case study. Aust J Audiol 1979;1:76-82.
  3. Rönnberg J, Borg E. A review and evaluation of research on the deaf-blind from perceptual, communicative, social, and rehabilitative perspectives. In press. Scand Audiol 2001.
  4. Rönnberg J, Andersson U, Lyxell B, Spens K. Vibrotactile speechreading support: cognitive prerequisites for training. J Deaf Stud Deaf Educ 1998;3:143-56.
  5. Proctor A. Tactile aid usage in young deaf children. In: Plant G, Spens KE, editors . Profound deafness and speech communication. London: Whurr Publishers; 1995. p.111-46.
  6. Wallin A. The cochlear implant: a weapon to destroy deafness or a support for lipreading. A personal view. In: Plant G, Spens KE, editors. Profound deafness and speech communication. London: Whurr Publishers; 1995. p. 219-30.
  7. von Békésy G. Sensations on the skin similar to the directional hearing, beats and harmonics of the ear. J Acoust Soc Am 1957;29:489-501.
  8. Gescheider GA. Cutaneous sound localisation. J Exp Psychol 1965;70:617-25.
  9. Frost BJ, Richardson BL. Tactile localization of sounds: acuity tracking moving sources and selective attention. J Acoust Soc Am 1976;59:907-14.
  10. Weisenberger JM. Communication of the acoustic environment via tactile stimuli. In: Summers IR, editor. Tactile aids for the hearing impaired. London: Whurr Publishers; 1992. p. 83-189.
  11. Borg E. Cutaneous senses for detection and localization of environmental sound sources: a review and tutorial. Scand Audiol 1997;26:195-206.
  12. Borg E, Neovius L, Kjellander M. A three-microphone system for real-time directional analysis: toward a device for environmental monitoring in deaf-blind. J Rehabil Res Dev 2001;38:XX-XX.
  13. Borg E, Wilson M, Samuelsson E. Towards an ecological audiology: stereophonic listening chamber and acoustic environmental tests. Scand Audiol 1998;27:195-206.
  14. Weisenberg JM, Heidbreder AF, Miller JD. Development and preliminary evaluation of an ear mold sound to tactile aid for the hearing impaired. J Rehabil Res Dev 1987;24:51-66.
  15. Rönnberg J, Samuelsson E, Borg E. Exploring the perceived world of the deaf-blind: on the development of an instrument. In press. Scand Audiol 2001.

View Contents

  Last Updated Monday, November 17, 2008 9:50 AM

Back to Top