VA Research Logo

JRRD logo
Volume 35 Number 4, October 1998
Pages 427-430

Form perception with a 49-point electrotactile stimulus array on the tongue: A technical note

Paul Bach-y-Rita, MD; Kurt A. Kaczmarek, PhD; Mitchell E. Tyler, MS; Jorge Garcia-Lara, DDS

Center for Neuroscience and the Department of Rehabilitation Medicine, Trace R&D Center, University of Wisconsin, Madison, WI, 53706; Medical School, Universidad Autónoma del Estado de Morelos, Cuernavaca, Mexico

Abstract--Form perception with the tongue was studied with a 49-point electrotactile array. Five sighted adult human subjects (3M/2F) each received 4 blocks of 12 tactile patterns, approximations of circles, squares, and vertex-up equilateral triangles, sized to 4×4, 5×5, 6×6, and 7×7 electrode arrays. Perception with electrical stimulation of the tongue is better than with fingertip electrotactile stimulation, and the tongue requires 3% (5-15 V) of the voltage. The mean current for tongue subjects was 1.612 mA. Tongue shape recognition performance across all sizes was 79.8%. The approximate dimensions of the electrotactile array and the dimensions of compartments built into dental retainers have been determined. The goal is to develop a practical, cosmetically acceptable, wireless system for blind persons, with a miniature TV camera, microelectronics, and FM transmitter built into a pair of glasses, and the electrotactile array in a dental orthodontic retainer.

Key words: blind, electrotactile stimulation, sensory substitution, tongue.


This project is based upon work supported in part by the National Eye Institute of the National Institutes of Health, Bethesda, MD 20892.
Address all correspondence and requests for reprints to: Paul Bach-y-Rita, MD, Department of Rehabilitation, University of Wisconsin Medical School, 600 Highland Avenue, Madison, WI 53792-3256;
email: pbachyri@facstaff.wisc.edu.

INTRODUCTION

  We have previously developed tactile vision substitution systems (TVSS) to deliver visual information to the brain via arrays of stimulators in contact with the skin of one of several parts of the body (abdomen, back, thigh, fingertip). Optical images picked up by a TV camera are transduced into a form of energy (vibratory or direct electrical stimulation) that can be mediated by skin receptors. The visual information reaches the perceptual levels for analysis and interpretation via somatosensory pathways and structures. After sufficient training with the TVSS, our blind subjects reported experiencing the images in space, instead of on the skin. They learned to make perceptual judgments using visual means of analysis, such as perspective, parallax, looming and zooming, and depth judgments. Our studies with the TVSS have been extensively described (1-8).

  The TVSS results have provided considerable information on brain plasticity, perceptual mechanisms, and sensory and motor coordination in the development of a "perceptual organ," as well as on other related subjects. However, the results have had limited practical applications, due in large part to man-machine interface considerations. Mechanical vibrotactor systems are bulky and require considerable energy, and electrotactile systems require relatively high voltages, especially in areas like fingertips, due to the protective layers between the outside world and the skin sensory receptors.

  The tongue is very sensitive and highly mobile. Since it is in the protected environment of the mouth, the sensory receptors are close to the surface. The presence of saliva assures good electrical contact. The present study was undertaken with a small electrotactile array developed for a study of form perception with a fingertip (8). We have demonstrated that perception with electrical stimulation of the tongue is better than with fingertip electrotactile stimulation, and the tongue requires only about 3 percent (5-15 V) of the voltage, and much less current (0.4-2.0 mA) than the fingertip.

METHODS

  The present studies were obtained with a 49-point 1.8×1.8 cm electrotactile (electrical stimulation of touch via a matrix of electrodes applying small, controlled electric currents to a touch-sensory area such as the skin or the tongue) display built for the fingertip (Figure 1). For these studies, the display was held somewhat awkwardly in front of the face and the tongue was placed against the array.


Electrode array
Figure 1. Photograph of a 7×7 array of stainless-steel electrodes, 2.54 mm apart. A U.S. penny is shown for size comparison.

Subjects
  Five adult human subjects (3M/2F) participated in these experiments. All were sighted.

Stimulus Patterns
  Figure 2 shows the 12 tactile patterns used in this study: discrete approximations of circles, squares, and vertex-up equilateral triangles, sized to 4×4, 5×5, 6×6, and 7×7 electrode arrays. While four sizes of each basic shape were presented, the subject responded only to the shape of each figure, with size serving as an independent variable.


Pattern set
Figure 2. Stimulus pattern set. Subjects only identified the shape (circle, square, triangle), regardless of the size (matrices of 4×4, 5×5, 6×6, 7×7). Adapted from Strong (9).

Electrotactile Display
  The electrode array consisted of a 7×7 arrangement of 0.89-mm diameter, flat-topped, stainless-steel electrode pins, each surrounded by a 2.36-mm diameter air-gap insulator. A flat stainless-steel plate, coplanar with the electrode pins, served as the return current path. The electrodes were arranged on a square grid with 2.54-mm interelectrode spacing.

  A 16-channel electrotactile waveform generator (VideoTact-4, Unitech Research Inc., Madison, WI) and accompanying scripting software were used to specify and control the stimulus waveform, pattern, and trial events. Active or 'on' electrodes (according to the particular pattern) delivered bursts of positive, functionally monophasic, capacitively coupled (zero net DC) current pulses to the tongue. The current was identical for all electrodes on the array. Inactive or 'off' electrodes were effectively open circuits. Each active electrode received bursts of three 25-µs pulses, where the pulse onsets were separated by 5 ms and the burst onsets by 20 ms. Electrode activation was staggered by 102 µs, so that each electrode in the array could be pulsed once before the next pulse in each burst. Subjects could freely adjust the stimulation current during the experiment.

Experimental Design
  The independent variable was pattern size (matrices of 4×4, 5×5, 6×6, 7×7), and the dependent variable was a perceived shape.

Procedure
  After familiarization with the nature of the electrotactile stimulus (normally perceived as a buzzing or tingling sensation), subjects were seated and the tongue was placed was against the array suspended vertically in front of them. Each was then allowed to freely scan the three size-7 patterns in order to develop an identification strategy. The experimenter did not identify the shapes, but did indicate whether the subject's identification was correct. The subject was then shown the bottom row of Figure 2 (the 3 geometric forms in the largest presentation), but not the other presentation sizes.

  When the subject could correctly identify the three size-7 shapes, he or she received four blocks of 12-pattern identifications trials and instructed to identify the pattern's shape (circle-C, square-S, or triangle-T) regardless of the size. No time constraints were imposed. Subjects were allowed to control the stimulation current at any time by adjusting a knob.

RESULTS

  Pattern identification responses were coded as correct=1, incorrect=0.

  Peformance across all sizes in recognition of shapes by the tongue with the 49-point electrotactile array has been graphed in Figure 3, which shows that raised dot (RD) and electrotactile tongue (TO) performance was better than the two electrotactile fingertip stimulation modes (ETv, electrotactile variable, and ETf, electrotactile fixed, averaged together in the graph), and the electrostatic (ES) mode (9). Mean current for tongue subjects was 1.61 mA, SD=0.40, which was smaller than the mean currents reported for ETf (3.39 mA) and ETv (3.54 mA). Current did not change significantly with each block of TO stimulation, unlike the fingertip ETv case where the subjects kept using more and more current as the experiment went on (8).


Identification performance
Figure 3. Pattern identification performance with the tongue (average for five subjects), compared to results (average for all six subjects) previously reported (8) with stimulation modes perceived through the fingertips: raised dots (RD), consisting of a 0.254-mm thick Lexan® polycarbonate sheet; electrotactile with variable, subject-controlled current and with fixed 'preferred' current; these have been averaged together (ET). For comparison, data reported previously on fingertip perception with electrostatic stimulation (ES) have been graphed.

  Tongue shape recognition performance across all sizes was 79.8 percent, compared to RD performance (96.7 percent), 76.0 percent for ETv, 73.6 percent for ETf and 57 percent for ES. Size×Shape interaction is significant, p=0.001, confirming that the differing pixelization at different sizes has an effect on performance.

  The dimensions and location of compartments that could be built into an orthodontic retainer have been determined. In the anterior part of the retainer, a space of 23×15 mm, by 2 mm deep is available. Two posterior compartments could each be 12×9 mm, and up to 9 mm deep.

DISCUSSION

  The eventual goal is to develop a practical, cosmetically acceptable system for blind persons, with the front end consisting of a miniature TV camera (Professor K. Meier, of the Institute of High Energy Physics of the University of Heidelberg, who will collaborate on future studies, has already developed a tiny 2×2-mm camera), the microelectronic package for signal treatment, the optical and zoom systems, the battery power system, and an FM-type radio signal system to transmit the modified image wirelessly will be included in a glasses frame. For the mouth, an electrotactile display, a microelectronics package, a battery compartment, and the FM receiver will be built into a dental retainer. The stimuluator array could be a sheet of electrotactile stimulators of approximately 27×27 mm, with the electronic and battery packs built into the underside of the array that plugs into those spaces. A standard array package would thus plug into individually molded retainers.

  The present results were obtained with an electrotactile array developed for a fingertip study that was not ideal for the tongue, since a somewhat awkward position with the tongue protruding from the mouth was required. Furthermore, the same wave forms and electrical stimulation parameters were used for the tongue as for the fingertip, without exploring the ideal parameters for the tongue. Some of the identification errors observed appear to be strictly due to the ambiguity of some of the figures: two sizes of the triangle have no peak, the smallest triangle is similar to a circle, and the second-smallest circle has four straight-line sides. The shape has a different effect on performance at different sizes, or alternately the size has a different effect on performance for different shapes: that is, the two effects are not independent of each other.

  A system with the electrotactile array built into an orthodontic retainer would not be appropriate for babies and small children. In a collaborative project in Strasbourg, Sampaio is using an electrotactile display on the abdomen of blind babies, and, since the mouth suction control is the most neurologically developed motor system in babies, she is adapting a pacifier-suction system for mouth control of zoom. She had previously shown that blind infants are capable of responding with appropriate motor behaviour to visual information presented to an intact sensory system (10,11). We propose to develop a baby TVSS built into a pacifier, with the 2×2-mm TV camera (mentioned above) built into the external end and the electrotactile display in contact with the tongue.

  The results presented here have demonstrated the feasibility of perceptual systems for blind persons using electrical stimulation of the tongue. This approach may also have applications to deaf persons, persons with high quadriplegia or limb prostheses, and for augmented communications systems such as in aviation, perception in dark environments, robotics, and underwater exploration.

CONCLUSION

  Tongue shape recognition performance with geometric forms presented through a 49-point electrotactile stimulus array was 79.8 percent across all of the four sizes presented. Perception with electrical stimulation of the tongue is better than with fingertip electrotactile stimulation, and the tongue requires 1-3 percent (5-15 V) of the voltage, and the mean current for tongue subjects was 1.612 mA. Orthodontic retainer compartment and electrotactile array dimensions have been determined. The feasibility of the development of a tactile vision substitution system using an electrotactile display on the tongue of images from a TV camera has been demonstrated.

REFERENCES

  1. Bach-y-Rita P, Collins CC, Saunders F, White B, Scadden L. Vision substitution by tactile image projection. Nature 1969;221:963-4.
  2. White BW, Saunders FA, Scadden L, Bach-y-Rita P, Collins CC. Seeing with the skin. Percept Psychophys 1970;7:23-7.
  3. Bach-y-Rita P. Brain mechanisms in sensory substitution. New York: Academic Press; 1972.
  4. Bach-y-Rita P. Physiological considerations in sensory enhancement and substitution. Eur Med Phys 1989;25:107-28.
  5. Bach-y-Rita P. Nonsynaptic diffusion neurotransmission and late brain reorganization. New York: Demos-Vermande; 1995.
  6. Bach-y-Rita P. Substitution sensorielle et qualia. In: Proust J, editor. Perception et intermodalité. Paris: Presses Universitaires de France; 1996. p. 81-100.
  7. Kaczmarek KA, Bach-y-Rita P. Tactile displays. In: Barfield W, Furness IIIrd T, editors. Advanced interface design and virtual environments. Oxford: Oxford University Press; 1995. p. 349-414.
  8. Kaczmarek KA, Tyler M, Bach-y-Rita P. Pattern identification on a fingertip-scanned electrotactile display. Proceedings of the 19th Annual International Conference of the IEEE Engineering in Medicine and Biology Society; 1997. p. 19.
  9. Strong RM. An explorable electrotactile display (thesis). Cambridge, MA: Massachusetts Institute of Technology; 1969.
  10. Sampaio E, Dufier JL. Suppléance sensorielle électronique pour les jeunes enfents aveugles. J Fr Ophtalmol 1988;11:161-7.
  11. Bach-y-Rita P, Sampaio E. Substitution sensorielle chez les adultes et les enfants aveugles. In: Christen Y, Doly M, Droy-Lefaix M-T, editors. Vision et adaptation. Amsterdam: Elsevier; 1995. p. 108-16.

  Submitted for publication December 5, 1997. Accepted in revised form March 3, 1998.

Back to Top  

Go to the Contents of Vol. 35 No. 4

  Last Updated Friday, June 24, 2005 9:28 AM