Volume 46 Number 9, 2009
Pages 1127 — 1134
Abstract — This study assessed the positive and negative predictive values and the sensitivity and specificity of a nursing dysphagia screening tool and the National Institutes of Health Stroke Scale (NIHSS) for the identification of dysphagia for veterans hospitalized with ischemic stroke. A secondary objective of this study was to evaluate the speech-language pathology consult rate before and after the nursing admission dysphagia screening tool. This retrospective cohort study evaluated veterans admitted to one Department of Veterans Affairs medical center with ischemic stroke during the 6 months both before and after the implementation of a nursing dysphagia screening tool, which was part of the admission nursing template. Stroke severity was measured with the use of the retrospective NIHSS. Dysphagia diagnosis was based on speech-language pathology evaluations. Dysphagia was present in 38 of 101 patients (38%) with ischemic stroke. The nursing dysphagia screening tool had a positive predictive value of 50% and a negative predictive value of 68%, with a sensitivity of 29% and specificity of 84%. The use of the NIHSS to identify dysphagia risk had a positive predictive value of 60% and a negative predictive value of 84%. The NIHSS had better test characteristics in predicting dysphagia than the nursing dysphagia screening tool. Future research should evaluate the use of the NIHSS as a screening tool for dysphagia.
Key words: aspiration risk, dysphagia screening, ischemic stroke, NIH Stroke Scale, nursing admission, pneumonia prevention, predictive value of tests, sensitivity and specificity, stroke swallowing assessment.
Dysphagia commonly occurs among patients with stroke. Depending on the detection method and the stroke severity, approximately 20 to 80 percent of stroke patients have a swallowing abnormality . Dysphagia is associated with adverse patient outcomes, including increased risk of both chest infection and poststroke mortality [1-4]. The diagnosis and management of patients with dysphagia improve poststroke outcomes, including reduced pneumonia rates and lowered mortality. Pneumonia is the third leading cause of death in patients during the first month poststroke, accounting for one-third of poststroke deaths [1,2,5-6]. For these reasons, dysphagia screening has been advocated as a performance measure for the assessment of stroke care quality .
In 2006, the Office of the Inspector General (OIG) of the Department of Veterans Affairs (VA) issued a report about the evaluation and management of patients with feeding and swallowing problems in VA medical centers (VAMCs). This OIG report indicated variations in care and opportunities for improvement of dysphagia evaluation and management. In response to the OIG report, VAMCs began screening hospitalized patients for dysphagia as part of the nursing admission process.
This study compared the diagnostic performance of two methods of screening for dysphagia among veterans hospitalized with an acute ischemic stroke. Specifically, the primary objectives were to assess both the positive and negative predictive values and the sensitivity and specificity of (1) a nursing dysphagia screening tool and (2) the National Institutes of Health Stroke Scale (NIHSS) for the identification of dysphagia. A secondary objective was to evaluate the speech-language pathology (SLP) consult rate before and after implementation of the nursing admission dysphagia screening tool.
This retrospective cohort study evaluated veterans admitted with an ischemic stroke to one tertiary care VAMC located in the Midwest. Administrative data from the VA electronic medical record system (VistA database) were used to identify patients with a discharge diagnosis of ischemic stroke (International Classification of Diseases-Revision 9 codes 434.X and 436) in the 6 months before the dysphagia screening tool implementation (October-March 2007) and in the 6 months postimplementation (April-September 2007).
The medical record review was conducted by an experienced nurse. Coding questions were addressed during weekly team meetings. Interobserver variability was assessed by a complete reabstraction by a second abstractor on a 10 percent random sample of the medical records. A total of 202 data elements were collected for each medical record with an overall agreement rate of 93 percent. Institutional review board and local VA Research and Development Committee approval was received.
The medical record review included data collection about sociodemographic variables (e.g., age, race). We used the Charlson comorbidity index  to assess the patients' past medical history. Higher Charlson scores indicate greater burden of comorbidity.
The dysphagia screening tool was implemented in April 2007 as part of an admission nursing template. The tool consisted of 11 items: decreased consciousness, decreased orientation, inability to follow commands, severe facial weakness, inability to control saliva, weak cough, abnormal speaking voice, poorly articulated speech, patient or family report of difficulty swallowing, cough after swallow, and voice change after swallow. If any one of these 11 items was present, the screening was considered "positive" and nurses were instructed to consult SLP and to make the patient nil per os (NPO) or "nothing by mouth." Nurses did not routinely provide water or other liquid or solid as part of this dysphagia screening. Nurses were required to complete an online education program about the anatomical and clinical features of dysphagia as part of the implementation process. The overall nursing compliance rate for mandated dysphagia education was 61.11 percent. Nurses were required to complete the dysphagia screening on all veterans admitted to the VAMC.
Dysphagia was diagnosed on the basis of the SLP consultation report. Dysphagia was categorized as present if there was evidence of any swallowing abnormality with solids, liquids, or both. Dysphagia was categorized as absent if the swallowing was normal or if there were oral problems without dysphagia (e.g., delayed oral phase).
Stroke severity was measured with the NIHSS. Higher NIHSS scores represent greater stroke severity (where zero represents normal neurological functioning). The NIHSS was obtained retrospectively from the medical record data about the admission neurological examination . The NIHSS was dichotomized at <2 ("low") or >2 ("high").
Pneumonia events were identified based on the medical record documentation of a clinical diagnosis of pneumonia (e.g., a problem included in the patient's problem list). A statement about aspiration without documentation of pneumonia was not classified as pneumonia in this study. We also evaluated whether patients had signs or symptoms consistent with pneumonia (i.e., fever, cough, shortness of breath, and/or chest radiograph findings) but that were not labeled as pneumonia by a clinician. This signs-or-symptoms method did not substantially change the pneumonia identification rate based on clinician diagnosis (data not shown); therefore, the pneumonia rates for this study were based on clinician diagnosis.
All data were analyzed with R version 2.7.2 (R Foundation for Statistical Computing; Vienna, Austria). Descriptive statistics (e.g., mean ± standard deviation [SD], range for dimensional data, and proportions for dichotomous data) were used to describe the characteristics and outcomes of the patients. We used Student's t-tests, Wilcoxon rank sum test, chi-square tests, or Fisher's exact tests to evaluate differences in characteristics and outcomes of the patients before versus after the implementation of the dysphagia screening tool.
We compared the two new methods of screening for dysphagia (the nursing admission screening tool versus the NIHSS dichotomized at >2) with the gold standard method (SLP consultation diagnosis of dysphagia). We restricted our analyses to patients who had been evaluated with one of the new methods and an SLP consultation. We calculated the sensitivity and specificity as well as the positive and negative predictive values for the nursing admission tool and the NIHSS compared with the SLP consultation result.
Among the 101 patients in this cohort, 62 were admitted before and 39 were admitted after the implementation of the dysphagia screening tool. All 101 patients were male. Table 1 compares the demographic and clinical characteristics of the stroke patients admitted before and after the implementation of the nursing screen. The two groups were similar with respect to baseline characteristics such as age (mean ± SD: before, 64.8 ± 10.8 years; after, 66.6 ± 11.7 years; p = 0.428) and stroke severity (NIHSS) (before, 3.85 ± 3.96; after, 3.08 ± 3.07; p = 0.047) (Table 1). The Charlson comorbidity score was significantly higher after the nursing admission screen was implemented (before, 1.74 ± 1.53; after, 2.56 ± 2.02; p = 0.47). The proportion of patients who were discharged to a skilled nursing facility significantly decreased from before to after the dysphagia screen (before, 8 out of 59 [13.6%]; after, 0 out of 39 [0%]; p < 0.0001).
Table 1.Comparison of population before and after implementation of nursing dysphagia screening tool. Characteristic Dysphagia Screening Time Frame
p-Value Before (n = 62) After (n = 39) Age Range (yr) 42-87 43-89 - Mean ± SD 64.8 ± 10.8 66.6 ± 11.7 0.43 White Race: n (%) 38 (61) 29 (74) 0.26 Charlson Comorbidity Score Range 0-7 0-9 - Mean ± SD 1.74 ± 1.53 2.56 ± 2.02 0.047* Admission NIHSS Range 0-19 0-13 - Mean ± SD 3.85 ± 3.96 3.08 ± 3.07 0.33 Number of Days NPO† 2.50 ± 4.95 3.83 ± 4.88 0.41 In-Hospital Death: n (%) 5 (8.1) 1 (2.6) 0.40‡ Discharge Disposition, Skilled Nursing Facility: n (%) 8/59 (13.6) 0/39 (0) <0.001‡ Discharge Functional Status, Not Independent in All ADLs: n (%) 25/56 (44.6) 16/35 (45.7) 0.91 New Pneumonia Cases: n 1/62 1/39 1.000 SLP Consult Ordered: n (%) 32 (52) 20 (51) 0.97 SLP Consult, n (%), Ordered by 0.009‡ Nurse 0 (0) 7 (18) - Physician or Midlevel 32 (52) 12 (31) - SLP (swallowing) Consult Performed: n (%) 31 (50) 18 (46) 0.71‡ Nutrition Consult Performed: n (%) 43/45 (95.6) 14/18 (77.8) 0.051 Patient Remained NPO Until Nursing Dysphagia Screening Completed: n (%) - 33 (84.6) - Results of SLP Consult: n (%) Normal 7 (11) 3 (8) - Dysphagia for Solids 14 (23) 7 (18) - Dysphagia for Liquids 2 (3) 3 (8) - Dysphagia for Solids and Liquids 4 (6) 3 (8) - Oral Problems Without Dysphagia 0 (0) 0 (0) - Any Evidence of Dysphagia (liquid or solid) 24 (39) 14 (36) 0.94 *p-Value based on Wilcoxon rank sum test.†Number of days patients were NPO based on data from 34 patients in before period and 6 patients in after period.‡p-Values obtained from Fisher exact test.ADL = activity of daily living, NIHSS = National Institutes of Health Stroke Scale, NPO nil per os = (nothing by mouth), SD = standard deviation, SLP = speech-language pathology.
No differences were seen in the dysphagia prevalence between the two groups: before, 24 out of 62 (39%); after, 14 out of 39 (36%); p = 0.94. No differences were seen in either the rate of SLP consultation being ordered (before, 32 out of 62 [52%]; after, 20 out of 39 [51%]; p = 0.97) or SLP consultation being performed (before, 31 out of 62 [50%]; after, 18 out of 39 [46%]; p = 0.71). Pneumonia was uncommon in both groups (before, 1 out of 62 [1.6%]; after, 1 out of 39 [2.6%]; p = 1.00). The number of days the patient was made NPO was not statistically different between the two groups (before, 2.50 ± 4.95 days; after, 3.83 ± 4.88 days; p = 0.413) (Table 1).
The nursing dysphagia screening tool had a sensitivity of 29 percent and a specificity of 84 percent, with a positive predictive value of 50 percent and a negative predictive value of 68 percent (Table 2). The NIHSS, compared with SLP consultation, had a sensitivity of 79 percent and a specificity of 68 percent, with a positive predictive value of 60 percent and a negative predictive value of 84 percent (Table 3).
Table 2.Comparison of nursing dysphagia screening tool versus speech-language pathology consultation results. Nursing Screening Tool Results Speech-Language Pathology Results
Dysphagia Present, n = 14 Dysphagia Absent, n = 25 Positive: n = 8 4 True positives 4 False positives Negative: n = 31 10 False negatives 21 True negatives Overall dysphagia prevalence for nursing screening tool: 14/39 (36%).Positive predictive value: true positives/all positives (patients with evidence of dysphagia by either diagnostic method): 4/8 (50%).Negative predictive value: true negatives/all negatives (patients with evidence of dysphagia by either diagnostic method): 21/31 (68%).Sensitivity: 29%.Specificity: 84%.
Table 3.Comparison of National Institutes of Health Stroke Scale (NIHSS) versus speech-language pathology consultation results. Stroke Severity (NIHSS) Speech-Language Pathology Results
Dysphagia Present, n = 38 Dysphagia Absent, n = 63 High NIHSS (>2): n = 50 30 True positives 20 False positives Low NIHSS (0-2): n = 51 8 False negatives 43 True negatives Overall dysphagia prevalence: 38/101 (38%).Positive predictive value: true positives/all positives (patients with evidence of dysphagia by either diagnostic method): 30/50 (60%).Negative predictive value: true negatives/all negatives (patients with evidence of dysphagia by either diagnostic method): 43/51 (84%).Sensitivity: 79%.Specificity: 68%.
These data confirm that dysphagia is common among patients with ischemic stroke, with an overall dysphagia prevalence of 38 out of 101 (38%). This prevalence rate falls within the range reported from previous studies: 17 to 81 percent depending on the timing, methods, and criteria of diagnosis [1-2,10-16]. We found that the nursing admission dysphagia screening tool had relatively poor performance in terms of positive and negative predictive value. The use of the NIHSS to predict dysphagia had somewhat better performance.
Previous research has reported that a formal protocol for dysphagia screening (e.g., check sheet and water swallowing performed on all stroke admissions) can decrease the risk of pneumonia threefold . Pneumonia was uncommon in our cohort. This study was not powered to detect differences in poststroke pneumonia. The present study did not evaluate aspiration outcomes.
This study has several limitations that deserve description. First, the study design did not include a prospective study of dysphagia screening methods, but instead involved a retrospective chart review. In an ideal evaluation of a new diagnostic test against a gold standard diagnostic test, all patients would receive all diagnostic procedures at the same time. In this study, we evaluated patients who received one of the new diagnostic tests (either the nursing screening tool or the NIHSS) as well as an SLP consultation (the gold standard). However, not all the patients in the study received all tests and the tests were not completed at the same time. Therefore, differences between the results of the diagnostic tests could possibly be due in part to changes in the patient's clinical circumstances.
Second, given the medical record review design, this study relied on data recorded in the patients' medical records  and may have not included dysphagia screening that was not documented in the chart. Nursing admission screening likely was not captured in the medical record, because the nursing admission dysphagia screening tool was part of the nursing assessment template, and therefore, the entire tool was either present or absent within an individual patient's medical record. (Note: 98% of all records at the participating VAMC during the study time frame had a nursing dysphagia screening assessment documented.) A possibility does exist that some patients may have received other dysphagia screening (not part of the nursing assessment) that was not considered in this study.
Third, because of the pre/post research design, in which data were abstracted 6 months before the screening and 6 months postimplementation, potential differences could exist in stroke severity by season . However, because the NIHSS was abstracted the same way from both groups and the two groups had a similar NIHSS, differential stroke severity should not be a problem.
Fourth, the current study was conducted at a single site and the findings may not be generalizable to other locations. Thus, this same nursing dysphagia screening tool might yield different results if it were implemented at alternative locations, with different organizational and personnel characteristics. For instance, the results may differ by augmenting the way the nurses were trained on implementing the nursing dysphagia tool. These results suggest that adding more screening components to the already lengthy VA nursing admission template may not be an effective approach to improving veteran care.
Fifth, the screening tool was evaluated only on stroke patients, even though the VA OIG mandate applied to both stroke and nonstroke patients. Nevertheless, if a dysphagia screening tool were going to effectively identify patients at risk of dysphagia, it would do so in a stroke cohort where the prevalence rates of dysphagia are much higher than the general hospital population [2,16]
Finally, the relatively small sample size included in this study gives the analyses of the differences in outcomes, such as pneumonia and other adverse clinical consequences, very low power. Although the sample size was adequate for the evaluation of the test characteristics of the screening tool, future studies should include sufficient numbers for evaluation of the association between various dysphagia screening tools and poststroke outcomes.
Poststroke dysphagia screening and management are key processes of care for patients with stroke. However, consensus is lacking on the best approach to dysphagia screening. Controversy exists about what tool should be used for screening and who should administer the screening (e.g., nurses or speech-language pathologists) . Our results confirm that dysphagia is common among stroke patients. Unfortunately, the nursing admission dysphagia screening tool implemented in the present study had relatively poor performance in terms of positive and negative predictive values. The use of the NIHSS to predict dysphagia was somewhat higher, suggesting that future research could implement the NIHSS as an early screening tool for dysphagia among stroke patients. Research is limited on the extent to which the NIHSS has been used to screen for dysphagia. Further research is warranted to determine the optimal screening strategy for the identification of poststroke dysphagia and how the NIHSS can be used as part of the dysphagia screening process.
Go to TOP
Last Reviewed or UpdatFriday, March 12, 2010 12:52 PM#EndDate -->