Several structural characteristics of ICUs have been linked to better outcomes. Although it is unlikely that there will be randomized controlled trials comparing different models of ICU care,44-57 nighttime availability of intensivists,7,58-68 staffing ratios,69-86 volume of admissions,87-94 specialized units,95,96 shift models,97 availability of technology,98-101 provider experience,102 teamwork,103,104 or organizational climate,105-108 the association between some of these structural features and outcome is quite strong. However, it must be kept in mind that organizational behavior is more complex than the individual structural factors and many organizations may actually perform quite well in spite of not being compliant with policy recommendations.
ICUs can use many different models of care, and the literature has confusing terminology for these different models. An ICU model usually refers to intensivists' degree of responsibility over patient care taken. In “closed” models, only intensivists have admitting privileges to the ICU and work in collaboration with the patient's primary physician. “Open” units allow the patient's primary physician to retain full responsibility over clinical decisions and consultation with an intensive care physician is optional. The term high-intensity staffing model refers to either a closed ICU or an open ICU with mandatory intensivist consultation. A systematic review of the available evidence demonstrates 30% lower hospital and 40% lower ICU mortalities, as well as decreased length of stay, in high-intensity staffing model ICUs.51 High-intensity intensivist staffing models are currently a major quality recommendation of several organizations.109 However, there are several issues to consider in this quality metric. First, with the exception of the United States, most large ICUs are run under what would be considered a high-intensity model; therefore, the open ICU model is primarily an issue for one country. Second, the available literature on this quality metric addresses how the ICU is organized, not whether an individual patient has an intensivist as their physician. At least one publication has demonstrated that, in a select group of critically ill patients, ICUs that have no access to intensivists can have good outcomes.54 This study supports the complexities of organizations and indicates the challenges of implementing system changes on the basis of population studies; some ICUs may achieve equally good outcomes with different models. Although it seems reasonable to suggest the closed ICU model as a policy, health care institutions could benefit from learning why these individual ICUs perform so well, in spite of not having a closed model.110
Nighttime availability of intensivists is another area where the literature uses confusing terms. It may refer to on-site 24 hours coverage by intensivists, to open ICUs where the evening is covered by intensivists, or to availability of consultants over the phone or via computer. Interest in the subject was raised by reports of an association between weekend hospital admissions and mortality for several acute diagnoses, such as abdominal aortic aneurysm, acute epiglottitis, and pulmonary embolism.111 Several investigators pursued the question whether ICU admissions at night or on the weekend were associated with mortality, which led to heterogeneous results.7,60-68 There is speculation that the heterogeneity of results may be due to different models of care: Units that have on-site intensivists may show no differences in mortality between daytime and nighttime admissions,60,61 while units without on-site coverage may have worse outcomes for nighttime admissions.7,65,67 A meta-analysis, including data from 10 studies and more than 100,000 patients, could not demonstrate a higher mortality due to nighttime admissions, even when stratified by subgroups according to intensivist coverage. The authors could demonstrate an association between weekend admissions and mortality, which may reflect the possibility that it is not only the availability of intensivists that makes a difference, but that a more complex organizational behavior on weekends, which might include limited access to other hospital services, may be the most important factor.112 More recent data, from administrative databases including 49 ICUs, demonstrated that in ICUs with a high-intensity staffing model the addition of a nighttime intensivist did not provide benefits; however, in low-intensity staffing ICUs, the presence of a nighttime intensivist was associated with lower mortality.113 Clearly this field is a current and exciting topic, still open for discussion, with authors debating whether 24-hour intensivist staffing should114,115 or not116,117 be adopted. Given the costs of staffing ICUs 24 hours a day, the unavailability of intensivists to staff ICUs even during daytime, and the lack of evidence beyond reasonable doubt, it would be premature to suggest that 24-hour intensivist staffing model should be universally adopted, although it seems reasonable that some organizations may benefit from it, especially those with a low-intensity staffing model.
The most expensive part of intensive care is labor. There is a considerable body of literature trying to identify the ideal nursing staffing ratios and a more limited set of studies looking at other clinician staffing. Not unexpectedly, an association between higher patient to nurse ratio and mortality has been demonstrated. Administrative data from general surgery, vascular and orthopedics patients in 168 hospitals in Pennsylvania showed that there is an OR for mortality of 1.07 per each extra patient per nurse. This represents five excess deaths per 1000 patients if the patient to nurse ratio goes from 4:1 to 8:1.72 Stemming from this important information from ward care, several authors have investigated this issue in more detail in the ICU. A meta-analysis of the current literature supports a decrease of 30% in nosocomial pneumonia, 50% in unplanned extubations, and 9% in mortality per increase in one registered nurse per patient per day.69,83 Interestingly, there seemed to be a dose response effect, consistent with causality, when the data were analyzed by quartiles of patients per nurse in the ICU: Models with 1.6 to 2 patients per nurse per shift were consistently better than models with 3 and even larger effects could be seen on the comparison with models with 4. It seems reasonable to recommend models where nurses do not take responsibility over more than 2 critically ill patients per shift. Obviously, organizations may choose a more fluid regimen, where nurses share responsibility over 4 patients, but one nurse may be dedicated to a more acute patient when needed, while the other takes over 3 less intense patients.
Unfortunately there are scarce data on the appropriateness of intensivist staffing ratios. A single center study, where the expansions of the ICU led to varying staffing rations over time (from 1:7.5 beds to 1:15 beds), provides the only evidence available: There was no effect on mortality with varying staff ratios, but length of stay seemed to be higher in the model with 1 intensivist caring for 15 beds.80 There currently are no data to support recommendations regarding the most appropriate intensivist staffing ratio.
Constant training is one of the hallmarks of highly reliable organizations.118 Much of the training in health care organizations is performed on the job. Therefore, it is intuitive to consider the possibility that institutions that have higher volumes of specific conditions should perform better. Higher volumes of specific conditions may also lead to better outcomes by decreasing variability in diagnosis and focusing nursing expertise. In fact, there is a large amount of evidence linking hospital volumes to better outcomes in several clinical conditions,89 including AIDS,119 cardiology,92,120 vascular surgery,121 cancer,122 orthopedics,123 urology,124 neurosurgery,125 and critical care.87,90 This is important for two reasons: (1) policy makers may choose to combine units to increase the volumes and (2) given the lack of adequate outcomes and process quality indicators for benchmarking, health care consumers may choose hospitals with higher volume as a surrogate of better outcomes.
Similar reasoning led to the concept of specialty ICUs in transplant, trauma, neurosurgery, and other areas. Some evidence points toward better outcomes in units with lower diagnostic diversity99,106 and in neurocritical care units for intracerebral hemorrhage.96 However, analyzing data from almost 100,000 patients in 124 ICUs across the United States, investigators could not demonstrate any benefit of specialty ICUs for six medical conditions, including acute coronary syndrome, ischemic stroke, intracranial hemorrhage, pneumonia, abdominal and cardiothoracic surgery.95 In fact their data support the possibility that “boarding” patients, those with specific conditions being cared for in a specialty ICU outside of the needs of the patient, may actually be harmed by these models.
Given the limitations in studying outcomes or structure as measures of quality, process of care seems like an appealing option. Process measures have intuitive appeal to clinicians who may find data showing that they are not doing something they believe they should be more compelling than recommendations about structure of the ICU or risk-adjusted mortality. It also seems a clearer way to address a clinical behavior than other quality reports. Finally, for statistical reasons it is easier to monitor changes in more common processes than in rare events like death or VAP. Selecting process measures, particularly in critical care, presents some challenges. Ideally process measures should be linked with compelling, usually randomized trial, evidence of a direct effect on outcome. These evidence-based process indicators may be referred to as outcome validated and represent direct measures of quality.126 Unfortunately, there is scarce availability of indicators that have been robustly validated in critical care. Even processes of care based on large randomized clinical trials, such as low tidal volume ventilation for acute lung injury,127 have been disputed in the literature.128 This is the very nature of science and to expect 100% agreement would break the safeguard against collective error that derives from differences in opinion.129 Although not unique to critical care, developing strict process measures of quality of care will always be difficult as the evidence base is modest and evolving. Glucose control and renal dose dopamine are just a few of the treatments that might have made excellent process measures of quality until they were shown to be ineffective or harmful.
There is a bit of confusion in the literature regarding what processes of care means. Examples of processes of care include deep venous thrombosis prophylaxis, sedation interruption strategies, daily assessment of readiness to wean, head of bed elevation, assessment for early enteral nutrition, compliance with evidence-based protocols, use of continuous subglottic aspiration, stress ulcer prophylaxis, and low tidal volume ventilation. Practices that are frequently cited as processes of care, but that we do not consider as such, include length of ICU stay, proportion of occupied beds, duration of mechanical ventilation,130 plateau airway pressures below 30 cm H2O,131 and central venous saturation above 70%.131 The reason for not considering these as processes of care indicators is that they are confounded by patients' characteristics and are not under the exclusive control of providers. It is easy to understand this concept when we discuss ICU length of stay or duration of mechanical ventilation. These end points are clearly influenced by more than just our clinical processes of care and cannot be compared across patients and/or centers without appropriate risk adjustment. However, it is harder to understand why physiologic targets of appropriate treatments are not ideal process of care variables. For example, lung protective ventilation for ARDS using one protocol prescribes the tidal volume and a target plateau pressure. The physician has complete control over setting the tidal volume, however, the resulting plateau pressure reflects a complex interaction between the process measure (tidal volume) and patient factors like thoracic compliance. Ideally, the quality measure would capture the attempt of the physician to respond to the plateau pressure and titrate the tidal volume, but this is difficult to measure. There is nothing wrong with including physiologic targets of evidence-based processes like plateau pressure, central venous saturation, or sedation scores as quality measures, however, they lack one of the basic advantages of process measures, specifically, insensitivity to patient factors and risk adjustment. Therefore, if an ICU looks bad because their patients tend not to achieve some physiologic targets, this might be due to failure to adequately implement the process of care or it might be due to age, obesity, severity of illness, or any of a number of patient factors. If physiologic targets of evidence-based process measures are included in quality assessments, some thought should be given to the need to risk adjust the results to the patient population.
Table 2-3 contains a list of selected processes of care indicators, with validated outcomes summarized to guide in understanding expected benefits from these processes. The last column contains a description of the suggested quality indicator to be measured. The definitions are intentionally broad to allow for local needs in defining eligible patients. Given the state of evidence, it is entirely possible that some of these evidence-based process measures will be under debate as you review this table.
Selected Process of Care Quality Indicators
|Favorite Table|Download (.pdf) TABLE 2-3
Selected Process of Care Quality Indicators
|Process of Care ||Validated Outcome ||Suggested Quality Indicator |
|Mortality ||Resource Utilization ||Other |
|Continuous aspiration of subglottic secretions (CASS)147 ||No effect ||No effect ||Reduced VAP rates ||Proportion of eligible patients using CASS |
|Daily assessment of readiness to wean148 ||Decreased when combined with sedation interruption149 ||Decreased LOS and LMV ||NA ||Proportion of ventilated patients assessed for readiness to wean |
|DVT Prophylaxis150 ||NA ||NA ||Reduced DVT rates ||Proportion of eligible patients using DVT prophylaxis |
|Early antibiotics in septic shock151 ||Decreased ||NA ||NA ||Median time to antibiotic administration after hypotension |
|Early enteral nutrition152,153 ||Decreased (meta-analysis of small trials)152 ||NA ||Reduced pneumonia rates ||Proportion of eligible patients receiving early enteral nutrition |
|No effect (cluster RCT)153 |
|Early goal directed therapy6 ||Decreased ||No effect on LOS or LMV ||NA ||Proportion of severe sepsis/septic shock patients monitoring central venous saturation in the first 6 hours of admission |
|Head of bed elevation154-156 ||No effect ||No effect ||Reduced VAP rates by 50% (results driven by a single small trial, n =86156) ||Proportion of eligible patients with head of bed elevated >30° |
|Hypothermia after cardiac arrest157-158 ||Decreased ||NA ||No effect on pneumonia or sepsis ||Median time to achieve temperature <34°C in eligible patients or proportion of patients achieving target temperature within 6 hours of cardiac arrest |
|Stress ulcer prophylaxis159 || |
No enteral feeding
|NA || |
No enteral feeding
|Proportion of eligible patients using stress ulcer prophylaxis |
|Protective lung ventilation127 ||Decreased ||Increased ventilator-free days ||Decreased nonpulmonary organ failure ||Proportion of eligible patients using low tidal volume ventilation |
|Sedation interruption160 ||Decreased when coupled with spontaneous breathing trial149 ||Decreased LOS and LMV ||NA ||Proportion of eligible patients receiving a daily interruption of sedation |
Mortality, despite its limitations, will always remain high on the list of quality measures stakeholders request when discussing quality. For obvious reasons, crude mortality is inadequate to assess this outcome, and intensive care has led the field of risk adjustment for decades.132-134 Scoring systems have helped us simplify our epidemiological description of critically ill patients and adjust for confounding due to severity of illness in research; however, they have not been validated to be used for (1) benchmarking40 or (2) identification of low performing units.34 One important question remains to be answered: Is it useful to monitor mortality over time as a quality improvement strategy in individual units? Intensivists advocate for several different methods of longitudinal follow-up, including serial standardized mortality ratios (SMRs), risk-adjusted p charts, risk-adjusted CUSUM charts, and other approaches.135 However, to date there are no data to validate the use of longitudinal SMRs to monitor quality.
What makes risk-adjusted mortality unsuitable to be used as a quality indicator?
SMRs can change due to factors unrelated to the quality of care, such as the way laboratory values and vital signs are recorded. In an elegant study, patients had laboratory values and vital signs recorded at ICU admission and then as per clinical indication (standard measurement), concomitantly, the authors measured laboratory values every 2 hours and vitals whenever they were abnormal (intensive measurement). The intensive measurements led to absolute SMRs 10% lower than the standard measurements, in both APACHE II and SAPS II.136 An ICU using more intensive measurement will look better than one that uses standard measurement, even when no real differences exist because the more intensive monitoring yields more extreme values for severity of illness variables.
Differences in case mix may lead to differences in the estimate of the SMR. Even though risk-adjusted models are supposed to deal with different patient characteristics, they are still far from perfectly calibrated. In fact, changing the severity of the case mix leads to differences in the SMR even when there are no real differences in observed outcome per category. In one study, the SMR was categorized by mortality risk, with a cutoff of 10% risk.137 Patients with lower risk had SMRs above 2, while those with higher predicted risk had SMRs close to 1. Obviously, units with higher percentage of low-risk patients may look worse than units that care only for sicker patients. This effect is also expected with different populations where the model may calibrate differently in different patient subsets. Therefore, even though risk-adjustment models were developed to allow for comparisons of different groups of patients, their imperfect calibration makes this use challenging.
Nevertheless, it seems inappropriate to completely ignore the information that may be present in risk-adjusted mortality data. The main concern is that the SMR and changes in it over time should prompt appropriate investigations. Hospitals with SMRs that indicate low mortality and good quality of care should not be overly confident that quality is excellent anymore than hospitals with poor SMRs should be punished for an isolated value.
Recent years have been marked by an increasing interest in nosocomial infections such as VAP and catheter-related blood stream infection (CR-BSI). Hospital-acquired infections are an exciting topic for many stakeholders. They are thought to be preventable and causally linked to higher mortality, morbidity, and cost. In the United States, Medicare will not reimburse providers for the treatment of hospital-acquired infections and they are more frequently presented in public reports.138 However, VAPs are notoriously difficult to diagnose, which impedes their use as a concrete and reproducible quality indicator. Up to a third of patients diagnosed with VAP are found to have no evidence of such in autopsy studies,139 on the other hand, up to one-quarter of patients who die without a VAP diagnosis are found to have evidence of pneumonia at autopsy.140 Physicians' frequently err when diagnosing VAP because signs and diagnostic findings are shared with a multitude of commonly encountered ICU situations: Fever, secretions, leukocytosis, and a new radiographic infiltrate can be seen in conditions as diverse as pulmonary embolism, atelectasis, pulmonary edema, acute lung injury, and pulmonary contusion. Using a model that took into account the uncertainty of clinical findings both from VAP and some of these commonly encountered conditions, investigators demonstrated that VAP rates could vary from 6% to 31%, in spite of a known prevalence of 10%.141 Not only are the findings nonspecific, but the assessment of key points such as secretions, worsening gas exchange, and radiographic infiltrates is quite subjective and prone to interobserver variability.5 It is difficult to demonstrate if recent decreases in VAP rates being published in the literature represent differences in interpretation of the diagnostic criteria as opposed to a real decrease in VAP rates.
CR-BSIs are also being increasingly tracked as a quality measure and suffer the same limitations as VAP. For example, observational data from 24 hospitals in the United States show that by using two different definitions of CR-BSIs, rates can change up to sixfold.142 In Australia, medical charts from six hospitals participating in a statewide surveillance system for CR-BSI were reviewed. Their results were impressive: Sensitivity of the reported cases was 35% and specificity was 87%, with a high false-negative rate, where more than 50% of the CR-BSIs could be missed.143
Based on the above data, it can be concluded that VAP and CR-BSIs share common problems that make them problematic quality indicators: (1) definition is not sensitive or specific; (2) there is large interobserver variability and potential for subjectivity in the diagnosis; (3) events are rare enough that even if definitions were sensitive and specific it would take a long time to collect enough cases to allow for identification of improvement or worsening; and (4) benchmarking between institutions should, but frequently does not, account for case-mix differences.
There are many additional potential outcome measures that might be explored for critical care. Approximately one in five deaths in the United States occur in or after admission to an ICU.144 It may be clear very early that these deaths are unavoidable and evidence-based processes of care may be withheld so some quality metrics may miss the quality of care provided to these patients. Markers of good end-of-life care in the ICU are being developed and could be deployed.145 Patient and family satisfaction with health care is a recognized marker of quality and there are validated instruments for use in the ICU.146 Like other measures, satisfaction does not necessarily correlate with other domains of quality but can be valuable information. Markers of staff retention, burnout, and teamwork have also been proposed as markers of quality and may provide a different perspective.