Chapter 3. Safety, Quality, and Value

Quality of care has been defined by the Institute of Medicine (IOM) as “the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge.” In its seminal 2001 report, Crossing the Quality Chasm, the IOM advanced six aims for a quality healthcare system (Table 3-1): patient safety, patient-centeredness, effectiveness, efficiency, timeliness, and equity.1 Note that this framework depicts safety as one of these six components, in essence making it a subset of quality. Note also that, though many clinicians tend to think of quality as being synonymous with the delivery of evidence-based care, the IOM's definition is much broader and includes matters that are of particular importance to patients (patient-centeredness and timeliness) and to society (equity).

Although the IOM makes clear that quality is more than the provision of care supported by science, evidence-based medicine does provide the foundation for much of quality measurement and improvement. For many decades, the particular practice style of a senior clinician or a prestigious medical center determined the standard of care (this tradition is now sometimes called “eminence-based medicine,” with more than a hint of derision). Without discounting the value of experience and mature clinical judgment, the modern paradigm for identifying optimal practice has changed, driven by the explosion in clinical research over the past two generations (the number of randomized clinical trials has grown from less than 500 per year in 1970 to 20,000 per year in 2010). This research has helped define “best practices” in many areas of medicine, ranging from preventive strategies for a 64-year-old woman with diabetes to the treatment of the patient with acute myocardial infarction and cardiogenic shock.

Health services researcher Avedis Donabedian's taxonomy is widely used for measuring the quality of care. “Donabedian's Triad” divides quality measures into structure (how is care organized), process (what was done), and outcomes (what happened to the patient).2 When used to assess the quality of care, each element of the Triad has important advantages and disadvantages3 (Table 3-2). In recent years, as clinical research has established the link between certain processes and improved outcomes, process measures have often been used as proxies for quality. Examples include measuring whether hospitalized patients with pneumonia received influenza and pneumococcal vaccinations, and measuring glycosylated hemoglobin (hemoglobin A1c) at appropriate intervals in outpatients with diabetes.

There is a nuance in the last sentence that is worth highlighting. If the latter measure was of whether the physician checked the hemoglobin A1c (or the blood pressure or cholesterol; the point would be the same) at appropriate intervals, that would be a classic process measure. On the other hand, if the measure was of the actual value of the hemoglobin A1c (i.e., fraction of patients with hemoglobin A1c below 7%), that would be an outcome—more specifically, an intermediate outcome, as it is really a proxy for outcomes we care about, such as mortality, kidney function, or retinopathy.4 Although such intermediate outcomes may seem like attractive hybrids between process measures and true outcomes, care is warranted as intermediate outcomes may, like true outcome measures (see below), require case-mix adjustment to fairly assess the quality of care. More importantly, the medical literature is replete with cautionary reports on the results of interventions that produced beneficial effects on plausible intermediate outcomes (such as suppression of premature ventricular contractions or raising levels of HDL cholesterol) but no effect—or even harm—on the main outcomes of interest.5–7

When the science of case-mix adjustment is suitably advanced (e.g., in cardiac bypass surgery8), measures of outcomes such as mortality rates are often used. The caveat is crucial: if case-mix adjustment is not done well, the surgeon or hospital that accepts (or is referred) the sickest patients may appear to be worse than the lesser surgeon or institution that takes only easy cases.

Finally, when the processes are quite complex and the science of case-mix adjustment is immature, structural measures are often used as proxies for quality. As with process measures, using structural measures in this way assumes that good research has linked such elements to overall quality. Examples here include the presence of intensivists in critical care units, a dedicated stroke service, nurse-to-patient ratios, and computerized provider order entry (CPOE).

It is worth highlighting another special kind of outcome measure. As we've seen, the IOM appropriately considers the patient's experience to be one of the key dimensions of quality. Accordingly, assessments of such experience—generally collected through patient surveys—have become important quality measures, both because they reflect an outcome that we intrinsically care about, and because patients may identify quality problems missed by other methods.9,10 In Medicare's new “value-based purchasing” program, rolling out in 2012–2014, 70% of the weight is placed on clinical measures such as care processes (e.g., administration of aspirin and beta-blockers to patients with myocardial infarction) and outcomes (30-day readmission rates, case-mix adjusted mortality rates), and fully 30% is given to the results of patient experience surveys.11

Harvard Business School professor Michael Porter has argued that we should focus our attention on outcome measures, as they are what really matters to patients.12 Although I agree in theory, as the Chapter 1 discussion of using hospital mortality rates as quality/safety measures made clear (and at the risk of being obvious), good process measures are more helpful and valid than bad outcome measures.13

Moreover, even good outcome measures don't obviate the need for process and structural measurement. Let's say we find that our readmission rates for heart failure patients or our mortality rates for stroke admissions are higher than we would like. The next step, of course, is to examine our processes and structures and compare them to those of our peers or to known best-practice benchmarks. This is what we mean when we say that one of the downsides of outcome measures is that they are not directly actionable.3 In the end, it is clear that a thoughtful mix of all elements of the Donabedian Triad—process, structure, and outcome—is important to any sound program of quality measurement and improvement.

In a series of pioneering studies, Wennberg and colleagues demonstrated large and clinically indefensible variations in care from one city to another for the same problem or procedure.14 Other studies have demonstrated large variations in the quality of care for patients based on race, income, and gender (healthcare disparities).15 Together, these studies hint at a fundamental flaw in modern medical practice: we see such profound variations in common processes and procedures, as well as in the outcomes of comparable patient groups, that we can only conclude that care is often inconsistent with evidence.

Stimulated by these early studies on disparities and variations, researchers have more directly measured the frequency with which doctors and healthcare organizations provide care that comports with best evidence. McGlynn and colleagues studied more than 400 evidence-based measures of quality, and found that practice was consistent with the evidence only 54% of the time.16 Adherence to evidence-based processes generally correlates with ultimate clinical outcomes,17 although, soberingly, some studies have found a weaker relationship than one would anticipate.18,19 Nevertheless, these large differences between best and actual practice have caused patients, providers, and policymakers to seek methods to drive and support quality improvement (QI) activities.

The problems described above have exposed several impediments to the reliable delivery of high-quality care, including the lack of information regarding provider or institutional performance, the weakness of incentives for QI, the difficulty for practicing physicians to stay abreast of evidence-based medicine, and the absence of system support (such as information technology) for quality. Each will need to be addressed in order to make substantial gains in the quality of care.

The first step in QI begins with quality measurement. Fifteen years ago, there were only a handful of generally accepted quality measures, such as whether patients with acute myocardial infarction received aspirin or beta-blockers. More recently, scores of such measures have been promulgated by a variety of organizations, including payers (such as the Centers for Medicare & Medicaid Services [CMS]), accreditors and regulators (such as the Joint Commission), and medical societies (Table 3-3). (In the United States, an organization called the National Quality Forum exists largely to review proposed measures and endorse those that meet prespecified validity criteria.20) These measures have identified many opportunities for improvement among individual physicians, practices, and hospitals.

Given the enormous amount of new literature published each year, no individual physician can possibly remain abreast of all the evidence-based advances in a given field. Practice guidelines, such as those for the care of community-acquired pneumonia or for deep venous thrombosis prophylaxis, aim to synthesize evidence-based best practices into sets of summary recommendations. Although some providers deride guidelines as “cookbook medicine,” there is increasing agreement that standardizing best practices is ethically and clinically appropriate; in fact, so-called High Reliability Organizations “hard wire” such practices whenever possible (Chapter 15). The major challenges for guideline developers are the need to keep guidelines updated as new knowledge accumulates21 and the difficulties in developing guidelines that are relevant to the care of patients with multiple, potentially overlapping illnesses.22 Clinical pathways are similar to guidelines, but attempt to articulate a series of steps, usually temporal (on day 1, do the following; on day 2; and so on). They are generally more useful for stereotypical processes such as the postoperative management of patients after cardiac bypass surgery or hip replacement.

Although one could argue that professionalism should be sufficient incentive to provide high-quality care, our recent recognition that the unwavering provision of such care depends on a system organized to reliably translate research into practice means that it will take significant investments (i.e., in developing guidelines, physician education, hiring case managers or clinical pharmacists, building information systems, and more) to deliver optimal care. The traditional payment system, which compensates physicians and hospitals equally whether quality is superb or appalling, provides no incentive to make the requisite investments.

This is changing rapidly, with a blizzard of initiatives to support and catalyze QI activities. Virtually all involve several steps: defining reasonable quality measures (evidence-based measures that capture appropriate structures, processes, or outcomes), measuring the performance of providers or systems, and using these results to promote change. Although each of these steps comes with challenges, the final one has probably created the greatest degree of uncertainty and been the subject of the most experimentation.

The first step in the series of activities designed to promote QI involves selecting appropriate measures. For local QI activities, it may be fine to use process measures that seem plausibly connected to an outcome of interest, even if the evidence connecting the two is not bulletproof. It is also often appropriate to use measures captured through self-report by the involved providers, even while recognizing the inherent biases in such data. Although we all aspire to perfect measurement, “don't let the perfect be the enemy of the good” is a reasonable rule for many local QI projects.

On the other hand, it makes sense to insist on higher standards for measures that are being used for public reporting or pay-for-performance programs. My colleagues and I have called these high-stake assessments “Accountability Measures” and proposed four criteria that they should meet (Table 3-4).23

This distinction is of more than academic interest. Jha and colleagues have shown that even if a provider checks a box attesting that he or she provided discharge counseling to patients with heart failure (the process measure), there is no correlation to readmission rates (the outcome), probably because the quality of such counseling varies widely.24 Moreover, after Medicare instituted a quality measure related to the timing of initial antibiotic administration for patients with pneumonia (the “Four Hour Rule”), subsequent studies demonstrated that many emergency department (ED) patients who were given antibiotics by rushed physicians later proved not to have pneumonia, or any infection, at all.25 It is clear that we have much to learn about which measures to target, how to collect the data, and how to promote improvement at a reasonable cost and with a minimum of unanticipated consequences.26

Once we settle on appropriate measures, the next steps involve promoting and supporting improvement and making it stick. Although one might hope that clear feedback will generate meaningful improvement in a provider's performance, experience has shown that this strategy leads to only modest change. Increasingly, a more aggressive strategy of transparency (disseminating the results of quality measures to key stakeholders) is becoming the norm. With simple transparency, the hope is that providers will find the public dissemination of their quality gaps of sufficient concern to motivate improvement. Further, many believe that patients (or their proxies, such as third-party payers) will begin to use such data to make choices about where patients receive care. To date, although there is little evidence that patients are using such data to make purchasing decisions, studies have shown impressive improvements in some publicly reported quality measures, supporting the premise that transparency itself may generate change.23 Berwick, James, and Coye's framework (Figure 3-1) is useful for considering the two interwoven pathways—selection by patients or purchasers, and change undertaken by the provider organization—through which measurement and transparency can stimulate improvement.27

The newest strategy to motivate improvement is to tie payments for service to quality performance (Pay for Performance, or “P4P”).28 Although P4P is conceptually attractive, it also raises a host of questions: Are quality data accurate? Should payments go to best performers or to those with the greatest improvements? Do existing measures adequately assess quality in patients with complex, multisystem disease? and does P4P erode professionalism or create undue focus on measurable practices and relative inattention to other important processes that are not being compensated?28,29 At this writing, various flavors of P4P (Table 3-511,30–33) are being implemented, and tested, in a variety of settings. Early evidence from these experiments indicates that P4P may produce only a little more change than that generated by transparency alone (Figure 3-2); the jury is still out on whether this marginal benefit (if it in fact exists) is enough to overcome the concerns cited above.29,34

Whether the motivation is ethics, embarrassment, or economics, the next question is how to actually improve the quality of care. There is no simple answer. In general, most institutions and physicians with success in this area use a variation of the “Plan-Do-Study-Act” (PDSA) cycle (Figure 3-3), recognizing that QI activities must be carefully planned and implemented, that their impact needs to be measured, and that the results of these activities need to be fed back into the system in a continuously iterative process of improvement. A number of more complex and sophisticated methods (Lean, Six Sigma, Toyota Production System) have been developed in the business, manufacturing, and engineering worlds, and have been adapted to healthcare, often (but not always) successfully.35,36 A detailed discussion of these methods is beyond the scope of this book; interested readers are referred to several excellent resources.37–39

In addition to the PDSA cycle, several other tactics are useful. For QI practices that require predictable repetition, efforts to “hard wire” the practice or use alternative providers who focus on the activity are often beneficial. For example, the strategy most likely to increase the rate of pneumococcal vaccination of hospitalized patients with pneumonia is to embed it in a standard order set (either paper-based or computerized).40 Having a nurse remove the patient's shoes before the doctor enters the room can increase rates of diabetic foot examinations in outpatient practice. Empowering critical care nurses or respiratory therapists to follow a weaning protocol safely shortens the time patients with respiratory failure spend on mechanical ventilators.41

In some areas, though, QI involves much more complex and interdependent activities. In these circumstances, bringing teams together to examine their practices and participate in a PDSA cycle is the most likely path to success. For example, a group of cardiac surgeons in the northeastern United States participated in a multicenter QI project in which they observed each other's care, agreed on best practices, scrutinized each other's outcomes, and suggested improvement strategies. The result: a 24% reduction in cardiac surgery mortality.42 This kind of approach is likely to be facilitated by team training and is in keeping with what we have come to understand about improvement in complex adaptive systems of care (Chapters 2 and 15).43

Although patient safety is a subset of the larger issue of quality of care, it is important to appreciate the sometimes subtle differences between the two areas, particularly when it comes to measuring performance and changing practices and systems.

Mr. S, a 74-year-old man, is admitted to the hospital with severe substernal chest pain. In the ED, his electrocardiogram (ECG) shows the ST elevation typical of a significant myocardial infarction. In this hospital, ST-elevation myocardial infarctions are managed with emergent balloon angioplasty, and there is strong evidence linking “door-to-balloon times” to ultimate outcome. There is a delay in reaching the cardiologist on call; when he finally arrives an hour later, the cardiac catheterization lab is not prepared for the procedure, leading to another delay. The cardiologist, Dr. G, orders a dose of metoprolol, a beta-blocker. Dr. G's handwriting is difficult to read, but the pharmacist is reluctant to page the doctor, who is known for his “difficult personality.” So the pharmacist takes his best guess at the prescription and dispenses a dose of metformin, a medicine for diabetes. Ultimately, the mistake is recognized and the correct medicine is administered as the patient is being wheeled up for his angioplasty. The door-to-balloon time is 150 minutes, well above the goal of 90 minutes or less. The patient survives, but is left with a moderate amount of heart damage (ejection fraction 35%, normal 55%–70%) and mildly symptomatic heart failure.

This case illustrates both quality and safety problems. The administration of the wrong medicine (metformin instead of metoprolol) is clearly an error. Future approaches to preventing such errors would likely include computerization, standardization, and changes in culture to ensure that the pharmacist would promptly call the physician to clarify an ambiguous order. But the prolonged door-to-balloon time also represents a quality problem, a failure in the process of care. There were no overt errors in this process, it just took far longer than it should have because of lack of coordination, planning, and training. The combined result of both kinds of problems was a poor outcome.

Differentiating these quality and safety problems is important as we consider their causes and how best to prevent them. Let's assume that the patient's insurer was interested in measuring the quality and safety of care in this hospital. It would be relatively easy to implement a transparent, auditable process to document the door-to-balloon time. Ditto for whether the patient received aspirin, a beta-blocker, or a flu shot at discharge. Turning to outcomes, it would be straightforward to figure out whether Mr. S and similar patients were alive or dead at the time of discharge (though remember that we'd need some pretty sophisticated case-mix adjustment to be sure that our outcome assessment wasn't unfairly disadvantaging hospitals or doctors that attract sicker patients). Public reporting of door-to-balloon times, perhaps accompanied by higher reimbursement for better performance (via P4P), would likely spur improved performance.

But how could this insurer learn of Mr. S's medication error? It is difficult to imagine a transparent, auditable and efficient system that does not depend on self reporting by the nurse, pharmacist, or physician to capture the error. Moreover, a strategy of public reporting (or P4P) for medication errors might well leave providers reluctant to report them, causing the system to be unaware that such errors were occurring and ill prepared to generate the shared knowledge—as well as the money and will—to fix them.

The process of fixing both problems (door-to-balloon times and medication errors) might be relatively similar, in that both are likely to require changes in core processes (all patients with chest pain will receive ECGs within the first 5 minutes of arriving in the ED; all prescriptions will include medication name, dose, and indication), thoughtful implementation of technology (perhaps an automatic ECG reader for the chest pain protocol; CPOE for the medication error), and changes in culture (an unwavering commitment to change by system leaders; simulation training to improve door-to-balloon time performance; teamwork training to improve physician–pharmacist communication and dampen down hierarchies for the medication error).44

The bottom line is that patients have a right to expect care that is both high quality and safe. The IOM report on medical errors, To Err is Human, helped catalyze a national push to improve safety, and has led to important changes in culture, training, regulation, and technology.45 However, because measuring safety tends to depend largely on providers' self-report (although this may change over time with the computerization of medical records and refinements in measurement strategies such as trigger tools [Chapters 1 and 14]), the twin strategies of transparency and differential payment are more likely to improve quality than safety.46 Happily, the two endeavors have enough commonalities (the need for improved information systems, standardization, and simplification, use of multidisciplinary teams, and improvement cycles) that QI initiatives will often result in better safety. Other cornerstones of patient safety, such as the creation of a safety culture (Chapter 15), may be largely independent of QI efforts and will require a distinct focus. And all of these quality and safety improvement activities are relevant to the increasingly important work of removing waste and unnecessary cost from our healthcare system.

Outside the world of healthcare, we base most of our purchasing decisions on our perception of value: quality divided by cost. Who among us is rich enough to always buy the very best thing, or frugal enough to always buy the least expensive one? Much of the focus of our lives as consumers is spent trying to determine the value of the things we're considering buying—whether it's a car, a house, a café mocha, or a college education—by weighing the measured (or perceived) quality against the cost. As we do this, we ask a simple question: is the item or service worth the price?

Healthcare decisions have traditionally not been made this way (I sometimes refer to this as “healthcare exceptionalism”), partly because of the limited ability of patients (and their doctors, for that matter) to make rational judgments about the quality of a given physician, surgeon, or hospital, and partly because one of the functions of healthcare insurance is to shield us from shouldering the full cost of our purchasing decisions. In other words, when you're buying your Starbucks coffee or an Audi sedan, you're well aware of the price and are in a perfect position to judge whether the quality (and safety, in the case of the car) is worth it. But in healthcare, the service is often “covered” or the patient is responsible for only a portion of the cost.

Much of the decade-long revolution in quality measurement and reporting can be seen as an effort to provide patients (or other interested parties, such as, in the United States, employers or insurers or, in countries with single-payer systems, the government) with the information they need to make rational decisions about healthcare value. This framework becomes even more important as the costs of healthcare continue to rise, crowding out resources for other worthwhile private and public pursuits. Increasingly, efforts to improve quality, safety, and patient satisfaction are not taking place in a cost-agnostic vacuum. Rather, healthcare providers and organizations will be pushed relentlessly to improve both the numerator and the denominator of the value equation.12

In this environment, we can already see the incentives shifting, from promoting the best care to promoting the best care at the lowest cost. Both Medicare's “value-based purchasing” and “no pay for errors” initiatives—in which anything but perfect performance leads to reimbursement cuts rather than additional payments for top performers—may well be early indications of the shape of the “pay-for-performance” initiatives of the future.11,32

• Safety is usually considered a subset of quality, but it is more difficult to measure, in part because identification of incidents often depends on self-reports by caregivers.
• Quality measurement is best thought of in terms of “Donebedian's Triad”: structure, process, or outcomes. Each type of measure has advantages and disadvantages when compared with the others.
• Measures being employed for public reporting and pay for performance (“Accountability Measures”) must meet a higher standard than those used for internal QI.
• There is tremendous activity in the healthcare marketplace promoting transparency (i.e., public reporting) and differential payments based on performance (“pay for performance”). The latter efforts are taking on multiple forms, including so-called “value-based purchasing” and “no pay for errors” initiatives.
• Cost pressures in the United States and elsewhere mean that the healthcare system's target will no longer be quality or safety alone, but value—quality (including safety, access, and the patient's experience) divided by cost.