Chapter 13. Interpretation of Epidemiologic Literature

• The literature on a topic of clinical interest often includes conflicting or inconclusive studies, thereby requiring the development of critical skills for interpreting the results.
• A systematic approach to evaluating the features of individual studies can provide thoroughness and consistency to a review of the literature.
• The evaluation of an individual study should include the research hypothesis, the study design, the predictor and outcome variables, the methods of analysis, possible sources of bias, and interpretation of results.
• In considering whether a statistical association is likely to represent a causal relationship, one should consider the strength of the association, the presence of a dose-response trend, correct timing of events, consistency across studies, and biological plausibility.
• A meta-analysis is a quantitative systematic review in which the results of multiple studies are combined to obtain a precise, and hopefully unbiased, estimate of the association under study.

In responding to the patient’s questions about breast cancer, the physician confirmed that a positive family history increases the risk of developing this disease. A number of other characteristics are associated with a reduced risk of developing breast cancer, such as early age at first full-term pregnancy and increasing number of pregnancies. Unfortunately, these factors are not easily susceptible to intervention, and the patient already had completed her childbearing. The physician was also aware of a controversy regarding the relationship between the intake of dietary fat and the occurrence of breast cancer. Before recommending that the patient reduce her fat intake, however, the physician wished to review the pertinent medical literature.

The recommendations that physicians make to patients depend on the current state of knowledge available about diseases, the underlying pathophysiology of diseases, and the most effective treatment for diseases. The knowledge base of clinical medicine is continuously expanding, and physicians must therefore develop methods to seek out and apply new information.

The first step in acquiring new medical knowledge is to locate the appropriate literature. This is an increasingly difficult task as the number of medical journals increases each year. It is not possible for any physician to read, as it is published, everything relevant to his or her practice. Fortunately, help of various kinds is available to assist with literature searching when necessary.

With the advent of computer technology, it has become possible to search the medical literature in an automated manner. For example, a search through a standard database, such as Medline, could be performed to assemble lists of articles that might be of interest. Medline is a database of articles published in any one of 4600 biomedical journals. The earliest citations in this database are indexed from 1966, with coverage up through the present time. Although journals published from around the world are included in Medline, the vast majority are English language publications. The National Library of Medicine has made the Medline database available through the Internet at the following address: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi

The search process is based on a few key words chosen to recover articles on relevant subjects. The search can be limited to a specific time period, which is helpful if you want to focus on the most recently published information. For example, a recent Medline search for the title words “breast cancer and dietary fat” retrieved about 70 journal articles over the previous 2-year period. These articles included a range of investigations, including laboratory animal studies, descriptive studies, case–control studies, cohort studies, randomized controlled clinical trials, and reviews, including meta-analyses. Most computer databases include abstracts of articles from major journals, and some even include the entire text of articles from selected publications.

Research Hypothesis

Once the purpose of the study is discerned, the reader should attempt to determine if the study addresses a question that has clinical importance. If the study does not, the results may have little relevance to clinical practice. For the physician who needs information to counsel a patient about the relationship of dietary fat and risk of breast cancer, it is necessary to identify articles that address that topic. A number of different kinds of hypotheses, however, may be relevant to the general topic. For example, a study of the effects of varying dietary fat composition on the occurrence of mammary tumors in mice may be useful, because studies conducted in laboratory animals can be more tightly controlled than studies conducted in humans. It may be useful to read a study on the effect of a high-fat diet on circulating estrogen levels in women, as this may have relevance to the biologic plausibility of a potential relationship between dietary fat and breast cancer. This type of research may yield information about the mechanism of disease development and prevention.

The various types of significance that may be ascribed to a research finding should be distinguished (Table 13–2). It is common to refer to results as significant if a statistical test indicates that the findings are unlikely to be attributable to chance alone. The evaluation of statistical significance—and therefore the likelihood of committing a type I (false-positive) error—is useful in the interpretation of results.

Even if a finding is statistically significant, however, it may not be biologically or clinically important. For instance, a small difference in the risk of developing breast cancer with increasing levels of dietary fat could be judged to be statistically significant if the finding were based on a large number of observations. Nevertheless, this elevation in risk may be so small that an individual woman’s risk of developing breast cancer would not be appreciably altered by changing her diet. As a result, a clinical recommendation to reduce dietary fat might not be supportable on the basis of the evidence. The biological significance of a finding addresses yet another issue: Do the epidemiologic observations help to clarify the causal mechanism? This type of insight is most likely to be gained if the epidemiologic study involves biological markers of exposure, susceptibility, and outcome.

Study Design

If the study question is of interest, the reader should then determine what type of study design was employed. As noted in Chapters 7: Clinical Trials, 8: Cohort Studies, and 9: Case–Control Studies, certain designs may be more or less useful in answering specific kinds of questions. Another factor that may have determined the type of study design used by the investigator is the current state of knowledge. Early studies of a particular hypothesis may have a simple design, such as a descriptive study. As the hypothesis is refined, more definitive study designs can be utilized.

The appropriateness of the study design to the research question should be assessed. The incidence rate of the disease in question may be a determining factor. For example, although breast cancer is the most common form of cancer among women in the United States, this disease is diagnosed among only a small proportion of women during a short period of time. Accordingly, a case–control study would offer an efficient approach to studying this disease, since the sampling scheme for this type of study identifies affected women once they are diagnosed. In fact, studies of dietary fat intake and occurrence of breast cancer have utilized several different designs, including descriptive, case–control, and cohort studies. The descriptive studies are useful for hypothesis generation, but not for hypothesis testing. The case–control and cohort designs provide more compelling evidence to test specific hypotheses. To date, all of the published studies of dietary fat and risk of developing breast cancer in humans have employed observational designs. A large, multicenter experimental study in which women are randomly assigned to groups receiving different levels of dietary fat and followed for the occurrence of breast cancer is underway in the United States, but this is a long-term study and results have not yet been published.

Outcome Variable

The outcome of interest in the Patient Profile is the development of breast cancer. In investigations of the relationship between dietary fat intake and risk of developing breast cancer, it is important to specify how the presence or absence of breast cancer was determined. There are several possibilities.

• 1. Death certificates limit information to deceased subjects. In addition, a variety of studies have shown that information on death certificates may be incomplete or inaccurate, as discussed in Chapter 4: Medical Surveillance.
• 2. Self-reports require that subjects be alive or have relatives who can provide information on breast cancer. If the subjects are not medically sophisticated, they may mistake benign forms of breast disease for breast cancer.
• 3. Medical records may provide more accurate information. However, it is possible that diagnostic criteria differ from physician to physician, over time, or across geographic regions or countries.
• 4. Histopathologic diagnoses provide the most definitive information, but adequate tissue must be available for pathologic examination.

It is desirable to have the most definitive information possible on the presence of disease. This will tend to minimize the likelihood of misclassification of subjects. For breast cancer, it is possible that a small proportion of apparently healthy women may actually have occult (undiscovered) breast cancer. This could be evaluated by performing a screening test, such as mammography, on all apparently unaffected subjects. However, because so few asymptomatic cancers are likely to be detected, study findings probably would not be affected greatly by limiting detection to routine histopathologic diagnosis.

It is also important to judge how precisely the investigator defines the outcome. In general, it is useful to specify a single disease entity when searching for causes. For example, a study of dietary fat and the risk of all cancers combined may produce misleading results, as different cancers have different causes, and only some causes may involve dietary fat. Restricting the study to breast cancer improves the likelihood of obtaining a definitive result for this disease of primary interest.

Predictor Variables

The predictor variable is the risk factor or exposure under investigation. Studies may involve a single risk factor of interest or several different predictor variables. If a number of exposure variables are included, they may or may not be closely linked.

In a study of the cause of breast cancer, an investigator might choose to examine a variety of exposure variables, including reproductive factors such as age at first full-term pregnancy, hormone levels, exposure to radiation, and dietary fat intake. Although this sort of study may provide a more comprehensive picture of the causes of breast cancer, it may limit the ability to collect detailed information on each exposure of interest. Even if a study is focused on the question of dietary fat and the risk of developing breast cancer, it is necessary to collect some basic information on other possible determinants of breast cancer that could act as confounders.

The reader must determine whether the methods used to characterize the presence or absence of exposure are reliable and accurate. Possible methods of ascertainment include subject or surrogate respondent reports, direct observation, or measurement of a biological marker. The reader should ask whether there are better ways to define the exposure levels of subjects.

The assessment of a dietary exposure can be particularly difficult. In one method, asking subjects about their past dietary habits, the subjects must remember the kinds and the amounts of foods that they ate. Various studies have indicated that such recall, although imperfect, may suffice to determine whether a subject consumed a relatively high, moderate, or low amount of dietary fat. Generally, it is desirable to have several levels of exposure defined, so that a dose–response relationship can be evaluated.

Another approach to determining dietary fat intake is to have subjects record what they currently eat. This can be done by keeping a diary or by using a list of commonly eaten foods to check off the meals and snacks being ingested. There are problems with this approach, however. Subjects may forget to record what they eat, or they may incorrectly estimate the size of portions. To help offset such problems, plastic models of different portion sizes are available to provide visual cues and reminders. It is important to remember that a subject’s current diet may not accurately reflect their past diet. In a case–control design based only on current diet information, it is crucial to know if subjects with breast cancer changed their diets because of (1) the disease, (2) the side effects of treatment, or (3) the hope of influencing prognosis.

Another approach to collecting information on diet is to measure what subjects eat. This could be useful for a prospective cohort study in which subjects are followed to determine whether they develop breast cancer. However, this approach to measuring dietary intake would be extremely difficult in practice, as it would require a tightly controlled environment in which the investigator could observe the foods eaten by subjects.

Epidemiologists occasionally take advantage of a situation in which people maintain certain dietary habits for religious or other reasons. Thus, an epidemiologist may identify a group of people (eg, vegetarians) who consume very little fat in their diets. The frequency of occurrence of breast cancer in this group could be compared with the experience of another group whose members consume large amounts of fat. However, the problem of determining the precise intake of fat for subjects in both groups still remains. Furthermore, it is likely that the groups will differ in life-style factors other than intake of dietary fat, giving rise to potential confounding.

The use of biological markers of exposure has become more common in clinical research (see Chapter 10: Variability & Bias). Biological markers are important because they can provide quantitative documentation of exposure in certain circumstances. No biological markers of fat intake are currently available, but to assess long-term intake of dietary fat, the fatty acid content of adipose tissue could be measured in biopsies. Obviously, the utility of such a measure depends on the extent to which it accurately reflects consumption patterns. The willingness of study participants to undergo a tissue biopsy must also be considered.

Methods of Analysis

The emphasis placed on a particular research finding often depends on the ability of the investigator to exclude chance as an explanation for the observed results. This is accomplished by the use of statistical tests. Readers should have a basic understanding of which statistical tests are appropriate for which types of analysis. The type of statistical test that should be used is determined by the goal of the analysis (eg, to compare groups, to explore an association, or to predict an outcome) and the types of variables used in the analysis (eg, categorical, ordinal, or continuous variables).

By convention, the 5% level of statistical significance is used as a standard in many biomedical studies. That is, the investigator is willing to accept a 1 in 20 risk that the observed effect is a result of chance variation alone. However, care must be taken to avoid oversimplistic interpretations of p values. One common mistake is to assume that a statistically significant result is biologically or clinically important. As discussed above, the clinical importance and biological plausibility of results are not assessed by hypothesis tests.

A second common mistake is to dismiss a finding because it has not reached the predetermined level for statistical significance. A p value of 0.08, for example, although not statistically significant in common practice, still represents a finding that is relatively unlikely to be attributable to chance. It would be unwise, therefore, to conclude on the basis of such a p value that there is no relationship between dietary fat intake and development of breast cancer in a particular study. The heavy reliance on p values is particularly dangerous when the sample size of a study is small and the statistical power is therefore low as well. In such situations, even moderate differences between groups may fail to reach a conventional level of statistical significance, and the ability to reach a definitive conclusion is limited.

Possible Sources of Bias

A result must also be examined to determine whether it could be due to systematic errors related to the sampling strategy or to data collection procedures. A statistical test cannot address whether biases of any sort are responsible for the observed results. As discussed in Chapter 10: Variability & Bias, the consideration of potential bias often cannot be assessed in precise quantitative terms. Bias can occur in any study, although certain designs are more susceptible to specific types of bias. Regardless of the study design, the potential for three distinct types of bias should be considered (Table 13–3).

The first concern is whether the selection of subjects is likely to have distorted the results (selection bias). Because few studies (if any) can examine an entire population, the investigator must draw a sample to make inferences about the population. The reader must determine if the samples are likely to be representative of the population to which extrapolations are made.

The methods section of any published medical research paper should include details of how subjects were selected for the study. In a case–control study, selection bias could arise from the approach used to select cases, controls, or both. In the context of a case–control study of dietary fat and occurrence of breast cancer, selection bias might occur if prevalent (surviving) cases are used rather than newly diagnosed (incident) cases. This distortion would occur if prediagnostic nutritional status were related to disease prognosis. For example, if women who consumed high-fat diets before developing breast cancer survived longer than women who consumed low-fat diets, prevalent cases would overrepresent high-fat consumption, and the observed risk ratio may be biased toward larger values. The bias is presented schematically in Figure 13–2.

In a case–control study, the sampling of controls can be as great a source of selection bias as the sampling of cases. Consider a hospital-based sample of control subjects with diagnoses other than breast cancer. If the diseases of the control subjects are caused by dietary factors (eg, atherosclerotic heart disease) or, conversely, if the diseases influence dietary intake (eg, gastrointestinal disease) a biased case–control comparison may result. Sampling from the general population typically results in control subjects who better reflect the exposure patterns of persons without the disease of interest. However, even general population sampling schemes can result in a distorted control group. For example, a telephone sampling technique might preferentially include higher income women, because they are more likely to have telephones than lower income women. Also, if higher income women are less likely to work outside of the home, they would be more available when the sampling is performed. Because the diets of higher income women may differ from those of other women, a biased case–control comparison may occur. To determine whether a telephone sampling scheme is likely to yield an unbiased sample of control subjects, it is necessary to know the completeness of telephone coverage of the target population. Also, multiple calls should be made to sampled residences, at varying times of the day and days of the week, to reach persons who work outside the home.

Cohort studies of the association between dietary factors and occurrence of breast cancer are also subject to potential selection bias. The major source of selection bias in such studies is loss to follow-up during the study. If women who eat a high-fat diet and develop breast cancer tend to discontinue participation in the study for some reason prior to the diagnosis of cancer, the investigator will underestimate the risk of developing breast cancer in women whose diets are high in fat. To date, cohort studies of this question have yielded conflicting results. An important consideration in such a situation is the extent to which bias due to loss to follow-up could explain the discrepancy.

Another source of bias can arise from systematic errors in measuring either the independent variable (exposure) or the dependent variable (disease). This type of bias is often referred to as information bias or misclassification bias. For example, in a case–control study, the validity of information on exposure may be questioned because the data are gathered retrospectively. Because the cases are aware of their disease and have undergone treatment for it, their reporting of past exposures may differ systematically from the reporting of control subjects. This is referred to as recall bias. Studies of dietary risk factors may be susceptible to recall bias. For example, if patients with breast cancer have wondered why they developed their cancer, they may tend to overestimate past exposure to dietary fat, particularly when the potential relationship has been widely publicized. Control subjects may be less concerned about past diet or may be worried about other problems such as obesity, which would tend to make them underreport exposure to dietary fat.

If recall bias occurred in a study, the investigator might overestimate the relationship between dietary fat intake and the risk of developing breast cancer. In fact, a number of studies have demonstrated problems of imperfect recall of dietary history. Only a few studies, however, have examined differential (biased) recall in cases versus controls by comparing prospectively collected dietary data with subsequent data collected retrospectively from the same subjects. In general, these investigations have not demonstrated differential recall of food intake in cases of breast cancer compared with controls. This would suggest that recall bias is an unlikely explanation for inconsistent results reported from case–control studies of the association of dietary fat intake and occurrence of breast cancer.

It should be remembered, however, that even if cases and controls do not differ in the ability to recall dietary exposure, misclassification bias still could occur. Errors in reporting that are comparable between cases and controls give rise to nondifferential misclassification. If nondifferential misclassification occurs, it may reduce the estimated risk ratio. In other words, such misclassification tends to make it more difficult to detect any true differences between cases and controls.

The final consideration of bias is to determine whether confounding could account for the observed result. A confounder is an extraneous correlate of disease that, because of its association with the risk factor of interest, accounts for some or all of the observed association between the risk factor and the disease. In studies of dietary fat intake and occurrence of breast cancer, it is important to determine if the investigator has accounted for the effects of known risk factors for breast cancer. These factors include age, race, reproductive characteristics (eg, age at first full-term pregnancy, number of pregnancies, duration of lactation), obesity among postmenopausal women, alcohol intake, and exposure to radiation. A schematic diagram for confounding of the relationship between dietary fat intake and occurrence of breast cancer by number of pregnancies is presented in Figure 13–3. If women who eat a high-fat diet have fewer pregnancies than those who eat a low-fat diet, an apparent association between consumption of dietary fat and occurrence of breast cancer could be attributable to the effects of reproductive history rather than to diet per se.

In an observational study, confounding can be controlled either in the design of the study (by restricting subject inclusion to persons with a narrow range of the confounder values or by matching study groups on confounders) or in the analysis of the results (through stratification by confounders or by regression techniques). All of these adjustment methods, however, are contingent on knowing which variables are confounders. Because the known risk factors for developing breast cancer do not account for all occurrences of the disease, other unknown risk factors must exist. Any observed association between dietary fat intake and occurrence of breast cancer could be explained, at least in part, in that way.

When the reader detects the potential for bias, it is important to try to estimate both the magnitude and the direction of the effect the bias could have on the results. In this way, the reader can determine if the bias is likely to have inflated the results or to have diminished the exposure’s effect. In a case–control study, for example, if it is suspected that women with breast cancer are more likely than control subjects to remember and report fat intake, the risk ratio could be overestimated. In contrast, if it seems likely that patients with breast cancer underreport their dietary fat intake in comparison with control subjects, the observed results may underestimate the true impact of dietary fat intake on the occurrence of breast cancer. In discussing the results, an investigator may attempt to convince the reader that the magnitude of bias would not be sufficient to skew the results, or that the true relationship is as strong as or stronger than that observed.

Interpretation of Results

If the investigator reports a statistically significant result that cannot be explained by bias, the reader must then decide whether the result is clinically important. Consider, for example, a study concluding that a 50% decrease in dietary fat intake is associated with a 5% decrease in risk of developing breast cancer. Even if this result is statistically significant, the magnitude of the reduction in risk is so slight in exchange for the major change required in diet that individual patients may be poorly motivated to make the dietary change. On the other hand, the benefit to society in eliminating 5% of all breast cancer cases may well justify a mass public education effort to reduce consumption of dietary fat.

Conversely, results that are not statistically significant need not be considered useless. In particular, when there is a small sample size or when the relationship between the exposure and the disease is weak, the possibility of a false-negative conclusion must be considered. The statistical power of the study to detect the observed effect may be too low to allow a definitive conclusion from the study.

Practical Utility of Results

When reviewing a published study, the reader must determine the practical utility of the results. The usefulness of a study finding depends on various factors, including the purpose of the study, the limitations of the study population, the clinical and biologic importance of the results, and consistency with findings from other published studies. Clinical and epidemiologic research has various purposes. The clinical utility of a particular research finding must be viewed in the context of the type of question posed. As indicated in Table 13–4, a particular study may lead to findings relevant to disease causation, early detection of disease, the prediction of prognosis, or improved treatment. Studies of the relationship between dietary fat intake and occurrence of breast cancer relate to disease causation. Unfortunately, there is no standard by which to judge whether an association between a risk factor and a disease is clinically important. Clearly, the stronger the association (ie, the farther the risk ratio is from the null value), the greater the potential impact of eliminating the exposure. In assessing clinical utility, consideration must be given to how difficult it is to change the risk factor (in this case, to reduce dietary fat intake) and to the amount of morbidity and mortality associated with the disease.

The ability to generalize the findings beyond the study population should be taken into account. For this purpose, the definition and limitations of the study sample must be understood. For example, some studies of risk factors for developing breast cancer have focused on postmenopausal women. This may limit the applicability of such studies to the premenopausal patient in the Patient Profile. Investigators are often forced to restrict the sample by age, race, or other factors. The reader must decide what effect these restrictions may have on the broader applicability of results.

In determining whether the findings of a particular study can be generalized to other populations, it is useful to assess whether similar results have been obtained in other studies. It often happens that the first evaluation of a risk factor, diagnostic test, or therapeutic regimen is favorable, whereas subsequent reports demonstrate more limited utility. One reason for this pattern is that initial assessments often involve selected populations that offer a best-case scenario. Subsequent attempts to broaden the applicability may prove less successful.

The strength of the observed association is a primary criterion in evaluating whether a risk factor causes a disease. The strength of the association is indicated by the distance of the risk ratio or odds ratio from the null value. When the association is very strong, it is less likely that the association can be explained by chance or bias. Weak associations also may be causal, indicating only a lower risk of disease development. With a weak association, however, it is more difficult to exclude other factors and biases that may account for the relationship.

It is useful to examine whether there is a dose–response relationship between the proposed risk factor and the disease. If there is, increased levels of the risk factor will be associated with a greater risk of developing the disease (or with protection in the case of a beneficial factor). For example, as the level of dietary fat intake increases, the risk of developing breast cancer would be expected to increase if a causal relationship exists (Figure 13–4). However, the absence of a progressive, graded dose–response relationship does not preclude a causal relationship. For example, there may be a threshold above which the level of the risk factor confers increased risk. In this case, risk of disease will not be affected by changes in exposure below a certain level, but risk does vary with exposure at higher levels.

With any association, it is helpful to compare the findings with the results of other studies. If other investigators studying different populations in differing settings find similar results, a causal explanation is supported. However, the reader must be careful when judging consistency of results, because it is possible that the same flaw could lead to incorrect conclusions in several studies.

The proposed causal relationship should be consistent with what is currently known about biology and the disease process. This is often referred to as biological plausibility. If the proposed cause-and-effect relationship is not in accordance with current knowledge, causality may be questioned. The assessment of biological plausibility often requires a review of research on other human populations, as well as a review of research that involves laboratory animal models.

The temporal relationship between a suspected cause and an effect is important. That is, a cause must always precede an effect in time. This seems intuitive, but in reality, factors that are suspected to be causes sometimes turn out to be effects of the disease. For example, a person with an early undiagnosed cancer may make a change in food choices because of unrecognized systemic effects of the cancer. Consequently, the dietary change may appear to be the cause of, rather than an effect of, the later diagnosed cancer. Case–control studies of chronic diseases with long latent periods are particularly susceptible to this problem. For a factor to be considered the cause of a disease, it is theoretically important that removal or modification of that factor will prevent the disease from occurring or will ameliorate the disease once it has occurred. This criterion may have been met for the example cited in Chapter 9: Case–Control Studies, as the incidence of eosinophilia-myalgia syndrome appears to have declined with the removal of l-tryptophan from the market. However, surveillance for eosinophilia-myalgia syndrome is not as active as it was when the syndrome was first reported, so unreported cases of eosinophilia-myalgia syndrome may still be occurring. For example, cases of eosinophilia-myalgia syndrome continue to occur in Canada, where l-tryptophan is available only by prescription (Spitzer et al, 1995). This means that the true incidence of eosinophilia-myalgia syndrome is not currently known; nor do epidemiologists know the extent to which the incidence may have changed.

In practice, the criterion of removal or modification of a suspected causal factor may not always be satisfied. There are some instances in which a causal factor may set off a protracted chain of events. Once established, this sequence may no longer depend on the presence of the causal agent for progression. For example, many cancers are thought to develop in response to an initiating event, followed by promotional effects that occur for many years. If the risk factor of interest contributes to initiation only, removal of the exposure during the promotional phase will not affect the subsequent risk of developing cancer. Thus eliminating an initiating risk factor for cancer may not affect the incidence of this disease for many years into the future.

As mentioned previously, the relationship between dietary fat intake and the risk of developing breast cancer was investigated in a variety of epidemiologic studies. These studies included case series reports, ecologic analyses, case–control studies, and cohort studies. To date, no randomized controlled clinical trial of the relationship between consumption of a low-fat diet and prevention of breast cancer has been published.

Ecologic or correlation studies have demonstrated a consistently strong relationship between dietary habits, as estimated by per-capita consumption of dietary fat, and breast cancer occurrence in different countries. Plots of these data have yielded a linear relationship, with increasing fat consumption associated with higher breast cancer occurrence. The problem with such studies is that they do not demonstrate that increased dietary fat in individuals is associated with breast cancer occurrence in the same individuals (ie, the ecologic fallacy may be involved). For example, in industrialized countries in which fat consumption and breast cancer mortality tend to be higher than in developing countries, it may not be the high fat consumers who are developing breast cancer. The comparatively high mortality rates of breast cancer in industrialized countries may be attributable to other factors, such as earlier menarche, delayed childbearing, or other reproductive factors.

Case–control studies have yielded conflicting results on the question of diet and breast cancer. The reasons for these conflicting results are not clear. One criticism is that dietary data collected retrospectively are inaccurate (ie, dietary exposure is misclassified). General difficulties in recalling diet could contribute to nondifferential misclassification, and case–control differences in ability or motivation to recall dietary fat could result in differential misclassification. Another potential explanation is that the influence of dietary fat could have been exerted many years prior to the diagnosis of breast cancer. Most case–control studies have included diet information from the recent but not from the remote past. Some reviewers have concluded that none of the studies, when viewed individually, had a large enough sample size to provide adequate statistical power. In other words, a true relationship between dietary fat intake and risk of developing breast cancer might be missed because of inadequate discriminatory ability (ie, a type II error may be involved).

Some of these problems can be avoided with the cohort study design. Prospective cohort studies eliminate the potential for distortion of results from differential recall of dietary history. This type of study was used to investigate the question of fat in the diet and the risk of developing breast cancer. Again, the published results of cohort studies of this question have yielded conflicting results. Some studies have produced evidence of a relatively weak association, with risk ratios in the range of 1.2 to 1.7. Other cohort studies, however, have indicated little or no association. In general, the dose–response relationship suggested by the ecologic studies has not been borne out in the case–control and cohort studies. Some reviewers have argued that the range of dietary fat intake in the analytic studies is smaller than in the international ecologic studies. Within a single country, there may be too little variation in fat intake to demonstrate a dose–response relationship in case–control and cohort studies. Others have argued that factors other than diet (ie, confounders) that could not be controlled easily in ecologic studies were the real explanation for the association between fat intake and development of breast cancer observed in correlation analyses.

Was there biological plausibility to the relationship? What pathophysiologic mechanism might be involved? Were there animal models that showed the relationship between fat in the diet and development of breast cancer? In fact, animal studies have found an association between fat intake and the development of mammary cancers in mice bearing the mammary tumor virus. Similar findings have been reported in other animal models. The pathophysiology involved in this process is less clear, but potential mechanisms have been discussed. It has been speculated that mammary neoplasms are controlled by endocrine balance, which in turn is affected by dietary factors, including fat intake. For example, women consuming high-fat diets have been shown to have more circulating estrogen than women on low-fat diets. In postmenopausal women, adipose tissue has been demonstrated to be a contributor to the production of estrogen. Dietary fat intake may also have modified DNA synthesis and cell duplication. If this were found to be true for breast tissue, this could be relevant to breast carcinogenesis. In animal studies, certain types of fatty acid in the diet modulate mammary tumor growth and metastasis. The evidence is strongest for a promotional effect of polyunsaturated fat. The potential differential effect of various types of fatty acid on development and progression of cancer might explain some of the variation observed in studies of fat intake and breast cancer risk in humans. Most epidemiologic studies have evaluated total fat intake. Some studies have used adipose tissue levels of fat as a surrogate measure of fat intake. Adipose tissue levels, however, do not provide an indication of dietary intake of fatty acids that can be synthesized internally. The Nurses Health Study did include a prospective evaluation of different types of fatty acid intake. In this study, none of the various types of dietary fat was associated with the risk of developing breast cancer.

Systematic Reviews

• 1. By combining a series of smaller studies, each with a statistically imprecise estimate of effect, a larger sample size is obtained, with a corresponding increase in statistical precision.
• 2. By identifying the differences in findings across different studies, sensitivity analyses can be conducted that may lead to greater insight into the sources of heterogeneity.

The steps in a systematic review should follow a clear sequence. The first step is to formulate a clear and meaningful question to be addressed. Elements to be considered in the formulation of this question are (1) the type of person(s) involved, (2) the type of exposure that the person(s) experiences (eg, a risk factor, a prognostic factor, a diagnostic procedure, or a therapeutic intervention), (3) the type of control with which the exposure is compared, and (4) the outcomes to be addressed. In the context of the patient profile, we might specify the question in the following way: For premenopausal women with a family history of breast cancer, is reduction of dietary fat consumption substantially below levels typical of the American diet likely to reduce the risk of developing breast cancer?

Note that even with this level of specification, there is some ambiguity. The characteristics of the population could have been delimited further, by noting other characteristics of the patient, such as her age of 40, a history of having her first child after age 30, and a recent normal mammogram. The precise level of dietary fat reduction was characterized as substantial, but not quantified further. Although greater specificity is desirable in formulating the question, if the question becomes too specific, there may be an insufficient amount of pertinent literature available to address it. With insufficient data, it may be impossible to answer the question. As a general rule, features of the patient or the intervention that are thought in advance to affect the answer should be considered.

Once the question is formulated, the next step is to decide what studies to include. A pragmatic approach is to limit the source studies to those published in the English language. There is a possibility that this will introduce some bias, as studies with positive findings may be submitted preferentially to journals with wider circulation, which tend to be the English language journals. A further consideration is whether to limit the review to published studies, as these are peer reviewed and presumably of higher quality. As noted in Chapter 7: Clinical Trials, however, a publication bias might result, as studies with positive findings may be more likely to be submitted and accepted for publication. Of course, the identification of unpublished work may be incomplete.

The next step is to search for the studies of interest. As suggested earlier in this chapter, an automated search of an electronic database, such as Medline, is an efficient way to identify the relevant published literature. Such a search, however, is limited to the articles contained in the database, which will tend to exclude the proceedings of meetings that are not published in journals, doctoral dissertations, and journals that are not indexed.

Once the articles for potential inclusion are identified, they must be reviewed one at a time. Specific eligibility criteria for inclusion must be specified. The included studies should be directly relevant to the question under consideration. In the present context, they must include, at a minimum, a study sample including healthy premenopausal women, information on dietary consumption of fats, comparison of higher and lower fat intakes, and estimation of breast cancer risk. If a qualitative review is under consideration, a wide range of studies, including those involving laboratory animals, human case series, cross-sectional, case–control, and cohort studies, as well as clinical trials might be included. If there is interest in a statistical integration of data from the various studies, the analysis may be limited to studies in which the data are combinable (eg, case–control and cohort studies and clinical trials). The next step is to extract the key data elements from the included studies. A pretested standard abstract form should be used to collect the relevant information from each source paper. To ensure reliability, it is advisable to have two independent reviewers collect information from each paper.

The actual analysis of the data begins with an estimation of the effect of interest in each of the included studies. It is useful to display the individual study results as shown in Figure 13–5. In this graph, the results of five separate hypothetical studies of the relationship between reduced dietary fat intake and risk of developing breast cancer are presented. The findings of each investigation are summarized with a point estimate of effect and a corresponding confidence interval around that estimate. The point estimate is represented as a solid circle, with a horizontal line stretching in both directions to the upper and lower bounds of the confidence interval. By convention, the effect typically is measured in terms of either an odds ratio or a relative risk (risk ratio).

In Figure 13–5, the results are displayed in terms of estimated relative risk of developing breast cancer associated with a reduced level of dietary fat intake. If reducing fat in the diet decreases the risk of developing breast cancer, a relative risk less than 1 would be expected. The point estimates for the relative risks in these hypothetical studies ranged from a low of 0.6 (Study E) to a high of 1.3 (Study A). Four of the five studies (Studies B through E) had point estimates lower than 1, consistent with the possibility of a reduction in breast cancer risk. On the other hand, most of these estimates are close to the null value of 1 (no association between dietary fat intake and risk of developing breast cancer). It is possible, therefore, that these small benefits suggested by a reduction in fat consumption may have arisen by chance.

Examination of the corresponding confidence intervals for the individual studies provides some insight into the statistical precision of the results and whether they are statistically significant. By convention, 95% confidence limits typically are calculated. The odds or risk ratio often is displayed on a logarithmic scale as in Figure 13–5, so as to make the confidence intervals symmetrical around the point estimate.

It can be seen from Figure 13–5 that the precision of the five study results varied, with the narrowest confidence intervals (greatest precision) for Study D and the widest confidence intervals (least precision) for Study A. Only one of the five studies had a result that was statistically significant at the 5% level, as indicated by the fact that its confidence interval did not cross the dashed vertical line corresponding to the null value of one.

Statistical integration of the results of the individual studies allows calculation of a summary estimate of effect. On the bottom of Figure 13–5, the combined point estimate for the five hypothetical studies is shown as a diamond. The vertical points of the diamond are located at the point estimate of the summary effect, and the horizontal points are located at the upper and lower bounds of the 95% confidence interval, respectively. In this example, the combined point estimate was 0.86, with a corresponding 95% confidence interval from 0.70 to 1.05. The summary estimate is based on a larger sample size than any of the individual study results. Accordingly, the combined estimate is more precise, which is reflected in the figure by comparatively narrow confidence intervals. In this example, it can be seen that the combined effect is consistent with a small reduction in risk of developing breast cancer, but even with the large sample size of the five studies combined, chance cannot be excluded as a possible cause of the findings.

Once the individual and combined estimates are obtained, it is useful to consider the level of heterogeneity across the individual results. A graphic display of the data, as in Figure 13–5, provides some insight into this question. Although there is some variation in the individual results, the impression is that there is considerable overlap in the values of the relative risk that are consistent with each result. This would suggest that heterogeneity of findings is not a problem in summarizing these data. A formal test of heterogeneity can be performed to address this issue from a statistical perspective.

Sensitivity analysis can be performed to identify patterns of results across the individual study results and potentially provide insight into any heterogeneity that exists. For example, two questions may be examined: do studies of a particular design (eg, case–control studies) tend to yield results that differ from studies of other designs (eg, cohort studies), and do investigations with younger study populations tend to have results that differ from investigations with older populations?

In the present hypothetical example, it is unclear that a sensitivity analysis would yield great insights for two reasons. First, there are a relatively modest number of studies involved, which limits the ability to explore findings across subgroups. Any differences that were observed might arise on the basis of chance, leading to false-positive interpretations. Second, the results across studies are reasonably consistent, further limiting the ability to find different underlying patterns. From a descriptive point of view, it might be useful to consider possible reasons (other than chance) for an apparently different result in Study A.

Meta-analysis can be useful in achieving greater statistical power, but it cannot overcome the limitations and potential biases of individual studies. The investigator performing a meta-analysis must also be careful that selection of some studies and exclusion of others do not lead to a distorted conclusion.

A meta-analysis published in 2003 explored the relationship between dietary fat intake and risk of breast cancer. This systematic review included 45 studies (31 case–control and 14 cohort), with a combined total of over 25,000 breast cancer patients and 580,000 control or comparison subjects. An overall small increase in risk of breast cancer was associated with elevated total fat intake in both the case–control (OR = 1.14) and cohort studies (RR = 1.11). The combined association was statistically significant and was higher in the studies judged to be of better quality. Similar findings were observed in analyses of saturated fat and meat intake.

In the Patient Profile, the physician wanted to determine whether dietary fat intake is causally related to the development of breast cancer. The findings of studies were inconsistent, and the strength of association was weak at best, without clear evidence of a dose–response relationship. The temporal sequence of dietary fat intake and development of breast cancer appeared to be reasonable.

The physician in the Patient Profile, however, is left without a firm answer to the question about the relationship between consumption of dietary fat and risk of developing breast cancer. Even if research has demonstrated a small increase in risk with high-fat diets, it is unclear whether the patient can meaningfully reduce her risk of breast cancer by changing her diet. The physician may believe, however, that a low-fat diet is justifiable for other reasons, including reduction of risk for cardiovascular disease. In addition, there are other cancers, such as colon cancer, for which the protective effect of a low-fat diet may be beneficial.

This type of uncertainty is common in clinical medicine. Physicians often must weigh potential risks and benefits of an intervention and make decisions without complete information. The goal of medical research is to continue to provide better answers to important clinical questions. The best way to resolve the issue of the relationship between dietary fat intake and risk of developing breast cancer is through large randomized controlled trials comparing occurrence of breast cancer in women randomized to a low-fat diet with women randomized to a high-fat diet. Clearly, this type of study would provide the most definitive evidence about a causal relationship between consumption of dietary fat and risk of developing breast cancer. Such a study is underway, but until the results are available, the best evidence is that which has been obtained through observational studies. Much of the satisfaction derived from patient care relates to the ability to incorporate new knowledge into the practice of medicine. It is imperative to develop the skills required for a critical review of the medical literature to keep current on the state of information. This is a difficult task, but the reward is enormous when it results in improved patient care.

In this chapter, a structured approach to reviewing published epidemiologic studies was presented. The ultimate goal of this approach is to help integrate epidemiologic information into clinical practice. As a focus for discussion, the literature on dietary fat intake and risk of developing breast cancer is used to illustrate the evaluation process.

The initial step in reviewing the medical literature is to conduct a thorough search for relevant recent publications. Screening for appropriate articles can be accomplished through a manual or a computer-assisted approach. In either case, a correct choice of key words calculated to retrieve pertinent articles is essential. Computer searches can save time and consequently are extremely popular.

The actual review of a publication begins with the stated purpose of the investigation. For the medical practitioner, a primary consideration is whether a particular study addresses a clinically important question. Are the results likely to influence the delivery of patient care? In the present context, clinical importance might be assessed in terms of whether the findings support a recommendation to lower dietary fat intake so as to reduce the risk of developing breast cancer.

The design of an investigation determines the types of inferences that can be drawn from it. The most compelling evidence for cause-and-effect relationships is derived from randomized controlled clinical trials. Among observational investigations, prospective cohort studies generally are least susceptible to bias, followed by retrospective cohort and case–control studies. Descriptive studies are useful for generating research questions but not for testing hypotheses.

The measurement of outcome (ie, development of disease) as well as the documentation of exposure to risk factors can be complicated in observational research. Emphasis must be placed on obtaining the most accurate information possible, recognizing that the ideal often cannot be achieved. Whenever possible, assessment should be performed in a blinded manner and should be based on objective, standard criteria.

A distinction must be made between the concept of statistical significance, which can be evaluated by a hypothesis test, and clinical or biological significance. P values should not be used to indicate the importance of a finding, but rather the likelihood that sampling variability can explain the results. Clinical importance relates to whether a difference in outcomes observed between study groups is large enough to warrant a change in clinical practice. Biological significance refers to the extent to which findings help to elucidate the underlying biological processes.

Systematic errors may arise in observational research from the approach to sample selection, information collection, or confounding. It is usually impossible to determine whether a particular bias actually has occurred. Attention, therefore, is focused on strategies to minimize the likelihood that systematic errors will affect the study conclusions. When bias is suspected in an investigation, the exact level of distortion generally is unknown. However, the direction of the systematic error (ie, a tendency to either overestimate or underestimate the strength of association) may be clear. The assessment of whether an observed association is likely to be one of cause and effect is based on specific criteria, including strength of association, presence of a dose–response relationship, appropriateness of the temporal sequence of exposure and outcome, consistency of results across studies, and biological plausibility.

When applied to the literature on dietary fat and breast cancer, this process of reasoning leads to an inconclusive assessment. The most consistent supporting evidence comes from the least compelling types of study. In situations in which an association has been observed, the magnitude generally has been weak. Results have been somewhat inconsistent across studies, perhaps due in part to limitations of the study design and in part to the weakness of the relationship, even if it does exist. The ultimate test of this hypothesis—a randomized controlled clinical trial—has not yet been published.

As with many issues in medicine, the dietary fat–breast cancer hypothesis remains unresolved. Because little harm probably results from restriction of dietary fat intake and because other benefits may accrue (ie, a reduction in risk of cardiovascular disease), it may be reasonable for the clinician in the Patient Profile to recommend this change in dietary habits to the patient. Until more definitive information is available, however, the patient should be counseled that the effect of restricting dietary fat intake on the risk of developing breast cancer is uncertain.