Cross design synthesis : a new strategy for medical effectiveness research : report to Congressional requesters / United States General Accounting Office.

  • United States. General Accounting Office.
Date:
[1992]
Catalogue details

Licence: Public Domain Mark

Credit: Cross design synthesis : a new strategy for medical effectiveness research : report to Congressional requesters / United States General Accounting Office. Source: Wellcome Collection.

    25/128 (page 23)
    Chapter 1 Introduction and Review of Study Designs for Evaluating Treatment Effectiveness other cross-study differences, such as sample size, attrition, amount of treatment given.» Building upon work by Light and by Lipsey, meta-analysts in the medical area could investigate, for example, whether studies evaluating a treatment on healthier patients show greater (or less) benefit than do studies conducted on more severely ill patients or those with comorbidities. Rating the quality of each study that is a candidate for inclusion in a meta-analysis has been a focus of many working in this field (see T.C. Chalmers et al., 1981). Some meta-analyses have examined the relationship between the size of the observed treatment effect and the quality score of the study. Others have excluded studies deemed to be of lower quality or have, in effect, “written off” such studies (see Cordray, 1990b). Rubin’s (1990a) approach, described below, includes all studies. The most obvious limitation of meta-analyses is that the results do not extend beyond the set of studies that have already been conducted. A meta-analysis of randomized studies cannot cover elderly patients if no (or virtually no) elderly patients were included in these studies. And the stricter the inclusion criteria, the more limited the pool of qualifying studies and available results (see Cordray, 1990a, 1990b). As previously noted, many meta-analysts assessing medical treatment effects have excluded all nonrandomized studies; however, to capture more information, some have included more diverse studies. For example, American social scientists working in the medical area (Wortman and Yeaton, 1983) included results of both randomized studies and nonrandomized “quasi-experiments”; results were stratified by type of study design.'® Critics of meta-analysis fear that the weaknesses and biases of existing studies may be compounded when they are quantitatively combined. For example, individual studies on a topic—far from being independent—may have been conducted by a single researcher and his (former) students. A convergence of results across such studies gives the appearance that a certain finding has been independently confirmed; in reality, such convergence may be merely an artifact, arising from masked correlations 1°Meta-analyses focusing not on treatments but on risk factors for various diseases have combined data from epidemiologic studies, which are not randomized. One example of such a study is the meta-analysis of studies of the relationship of blood pressure to stroke and coronary heart disease (MacMahon et al., 1990). Another is the meta-analysis of studies of the relationship between alcohol consumption and breast cancer (Longnecker et al., 1988).