Assessment of funnel plot asymmetry and publication bias in reproductive health meta-analyses: an analytic survey

João P Souza¹^,2 , Cynthia Pileggi² and José G Cecatti¹

¹Department of Obstetrics and Gynecology, School of Medical Sciences, University of Campinas, Brazil

²Clinical Epidemiology Collaborative Group, Women's Integrated Health Care Center, University of Campinas, Brazil

author email corresponding author email

Reproductive Health 2007, 4:3doi:10.1186/1742-4755-4-3


Published:	16 April 2007

Abstract

Background

Despite efforts to assure high methodological standards, systematic reviews may be affected by publication bias. The objective of this study was to evaluate the occurrence of publication bias in a collection of high quality systematic reviews on reproductive health.

Methods

Systematic reviews included in the Reproductive Health Library (RHL), issue No 9, were assessed. Funnel plot was used to assess meta-analyses containing 10 or more trials reporting a binary outcome. A funnel plot, the estimated number of missing studies and the adjusted combined effect size were obtained using the "trim and fill method". Meta-analyses results that were not considered to be robust due to a possible publication bias were submitted to a more detailed assessment.

Results

A total of 21 systematic reviews were assessed. The number of trials comprising each one ranged from 10 to 83 (median = 13), totaling 379 trials, whose results have been summarized. None of the reviews had reported any evaluation of publication bias or funnel plot asymmetry. Some degree of asymmetry in funnel plots was observed in 18 of the 21 meta-analyses evaluated (85.7%), with the estimated number of missing studies ranging from 1 to 18 (median = 3). Only for three meta-analyses, the conclusion could not be considered robust due to a possible publication bias.

Conclusion

Asymmetry is a frequent finding in funnel plots of meta-analyses in reproductive health, but according to the present evaluation, less than 15% of meta-analyses report conclusions that would not be considered robust. Publication bias and other sources of asymmetry in funnel plots should be systematically addressed by reproductive health meta-analysts. Next amendments in Cochrane systematic reviews should include this type of evaluation. Further studies regarding the evolution of effect size and publication bias over time in systematic reviews in reproductive health are needed.