Studies gain greater validity after they undergo peer review. In peer review, a panel of scientists reviews a study to validate or refute its findings, usually before publication, assessing the quality of the data and methodology. Peer review is designed for an industry to self-regulate to reduce fraud and poor quality research.
However, peer review has serious limitations. Statistician William Briggs is so skeptical of peer review that he describes it as “the weakest filter of truth that scientists have.” “Yet,” he continues, “civilians frequently believe that any work that has passed peer review has received a sort of scientific imprimatur. Working scientists rarely make this mistake in thinking.”(1) In fact, plenty of studies that have passed peer review were later discovered to be fraudulently produced or simply full of enough errors to require retraction.(2) Hence, peer review is helpful, but it alone isn’t sufficient to assure a study is completely sound.
(1) William M Briggs, “A Case Of Failed Peer Review: Dust And Death,” blog post on wmbriggs.com, October 20 2011.
(2) Many examples are available on Retractonwatch.com. It’s worth noting that retractions of scientific studies may indicate failure of peer review, but Retractionwatch.com bloggers explain it in a more positive light: retraction reflects a “self-correcting” element of science, they say.
