SCIENTIFIC OVERSIGHT FOR PEER-REVIEWED RESEARCH IS SO BAD THAT EXPERT SAYS IT NO LONGER MATTERS
Writing for MedpageToday a few days ago (Does Journal Peer Review Matter Anymore?), Dr. Milton Packer, a cardiologist and academic currently at Baylor, did a good job of explaining just how desperate the situation really is concerning scientific oversight of biomedical research. After talking about a journal we discussed just the other day (HERE — the Journal of the American College of Cardiology), he stated……
“Because most journals in medicine at the time were led by unimpeachable intellects, it was easy to keep up with scientific advances. For many, one only needed to read the New England Journal of Medicine each week and a few other journals each month and remember what they published. Readers trusted journals to be a reliable source of information. But 40 years later, these principles, philosophy, and practices have disappeared. The trust that physicians formerly placed in journals has evaporated. The reason: the peer-review process doesn’t work anymore.”
Firstly; None of this is a surprise if you follow my site. Most of you are already aware that half of all biomedical studies are never published (HERE), making it the easiest way to hide poor outcomes (outcomes that don’t cast drugs or devices in the sort of light that will boost sales). The other half of the research? It can’t be reproduced (HERE) — the hallmark of the scientific method. And as DR. IONNIDAS showed us, most of it is wrong anyway (and why would we expect any different anyway with no scientific oversight?).
As for Packer’s statement about the NEJM; while this was undoubtedly more true then than now, I think he’s looking at the past as we all tend to do; through the lenses of his rose colored glasses. Don’t forget that it was DR. MARCIA ANGELL, medical doctor and an editor of NEJM from 1988 to Y2K (she was Executive Editor in 1999-2000), whose blistering editorial, Is Academic Medicine for Sale? blew the lid off the scamming that had become the norm in EVIDENCE-BASED MEDICINE two decades ago, revealing that scientific oversight was a thing of the past then.
Secondly; if Packer’s premise is true, what the heck are we supposed to do? His suggestion? According to the header directly under his article’s title, “Packer thinks that readers must now be the decisive judges of quality.” Stop and let the magnitude of what he is saying sink in for a moment.
According to a person considered a top thinker in the field of cardiology, things are so bad across the board in the research side of the medical field that nothing that comes out of it can be trusted (see previous link) — a fact confirmed recently in a guest post by a renowned but anonymous research expert (HERE).
Read his short piece on the lack of scientific oversight and ask yourself whether or not you have the time, energy, or knowledge to look at research and make any sort of determination as to its validity. Here are some completely CHERRY-PICKED statements from Dr. Packer confirming all this.
“How easy is it to get good reviewers? It is impossible. Currently, most leading researchers in any given discipline routinely decline to be reviewers. Doing a good review takes hours, and they just don’t have (or won’t make) the time. Many simply say no. Others hand the work over to junior associates — without carefully reviewing their submitted opinions. Editors… must invite 10-15 people to find two or three who agree to review.
Even then, the reviews are often superficial and unhelpful. Some reviewers spend only a few minutes looking at the data, and make recommendations based on their fondness (or lack thereof) for the authors or for the conclusions — rather than based on solid standards of scientific examination. Editors often feel compelled to accept the opinions of the reviewers even if they are inadequate or biased. Editors are reluctant to overrule the reviewers, fearing that they will refuse to review again in the future. The desire to keep reviewers happy means that even minor revisions are returned to them for a final blessing, thus adding months to the peer-review process.
What happens when a paper is rejected? Typically, it makes little difference. The authors will instantaneously resubmit to another journal — without necessarily fixing any of the errors or limitations that led to the previous rejection. The process continues until some journal is willing to publish the work.”
As is frequently the case, the comment section contained as much insight as the paper itself. Not only were there plenty of physicians agreeing with Packer’s perspective, there was some disagreement as well, including someone beating that old drum from the 1970’s — that dietary cholesterol causes high blood cholesterol and subsequent hardening of the arteries — something that all but the most staunch adherents of ye olde tyme dietary theology abandoned years ago — or at least should have (HERE).
My conclusion? Trust medical research at your own peril. Who can you trust? As the late pediatrician Robert Mendelsohn stated so eloquently; when it comes to healthcare (especially taking care of your children), “one grandma is worth two MD’s“.
If you are enjoying our site, be sure to spread the wealth so that others can enjoy it as well. Even though my four children tell me that FACEBOOK is for “old people,” it’s still a great way to reach those you love and care about most with information that has the potential to change their lives. If you like what you are seeing, be sure to show us some love on Facebook today.
