OAKLAND RAIDERS ARE PROFESSIONAL FOOTBALL’S GREATEST TEAM: SO SAYS MEDICAL RESEARCH
“One can lie with statistics, because numbers can be manipulated to support any argument. Statistics, done honestly, can make a statement like no other, but done dishonestly are deeply deceptive because the readership believes the numbers have been run up honestly. Medical statistics is not any exception.” From a 1997 issue of Research Gate (How to Lie with Statistics or How to Extract Data from Information)
“So it is with much that you read and hear. Averages and relationships and trends and graphs are not always what they seem. There may be more in them than meets the eye, and there may be a good deal less.” Darrell Huff from his famous 1954 book, How to Lie With Statistics
Writing in yesterday’s edition of Medpage Today, Dr. Jeffery Keller penned an article so audacious that the title alone would make anyone with even a cursory knowledge of professional football (or for that matter, medical research) stop and do a double-take — How to Massage Data to Feel Like a Winner: Even the Oakland Raiders can Come Out on Top.
Even though the Raiders have the NFL’s classiest receiver, Kansas State’s Jordy Nelson, the truth is, they stink. Bad. As in get out the Lysol and spray the whole can. Playing in a division that was arguably the cream of this past season’s crop — the AFC West — the Raiders finished a lowly 4-12. Like I said, they were terrible. But in the same way that REVISIONIST HISTORY has become the new norm in politics, so it has in medical research as well.
Although I have spoken of all of the medical research community’s tricks-of-the-trade at various times in my EVIDENCE-BASED MEDICINE column; commenting on Dr. Keller’s article was too good to pass up. Today you’re going to learn that TOM BRADY really doesn’t have anything on Derek Carr!
“The Raiders can quickly and easily turn their season around by using the tried-and-true techniques of medical research. If a pharmaceutical company did 16 clinical trials of their new potential blockbuster, Drug X, they would never let a 4-12 outcome get them down. When published, I guarantee those trial results would look a lot better than 4-12. The Oakland Raiders can use the same techniques to improve their season record.”
Of course Keller mentioned INVISIBLE & ABANDONED studies, which I have dealt with extensively on my site. This is the ultra-common practice (50% of all medical studies) of simply not finishing or not publishing research that’s not conducive to selling your company’s products to the public. The RIAT Act (Reversing Invisible & Abandoned Trials) recently showed that when added to the published medical research on depression, the unpublished medical research — millions of pages worth — proved that ANTIDEPRESSANT MEDS ARE A WASH; no better than placebo. And the real mud in your eye is that 1/3 of everyone who takes meds is being prescribed meds that actually cause depression (HERE).
He also talked about changing primary endpoints midstream. Changing your study’s primary endpoint is roughly the equivalent of throwing darts at a dart board, missing the target every time, and then taking the dart board off the wall and painting an extra-large target of your own to make it appear like you’re ready for the English Pub-League Championships.
“The Raiders lost 12 games in the 2018 season using the primary outcome of final score. But if we look closely at each of these 12 games, we might be able to find, by chance, another potential outcome we could switch to. Take, for example, when Oakland played the Indianapolis Colts on October 28th. The Raiders lost that game 42-28. But if we were to switch the outcome to the score at the end of three quarters, the Raiders win 28-21! We’ll publish that as a victory without saying that we changed the primary outcome. Similarly, in their second game of the season, the Raiders lost to the Denver Broncos 28-20. But if we change the outcome to the score at halftime, we can publish this as a win, 12-0! We can do the same thing for their first game against the Los Angeles Rams.”
A similar trick is using composite endpoints. Instead of limiting your study to one premise or hypothesis, you could have dozens (although you would never mention this in the final draft). The result is that if you throw enough darts, you will probably hit upon something. Especially if I add a little twist. If I have 25 dartboards on my wall instead of just one, I can throw my darts, lots of darts, and pick the board(s) that the most land on, later claiming that my results were targeted and significant. Using similar tricks, the Raiders would have beat the Patrick Mahome’s-led Chiefs — twice.
“Take, for example, the third game of the season against the Miami Dolphins. Oakland lost that game 28-20, but Oakland had more total yards, more first downs, and a longer time of possession than Miami. Clearly, we can publish this as a victory for Oakland using our composite endpoints. Applying our composite endpoints, we can similarly change five other losses to victories. Oakland’s record now is 13-3.”
Next is a little trick that is exactly the opposite of the invisible and abandoned studies mentioned earlier — publish your team’s positive findings multiple times in multiple ways. With so many journals, and so many of these many journals desperately vying for what amounts to “breaking news,” make sure that the data for each individual (positive) endpoint or hypothesis is published as it’s own stand-alone study. Even better if you can mix it up a bit and publish the same findings in a different journal.
“Let’s apply this principle to the Oakland Raiders. Their most impressive victory of the entire season was when they upset a very good Pittsburgh Steelers team — on the road, no less — on December 9th. We certainly want to publish that twice! Let’s also duplicate-publish the Raiders’ victories over the Denver Broncos and the Cleveland Browns. The Oakland Raiders’ final record after applying the principles of medical research is an undefeated 16-0.”
Lest you think that this is an over-hyped absurdity, simply spend some time skimming through titles of my EBM column (see earlier link) and realize that in many cases —- particularly cases where there is a lot of money at stake —- this is a perfect description of a few of the ways you are being swindled. Only a few. The reality is that there are hundreds of ways that scientists are trained to “massage” their medical research in order to get the results they wanted, regardless of the truth. Even JOHN Q AVERAGE DOCTOR knows this (Keller is an ER doc with 25 years of experience who works in prisons).
It all goes to prove what I have been telling patients for nearly three decades — your health is up to you. And while your doctor is undoubtedly a wonderful person, the reality is that IN THEIR PROFESSION, THIS IS THE NORM. HERE is a post that might provide some ideas for starting the process of taking your health back. And if you appreciate what you are seeing on our site, be sure and like, share, or follow on FACEBOOK as it’s still a good way to reach the people you love and value most.
