by Jon Robison, PhD, MS
In my 25 or so years of being involved in the health field I have witnessed the comings and goings of a multitude of highly acclaimed scientific findings. Over and over again, a new piece of data, often something related to the risk of death or disease will appear in the scientific literature and instantaneously saturate the global media. And over and over again, often not far down the road from the initial announcement, will come another study or number of studies convincingly refuting the original findings. Somehow however, these new studies rarely garner even a fraction of the attention paid to the original, whose conclusions continue to be presented as indisputable fact.
Those of us who have been involved with the Health At Every Size® model over the years are all too familiar with this phenomenon. Remember the 300,000 people killed by obesity statistic that first surfaced in the prestigious (but not infallible) New England Journal of Medicine (NEJM) in 1996? The claim was dredged up incorrectly from a study published a few years earlier in the Journal of The American Medical Association (JAMA). In fact, the authors of this JAMA article published a response to the NEJM article protesting the bogus claim. In addition, the 300,000 number was formally debunked in 2005 by epidemiologists from the CDC. Yet to this day, some 7 years later, this bogus, fear laden statistic is commonly bandied about (and rarely challenged) in the media and at health-related conferences.
And who can forget the our children will be the first generation not to live as long as their parents because of obesity claim made by Olshanski and Ludwig again in the NEJM in 2005. Confronted about the origins and accuracy of their frightening claim just a few months later in an expose in Scientific American the authors responded with “these are just back of the envelope plausible scenarios. We never meant for them to be portrayed as precise.” In spite of this and even though the World Health Organization and The U.S. Social Security Administration have projected that life expectancy will continue to rise in the foreseeable future, this scary piece of non-science is commonly repeated today by health professionals and the lay public as if it were an ”evidence-based” fact of life.
I call this unfortunate and all too common phenomenon The Cockroach Effect. Cockroaches are amazingly resilient little creatures. They can continue to be active after not eating for 30 days. They can survive without air for 45 minutes and they have been known to recover after being underwater for 30 minutes! If you have ever tried to kill a cockroach or get rid of an infestation you know it is no easy task. Neither is trying to erase attention grabbing but faulty health-related claims once they have made their appearance.
The “Cockroach Effect” is certainly not limited to weight-related research. Dr. John Ioannidis has spent his career challenging his peers by exposing the personal biases, economic pressures and downright bad science that plague the medical profession. In a seminal paper in PLoS Medicine online in 2005 he presented a model which predicted correctly that 80% of non-randomized studies, 25% of randomized trials and 10% of large randomized trials were typically refuted by later research. While we expect contradictions as part of science, Ioannidis also found that even when faulty research was debunked, its conclusions typically persisted for years or even decades. The details of his fascinating findings are explored in an article entitled “Lies, Damned Lies and Medical Science” which appeared in the Atlantic Magazine in November 2010.
Sometimes the lingering of these faulty claims is fairly benign. It probably doesn’t hurt to drink 8 glasses of water a day; even though the scientific basis of this recommendation is elusive at best (I am guessing it may have originated with the promotion of drinking more water as a weight loss technique). Sometimes, after enough years have passed and sufficient conflicting evidence has accumulated, the faulty claims actually get laid to rest. This may finally have happened with the if you don’t exercise for at least 30 minutes it won’t do you any good mantra.
But when it comes to issues like weight and health, where there is so much deeply rooted socio-cultural and economic investment in the status quo, it can be quite a different story. We see all too clearly the heartbreaking consequences in the billions of dollars spent on fruitless and sometimes dangerous weight loss schemes and scams and the lives torn apart by disordered eating and weight stigma. My recommendation is that we stay vigilant and be wary of The Cockroach Effect – faulty data that will not die!