We live in a world that is constantly connected to the internet and as a result is overloaded with information. Filtering out fact from fiction can be challenging in both mainstream media and peer-reviewed literature. At this year’s Lorne Trottier Public Science Symposium Series, the question everyone asked was, “Is that a fact?”
The symposium took place on October 28 and 29 at the Centre Mont Royal, and was organized by the McGill Office of Science and Society (OSS). The symposium is an annual event held for the purpose of informing the public, encouraging debates, and raising awareness around contemporary issues in science.
Joe Schwarcz, Director of the OSS and the symposium moderator, opened the symposium by describing the confusion caused by the sheer number of academic studies with contradictory results. One example he gave was debate surrounding the benefits of Vitamin D. “We don’t know who or what to believe, especially when it comes to nutrition,” Schwarcz explained.
The Reliability of Peer-reviewed Literature
John Ioannidis, a professor of medicine at Stanford University, echoed the need for caution when it comes to scientific peer-reviewed literature. Ioannidis conducts meta-analyses – a method involving combining and comparing findings of various studies – on the validity of scientific studies, and his research has shown that a large number of peer-reviewed studies are flawed. Ioannidis presented some staggering statistics for the 1996 to 2011 time period: there were 15,153,100 scientists publishing papers in major scientific journals with 25,805,462 papers published. With the sheer volume of scientific claims being made, the question remains about the accuracy of these claims and their ability to pass the test of time.
“Scientists are under pressure to deliver very nice papers in order to get published and receive funding,” said Ioannidis. Science runs on the principle that successful replication of results is required to validate the accuracy of scientific discoveries. However, the push to keep publishing significant and novel results is accompanied by a lack of replication studies to ensure quality control. According to Ioannidis, there are several reasons for the lack of such studies. Replication studies can be seen as mundane compared to the hype of new discoveries; they are also expensive to carry out and more difficult to receive funding for.
According to a Lancet paper published by Ioannidis in 2005, most of the claims of statistically significant effects in traditional medical research are either false positives or substantial exaggerations. The lack of replication studies has serious effects in both academia and industry. In a Nature study published in 2012, scientists were only able to reproduce 11 per cent of findings in 53 published papers on pre-clinical cancer research.
To increase the reproducibility of studies, Ioannidis proposed a new system to be used for publishing papers – which he describes as “upfront registration of studies.” Authors would share certain information with the scientific community during the publication stage, such as the data analysis or methodology used. This would allow the scientific community to correct any biases or errors at an early stage, while gaining knowledge as to how the claims made by the paper were formulated.
Health in the headlines
Timothy Caulfield, a law and public health professor at the University of Alberta and author of The Cure for Everything!: Untangling the Twisted Messages about Health, Fitness and Happiness, stressed the importance of critically looking at health headlines. He even explained that the title of his own book was misleading to some as it does not actually contain the ‘cure for everything’ but rather describes the search for the ideal healthy lifestyle.
There are hundreds of unhealthy diets and lifestyles advocated for by celebrities. Celebrities, who hold a prominent place in the public eye, can have powerful impacts on society. Often, these diets and lifestyles have no scientific basis and can have negative health effects on those who choose to follow such regimes. Caulfield pointed out the recent Jenny McCarthy anti-vaccination campaign as an example of bad health practices being propagated by a celebrity through the media.
According to Caulfield, despite the vast number of diets and lifestyles presented to the public with miracle results and benefits, only five major factors account for 95 per cent of a healthy lifestyle: avoiding smoking, exercising, eating a balanced diet, managing weight, and following safety regulations. Some other emerging studies suggest potential benefits of sleeping, standing more often, and having good relationships with people.
Unfortunately, as Caulfield said, “Non-science sells more than science,” when describing the abundance of bad science in mainstream media. Popular notions surrounding exercise and diet are often untrue. It is important to approach scientific headlines with a critical eye and not take everything seen for granted.