By: Courtney Pinard, Ph.D.
Photo credit: Novartis AG via photo pin cc
How Prevalent is Scientific Bias?
Scientists and clinicians conducting clinical trials must abide by rigorous standards to safeguard against biases. Biomedical animal research has not been held to the same standards, and advocates of robust science have argued that this lack of scientific rigor is why more than half of pre-clinical studies are irreproducible. A recent study published at the University of Edinburgh in the U.K. shows that animal researchers are not using the same standards to prevent bias in study design. Such standards include 1) using randomized trials to prevent scientists from, for example, assigning unhealthy animals to the control group to boost a drug’s effect on the treatment group; 2) ensuring that researchers are blinded when accessing outcomes of an experiment; 3) calculating the correct sample size before starting an experiment; and 4) disclosing any conflicts of interest. The authors of the study examined 2500 papers published between 1992 and 2011 on drug efficacy, and the results were dismal. Only 30% of papers analyzed outcome in a blinded manner, 25% stated randomizing animals to groups, 12% included a conflict of interest statement, and less than 1% of papers reported calculating the needed sample size in advance. When the authors looked at whether institute quality or journal impact factor predicted bias, they found no correlation. The U.K. study is one out of many studies on the topic of scientific rigor that have resulted in growing concern from scientists and the public about irreproducible results in pre-clinical biomedical research.
According to an NIH commentary published last year, the reasons for why scientific bias in animal research is so prevalent are complex and have to do with the attitudes of funding agencies, academic centers, and scientific publishers. Authors of the commentary, Francis Collins and Lawrence Tabak, discuss these attitudes: “Funding agencies often uncritically encourage the overvaluation of research published in high-profile journals. Some academic [centers] also provide incentives for publications in such journals, including promotion and tenure, and in extreme circumstances, cash rewards.”
Given the continuing budget restraints, and Congress’ awareness about the reproducibility problem, national funding agencies have started to act. The NIH, for example, organized a workshop with over 30 basic/preclinical science journal editors to put together principles and guidelines to enhance research rigor and reproducibility. One such principle is “Transparency in Reporting”, and includes the bias safeguarding standards described above. Strengthening pre-clinical biomedical research will only occur when scientists and policy makers at funding agencies, academic institutions, and journals work together to put these principles into practice, and acknowledge that the “publish or perish” attitude rampant in the scientific culture needs to change. The situation and solution was described succinctly in a recent Nature Editorial on cognitive bias: “Finding the best ways to keep scientists from fooling themselves has so far been mainly an art form and an ideal. The time has come to make it a science.” (Martin Enserink, ScienceInsider)
Proposed Study to Track 10,000 New Yorkers
A new proposed longitudinal study will attempt to monitor thousands of households in New York City over the span of decades. Information will be gathered in intimate detail about how people in these households lead their lives, including information about diet, exercise, social activities and interactions, purchases, education, health measures, and genetics. This ambitious project is called the Kavli Human Understanding through Measurement and Analysis (HUMAN) project, and aims to quantify the human condition using rigorous science and big data approaches to understand what makes us well and what makes us ill. According to project leaders, existing large-scale data sets have only provided detailed catalogs of narrow aspects of human health and behavior, such as cardiovascular health, financial decision-making, or genetic sequencing. By measuring the feedback mechanisms between biology, behavior, and our environment over decades, researchers believe that that much more will be understood about how these factors interact to determine human health over the life cycle. For example, according to articles written by scientists in support of the project, the new data could measure the impact of cognitive decline on performing activities of daily living, on family members and caregivers, and on healthcare utilization or end-of-life decisions. A further goal of the project is to provide data to policy makers in order for them to develop evidenced-based public policies.
Anticipating privacy and cybersecurity concerns inherent in such an invasive study, Kavli HUMAN project researchers have established a Privacy & Security Advisory Council, comprised of members in the private, public, and academic sector. The Advisory Council includes bioethicists and patient privacy advocates. In addition to establishing the Advisory Council, project leaders conducted an opinion survey of diverse group of Americans asking whether they 1) think the study should be done, and 2) if they would be willing to participate. The results of the survey suggested that nearly 80% think that the study should be done and more than half were willing to participate. When questions arise about the ethics of collecting such information, Kavli HUMAN project researchers publicly argue that corporations already track Americans’ spending habits, location, and use of technology, and that “people’s data can be better used to serve them, their communities, and society.” (ScienceInsider, Kelly Servick)
Nutrition and Cancer
A Diet High in Red Meat and Processed Meat Increases Risk for Colorectal Cancer
The World Health Organization International Agency for Research (IARC) announced on Monday that eating too many processed meats are cancer-causing and eating too much red meat is “probably carcinogenic to humans.” Red meat is defined as all types of mammalian muscle meat, such as “beef, veal, pork, lamb, mutton, horse, and goat,” and processed meat is defined as meat that “has been transformed through salting, curing, fermentation, smoking, or other processes to enhance flavor or improve preservation.” The IARC reviewed 800 studies that looked at the association of cancer with consumption of red or processed meat in people around the world, of diverse ethnicities and diets. Results of this analysis revealed that the positive association between red and processed meat consumption and cancer was strongest for colorectal cancer. The Global Burden of Disease Project, an independent academic research organization, estimates that 34,000 cancer deaths per year worldwide are attributable to diets high in processed meat. Studies show that meat processing techniques and cooking this kind of meat at high temperatures can lead to the formation of carcinogenic chemicals, and that these compounds appear in parts of the digestive tract. Specifically, the agency said its experts concluded that each 50 gram portion of processed meat eaten daily increased the risk of colorectal cancer by 18 percent. Red meat was not as strongly associated with cancer as processed meat. Some public health experts criticized the bravado of the IARC announcement. In response to public inquiries, they have published a FAQ page where they state that smoking and asbestos are more likely to be causal for lung and other types of cancers. The announcement did not mark a new discovery, since the original report has been out for several years; it was meant to attract public attention and help countries looking to WHO for health advice. According to the director of IARC, “these findings further support current public-health recommendations to limit intake of meat.” (NPR; Anahad O’Connor, New York Times)
Have an interesting science policy link? Share it in the comments!