Researchers develop clearer, more intuitive method of relaying scientific results
Faced with confusing or contradictory COVID-19 health news, the public needs more help in understanding complex scientific studies. Michigan State University researchers say.
“Scientific studies, especially those in health and medicine, are often interesting—or at least relevant—to the general public. If results are conveyed accurately and thoughtfully, the hope is that they can inform heathy choices and behavior,” said Kenneth Frank, MSU Foundation professor of sociometrics. “The problem is that science isn’t certain. And it can be difficult to communicate the uncertainty of scientific results.”
A new paper authored by Frank and MSU colleagues Thomas Dietz and Lixin Zhang—as well as colleagues from University of Chicago, Arizona State University, the University of Connecticut, the University of Tennessee and Optinose Inc.—maps out a clearer and more effective way of expressing this uncertainty. Their method is all the more relevant now, since the coronavirus pandemic has jump-started the rate at which scientific information is disseminated.
Frank said that a good example of the difficulty in communicating study results comes from a recent talk by Anthony Fauci, the director of the National Institute for Allergy and Infectious Diseases. Fauci relayed trial results from a potential COVID-19 therapy, remdesivir: “Remdesivir improved recovery from 15 days to 11 days with a P value of 0.001. … We would have normally waited several more days. The data may change, but the conclusion won’t.”
Some people may be familiar with the P- value as a way of expressing certainty or uncertainty in a study result, but Frank said that the concept is unintelligible to most.
“Even Fauci doesn’t have the language to talk about this kind of finding in easily relatable terms,” Frank said. “Many people can’t understand it. So, our goal is really to improve the conversation. We propose a novel way of framing the uncertainty of results in terms of patient outcomes, rather than P-values alone.”
The crux of the method is this: Results are defined as the number of patients who would have had to experience a different outcome for the study’s conclusion to change.
“In this way, the method generates statements such as ‘xx of the treatment cases that experienced improvement would have to have experienced no improvement to change the conclusion at conventional levels of statistical significance,'” Frank said.
Sometimes as little as one switch is necessary to shift the results from “significant” to “insignificant” or vice versa.
Take as an example an early randomized trial of hydroxychloroquine, a medication that has been tested as a potential treatment for COVID-19. Frank’s paper finds that if just one participant in the study had had a negative or neutral response rather than a positive one, it would shift the results from significant to insignificant.
“One switch is not a lot,” Frank said. “It means that an estimate is right on the borderline of what is typically considered conclusive-inconclusive. And the new scientific debate that has ensued about the hydroxychloroquine studies coming out reinforces that uncertainty.”
Frank and his team have already made the algorithm available to the public in the form of an app konfound-it.com that performs the calculation after the user inputs basic data found in most results tables; they are also sharing real-time updates on their blog.
Some researchers are already beginning to adopt the method and Frank’s hope is that more will begin to do so, adding the extra conceptualization to their papers as an addendum to the usual reporting of results.
Source: Read Full Article