WebMD BlogsWebMD Interviews

How Good Is Our Data on the Coronavirus Pandemic?

photo of pandemic map
By Brenda Goodman, MAMarch 20, 2020
From the WebMD Archives

As governments around the world grapple with an emerging pathogen that threatens to overwhelm healthcare systems and stall the global economy, experts have wondered about the quality of the information that’s available to inform those decisions.

In an editorial published in STAT, John Ioannidis, MD, a noted critic of medical research who co-directs the Meta-Research Innovation Center at Stanford University, wrote that COVID-19 has been called a once-in-a-century pandemic. “But it may also be a once-in-a-century evidence fiasco.” Ioannidis says we lack reliable information on how many people have been or will be infected. Yet governments have taken dramatic steps to curb the pandemic. While people and businesses may be able to tolerate lockdowns for a short time, we don’t know how long these measures should continue if the virus continues to spread. “How can policymakers tell if they are doing more good than harm?” he asks.

We talked to two experts about the challenge of acting when evidence is still developing. Here’s what they had to say about what we know, how firm that knowledge is, and the problem of knowing what to do with it. They are Isaac Bogoch, MD, an infectious disease specialist and clinical investigator at the Toronto General Hospital Research Institute in Canada, and Perry Wilson, MD, an associate professor and expert on evidence-based medicine at the Yale School of Medicine.

WebMD: How difficult is it to be in government right now, to have to make decisions about the COVID-19 pandemic?

Isaac Bogoch, MD: Ultimately, we have to make major decisions in real time without a robust evidence base, and those decisions are going to have significant impact on populations. There isn’t time to wait for a strong evidence base to guide these decisions, so we have to extrapolate from existing data and we also have to factor in common sense and humanity with the policy decisions that are being made.

We know for example, how flu pandemics have played out in the past. We’re drawing on some of that data, but it’s not going to be a perfect fit because this is a brand new virus. We don’t know everything we need to know about it.

There are lots of questions. How long can it live on surfaces? We didn’t know until about a week ago. Do flight restrictions work? We don’t know for this one, but there are data and models from other epidemics.

We have to appreciate the strengths and weaknesses of the data that we’re tapping into and that we’re completely transparent and acknowledge those limitations when we communicate the data.

It’s also reasonable to expect that policy will change significantly throughout the course of this pandemic because there are growing evidence bases that will help public health policy.

One of the issues is communicating to the general public that policy changes aren’t a failure but represent keeping abreast of the most current evidence base.

WebMD: What about modeling? There is some really good and very fast work being done in modeling right now, but it can also feel pretty scary to see those predictions:

Bogoch: Modeling is extraordinarily important, but we have to remember that those are models and they are used to inform policy but there are inherent weaknesses in any model system. They are not crystal balls, but we need them to help us make decisions.

WebMD: What about the evidence behind drugs and therapeutics to treat COVID-19?

Perry Wilson, MD: The evidence that we have right now is not of the quality that we’ve come to expect in the context of the best medical literature. It’s not gold-standard randomized, controlled trials. What the coronavirus epidemic has forced us to do is to come to grips with the idea that the perfect is the enemy of the good. Time is simply not a luxury we have right now. It would be unwise of us to throw out the evidence that’s coming in because it’s not top quality.

We have to take everything that’s coming in with a grain of salt. We have to realize that this data may be telling us something that isn’t true. That isn’t a medical reality. But that’s not a reason to act.

The key is going to be something called biological plausibility—whether or not an agent behaves in a way that we can understand based on what we already know about it.

In large, gold standard randomized, controlled trials, the data often stood for itself.

When we’re looking at much smaller studies in data that’s not collected in as rigorous a way, and we have to ask ourselves, does this data make sense?

One current example of this is chloroquine. The data is very limited on this drug and whether it can treat COVID-19. It’s based on small number of patients. It has not peer reviewed. It has not been randomized. In the old days we would have said ‘Well, come back later when you’ve conducted a rigorous study.’ Now we have to take what we have and look at the biological plausibility. Is it possible it could treat this infection? We have good cell culture data that the drug might have some impact based on how we know about how the virus infects cells and how it acts in the body. So we have to act on that.

Right now, available evidence is worth more than best evidence, because we can only use what’s available.

WebMD: What about the flood of preprints that are being published to open access sites like MedRxiv. There are some scientists who have shared new information via Twitter.

Wilson: I think it is inspiring to see all the work that’s being done on this right now. It feels like trying to go to the moon, this scramble for information. There are so many researchers and scientists who are working very, very hard to turn the tide, and I find a lot of optimism and hope in that. It won’t all turn out to be true, but I think it’s amazing to see the effort.

WebMD Blog
© 2020 WebMD, LLC. All rights reserved.

More from the WebMD Interviews Blog

  • photo of coronavirus el paso

    El Paso COVID Surge: 'It's Just Constant'

    With nearly 30,000 active COVID-19 cases and a test positivity rate of 20% according to the city's dashboard, El Paso, TX,  is averaging about 1,000 new cases a day.

  • coronavirus vaccine

    CDC’s Messonnier Says She Trusts COVID Vaccine Process

    Nancy Messonnier, MD, director of the National Center for Immunization and Respiratory Diseases at the CDC, talks about the COVID vaccine rollout and having flu season in the middle of a pandemic.

View all posts on WebMD Interviews

Latest Blog Posts on WebMD

View all blog posts

Important: The opinions expressed in WebMD Blogs are solely those of the User, who may or may not have medical or scientific training. These opinions do not represent the opinions of WebMD. Blogs are not reviewed by a WebMD physician or any member of the WebMD editorial staff for accuracy, balance, objectivity, or any other reason except for compliance with our Terms and Conditions. Some of these opinions may contain information about treatments or uses of drug products that have not been approved by the U.S. Food and Drug Administration. WebMD does not endorse any specific product, service or treatment.

Do not consider WebMD Blogs as medical advice. Never delay or disregard seeking professional medical advice from your doctor or other qualified healthcare provider because of something you have read on WebMD. You should always speak with your doctor before you start, stop, or change any prescribed part of your care plan or treatment. WebMD understands that reading individual, real-life experiences can be a helpful resource, but it is never a substitute for professional medical advice, diagnosis, or treatment from a qualified health care provider. If you think you may have a medical emergency, call your doctor or dial 911 immediately.

Read More