In his book “Willful Ignorance: The Mismeasurement of Uncertainty”, Herbert I. Weisberg notes: “Medical research is often held up as the paragon of statistical research methodology. Evidence-based medicine, based on randomized clinical trials, can provide proof of the effectiveness and safety of various drugs and other therapies.” (1) He goes on to note that clinical practitioners, who’s judgement is informed by evidence-based data, are finding little in the research that is useful.
More generally. “For many, the concept of scientific method has been restricted to a narrow range of approved techniques, often applied mechanically. The result is to limit the scope of individual creativity and inspiration in a futile attempt to attain virtual certainty.” (2)
The Coronavirus has brought this concern to the forefront. This concern has implications for risk assessment. This piece will discuss the general growth of statistical analysis in academe. It will also discuss the conflict between research and practice in relation to the Coronavirus.
Statistical Analysis in Academe
The use of statistical analysis and research methodology in medicine, public health and other academic fields has become a main stay. The historical root of development of statistical analysis go back centuries. For instance, Pascal and Fermat Huygens began developing probability analysis in 1654. Data collection and observation were key elements in this development. Over time, the approaches used to gather and analyze data have become standardized.
As a PhD student, I had to take courses on research design and statistical analysis. My dissertation was entitled “Star Academics Do They Garner Increasing Returns?” It looked at the characteristics which assisted in obtaining National Institute of Health (NIH) grant funding. One of the conclusions was that if an academic had an MD/PhD he/she was more likely to receive NIH funds. This reflects the recognition that having an MD degree, when supplemented with a PhD, reinforces the scientific research methodological mind set. It also results in more grant funding, which is the life blood of scientific research.
It should be noted that my PhD is in Urban Studies. That field, like the fields of Political Science and Economics, to mention a few, is behavioral science (statistically) oriented. That orientation has long been recognized as a necessary norm. For instance, in 1963, in order to improve Stanford University’s national rankings for its Political Science Department, management sought to hire star academics with a behavioral science pedigree. As a result, the national ranking of the department went from thirteen to seven. The point is regardless of discipline, a scientific methodologically based approach is viewed as a necessity. Thus, the emphasis on statistical analysis and research methodology shapes views on acceptable professional behavior.
Professional and Bureaucratic Behavior
This inculcation of research and professional norms frames the attitude and approach taken towards issues like therapeutics for the Coronavirus. There is controversy about the use of Hydroxychloroquine. This is because there has not been a randomized clinical trial on its usefulness of mitigating the impact of the Coronavirus.
Anecdotal reports from patients and front-line physicians indicate it has therapeutic capabilities. This disagreement reflects the problem noted by Herbert I. Weisberg. It also reflects different medical priorities.
The differences can be seen in the mission of the Food and Drug Administration (FDA) and aspects of the modern Hippocratic Oath.
The FDA is responsible for “protecting the public health by assuring the safety, effectiveness, quality, and security of human and veterinary drugs, vaccines and other biological products and medical devices.”
The modern Hippocratic Oath lists the following requirements.
“I will apply, for the benefit of the sick, all measures (that) are required, avoiding those twin traps of overtreatment and therapeutic nihilism.”
“I will remember that I do not treat a fever chart, a cancerous growth, but a sick human being, whose illness may affect the person’s family and economic stability. My responsibility includes these related problems, if I am to care adequately for the sick.”
The emphasis of the FDA and the requirements of the modern Hippocratic Oath are beneficial and laudable. However, the focus of each is different. The FDA is concerned with the protection of public health generally. The Hippocratic oath focuses on the patient and his/her family. This difference is not overly important except in a medical crisis. In such a crisis, where lives are on the line, the difference is stark. Physicians on the front line are concerned with saving the lives of their patients. To do that they often need to use drugs that are available. The FDA is charged with making sure drugs are safe. In order to do that it uses an accepted scientific and research methodology to ensure professional acceptance. Professional acceptance is needed to maintain the reputation of the FDA and ensure the drugs prescribed by physicians are safe.
The methodology most frequently used is randomized research approach. This type of study takes time. Unfortunately, time is something that patients infected with a virus, like the coronavirus, do not have. This requires FDA to adjust. For instance, on March 28, 2020 it issued a letter allowing hydroxychloroquine’s use to fight Coronavirus (COVID-19). The letter lists three stipulations.
- “FDA-approved hydroxychloroquine sulfate that is approved by FDA for other uses accompanied by its FDA-approved labeling and authorized Fact Sheet.
- The chloroquine phosphate must be administered by a health care provider pursuant to a valid prescription of a licensed practitioner.
- The chloroquine phosphate may only be used to treat adult and adolescent patients who weigh 50 kg or more and are hospitalized with COVID-19, for whom a clinical trial is not available, or participation is not feasible.” (1)
The letter permits physicians to use hydroxychloroquine sulfate, (same basic drug as hydroxychloroquine) but within limits. The patient must be hospitalized with COVID-19 and no clinical trial is available. The point is that while hydroxychloroquine sulfate may be used, it can only be used if no clinical trial is available. While a clinical trial is desirable, in such a trial the patient may get varied doses or a placebo. The placebo is used to establish a base line against which the effects of the drug can be measured. However, a placebo essentially withholds a possible cure from a sick patient. Further, the patient must be sick enough to be hospitalized. It does not authorize prophylactic use.
Such a circumstance is troublesome ethically. Equally troublesome is the possibility the mindset which requires rigorous statistical validation might inhibit the transition from prevention to management.
Managing the Risk of COVID -19
There is a lot of discussion about reopening the economy. Fundamentally, there are two basic camps. One camp says do not open until COVID-19 cases are almost nil, a vaccine is developed, or therapeutic are statistically validated. The other says it is time to open the economy. We cannot afford to have it shut down any longer. Each side argues that there are risks which need to be mitigated. But how great are the risks?
In truth, no one knows. The reason is no one knows the denominator for COVID-19. In other words, there is good data for those who have been hospitalized and those who have died, the numerator. There is no good data on those who are asymptomatic. It is only by knowing the asymptomatic figure and adding it to those hospitalized that the denominator can be determined. Once that is determined, one can determine the infection and fatality rates. The results help determine whether the risk is acceptable. For instance, the fatality risk is substantively different if the rate is 4% versus .4%. If the actual mortality rate is 4% then considerable caution is needed in reopening the economy. However, if it is .4% the risk is considerably lower. When combined with therapeutics that maintain a low fatality rate, the risks of reopening the economy may be more than manageable.
A non-peer reviewed study entitled “COVID-19 Antibody Seroprevalence in Santa Clara County, California”, indicates, based on the testing of 3,300 residents, that the prevalence of COVID-19 antibodies in the population is greater than previously thought. In fact, if the data is correct, it may be as much as 50-85-fold more than the number of confirmed cases. The data indicates an infection fatality rate of 0.12-0.2%. (2)
Now the study is specific to Santa Clara County California. But it has broader implications for the entire United States. Consequently, it begs the question of: Why hasn’t a nation-wide sample seeking the same information been conducted?
The answer reflects three issues. First, COVID-19 is highly infectious. The speed with which it spread, caught the federal, state and local governments by surprise and basically unprepared. Second, the speed with which it spread, and its virulent nature meant priority had to be given to treating patients and containing the spread. Lastly, epidemiologist tend to focus contact tracing. When anti-body testing is discussed, it is focused on first responders.
A Group of Workers Who Are More Representative of Back to Work Environment
While anti-body testing of first responders is a good idea for safety and morale purposes, first responders do not work in the conditions most people will encounter when the economy is opened. If a greater representation of normal work conditions is desired, there are groups which can be anti-body tested that is more representative.
One such group is U.S. Postal workers. The Postal Service operates processing plants and retail operations (Post Offices) nationwide. The workforce is diverse. The work is conducted under consistent rule and procedures. The work activities are like activities many workers will encounter. The processing plants are like manufacturing activities. The retail operations are retail operations. Since Postal workers have been working consistently through out the pandemic, their exposure is like what will be encountered when everyone goes back to work.
Containment versus Mitigation
The question being raised is: Has a professional and bureaucratic bias creeped in? Just like the difference between physicians on the front line and scientific researcher during a medical emergency, the bias may be towards nullification of the virus, versus accepting the risk the virus will stay active in the population for some time. The perspective has substantial policy implications.
Managing means accepting some risk. If that is the strategy and it seems necessary, if the economy is to open, determining the level of risk is important. Large-scale anti-body testing would provide a base line. Equally important is the identification and development of therapeutics which will mitigate the adverse health impact of the virus.
The legislation passed by Congress on April 23, 2020 contains money for testing. It requires each state to submit a testing regime to the federal government. Which direction the states take on testing, contact tracing versus anti-body testing, will indicate acceptance or rejection of some level of risk. It will also indicate the extent to which state and local economies must remain on hold.
End Notes
- Weissberg, Herbert I., 2014, “Willful Ignorance: The Mis measurement of Uncertainty”, John Wiley & Sons, Hoboken NJ, page 2.
- Ibid, page 2.
- United States Food and Drug Administration, 2020, “Request for Emergency Use Authorization for Use of Chloroquine Phosphate or Hydroxychloroquine Sulfate Supplied from the Strategic National Stockpile for Treatment of 2019 Coronavirus Disease” March 28, by Dr. Rick Bright PhD.
Bio:
James J. Kline, Ph.D., CERM, is the author of numerous articles on quality in government and risk analysis. He is a senior member of the American Society for Quality. A Manager of Quality/Organizational Excellence and a Six Sigma Green Belt. He has consulted for the private sector and local governments. His recent book, Enterprise Risk Management in Government: Implementing ISO 31000:2018, is available on Amazon. He can be reached at jeffreyk12011@live.com.