 |
 |
Dear all, I copied below the question (and responses), which was posted on the list serve of the Society for Judgment and Decision-Making. Many people on this list have voiced views that may or may not agree with the views outlined below. I want to invite you all to add your views to the list below (you can add anonymously if you wish and send it directly to me after which I will share the responses with you all). Thanks! Benjamin Djulbegovic, MD, PhD Professor City of Hope, Duarte, CA —- A question: “For many countries, the decision of when and how to lift control measures and re-open the economy is an important one, to say the least. Suppose that as a decision researcher, you are asked not to make a decision, but to determine the process that should be followed to make a decision. What process would you propose following in order to arrive at this decision? Is there evidence to support this being a good process? Response 1: Maybe I'm missing something, but: cost-benefit. Estimate additional deaths from whatever re-opening policy one considers. Value those lives at $7-$9 million, as policy makers typically do. That's cost. (Other stuff could be included that we always cause through daily activity: increased auto accidents, more pollution, etc.). Then, try to monetize all the benefit from not staying closed longer. How much would we have spent averting economic calamity, lost in GDP, how much increase in depression, domestic violence, suicide we avoid. Open if benefits exceed costs. Response 2: In general this would be a process of following a testing or test sampling process and models that predict rates of spread as a function of controls. What you are looking for is to dampen oscillations in a feedback system where more controls produce less spread but at a cost. You don't want to oscillate too widely, and want to head for a gradual overall reduction in rates of spread and lower controls. Eventually, either there is a vaccine that reduces the spread near zero and hence reduces the need for controls, or there is sufficient herd immunity from slowly allowing the spread (assuming immunity lasts for a year or two, still undetermined). The other side is how much economic dislocation is valued against the loss of life from the spread. Countries could set their relative values differently about how much disruption due to disease vs. disruption due to controls is tolerable. Those in turn depend on aspects of the health care system (capacity, etc.) and the resilience of the economy and political influence of those who are most at risk (poor, minorities, immigrants, aged). This is a tough system dynamics problem. Response 3: Decision analysis. Turn it over to Ralph Keeney and/or Detlof von Winterfeldt. The trick is getting the right group. But clearly the relevant experts are epidemiologists, economists of the sort who work in the relevant areas (e.g., the details of how the economy fits together, supply chains, etc.), probably at least one anthropologist familiar with social behavior in the region of interest, and at least a couple of lawyers who know stuff who who has the power to do what (possibly office holders). It would take a few very intense days, the first few of which would not involve any formal analysis. Maybe it would never even get to that point because the plan would become obvious. The group should be run in ways that discourage groupthink (i.e., that promote actively open-minded thinking). But that is a side effect of the usual procedures used in decision analysis. What is needed is a plan with contingencies. It should be sensitive to changes. Response 4: I would estimate the probability of infection (given estimated actual exposure and tested antibody prevalence) and open the economy when the expected value of patients needing care does not exceed available resources for care. The main optimization is over the health care system capacity. I apologize if this seems like a trivial answer... ;-) Response 5: QALY calculations seem apt to me. those might include QALY cost of the lockdown too. nobody likes QALY, however for coronavirus......... it's amazing that over two months I've seen just one mention of it! Response 6: Looking at behavior and wisdom of crowds, I would look closely at Barb Mellers' work Response 7: For one thing, it would have been best to establish the criteria for evaluating the effectiveness and ultimately for lifting them as much as possible in advance of enacting the policies. The research on escalation of commitment suggests that once the decision to enact a control measure has been made, it will become very difficult to justify lifting it, even if it isn’t working as hoped or is having unforeseen consequences, without being accused of prioritizing the economy over public safety. Establishing the criteria in advance could help to remove a lot of the partisanship from the decision. Establishing these criteria demands difficult tradeoffs between public health and safety and the economy, which has real, though less direct, consequences on health and safety as well. I would establish a committee of non-partisan health professionals and economists to help model the tradeoffs between COVID-19 related illness/death and poverty/unemployment for various timelines, triggers, and levels of control. Optimizing the policy in any meaningful way requires a quantifiable tradeoff which is unsavory but also common in other areas of regulation such as environmental and transportation safety. At the very least, analysis could make the value of human lives implied by various policies explicit to add clarity to the discussion. The results of this analysis should be made public, but ultimately, this is a policy decision that belongs in the hands of an elected official. This is precisely why we have elections, to choose the people we trust to make difficult decisions. Elections have consequences. God help us all. Response 8: I would approach this following the utility model in the paper I wrote with Edwards and my wife on everyday decision making. First, make a list of the consequences anticipated for each option, along with their expected values. Next, attach probabilities and saliencies (importance weights) to each consequence. Finally, sum the products of the three parameters over consequences to see which option gets higher utility. Do this for various dates under consideration. From a computational perspective, this is straightforward. The difficulty is that values are subjective. For Trump, a particular increase in the Dow will have a value that is far more than Cuomo's value for the same event. Similarly, a drop in the number of people getting infected will have lower value for Trump than for Cuomo. There is also subjectivity in how to define consequences. Whether infection rate is considered as a whole, as opposed to breaking into age categories, will affect the result (I am presuming Trump doesn't really care if old people, who in his view drain rather than contribute to the economy, get infected and die). The message is that the decision will depend on the decision maker's perspective, even if their process is the same. Response 9: Yes, I’m a decision researcher, but I was the chair of two units within the College of Public Health. That’s the main hat I’m wearing to answer this question. I’ve also worked for lots of agencies within the federal government, so my answer will be tainted by political considerations, not just JDM ones. There are two big obstacles to overcome in order to make a good decision. The first obstacle is that “died of covid-19” is being defined very differently in different jurisdictions. Some medical facilities are told to classify someone as “died of covid-19” even if no test has been done. Symptoms consistent with that diagnosis are sufficient! Other facilities, such as nursing homes, are classifying people as “died of covid-19” even if the victims have other very serious co-morbidities. This is a very common confound. I could go on, but all of my examples would illustrate the problem of getting good data. To help promote the best decision by conquering this first obstacle I would look at overall mortality rates. This will put the decision making process on the best footing, because a significant increase over the normal mortality base rate would be informative. Also, unlike diagnoses, there isn’t any ambiguity about mortality. I would modify the definition of “significant” to something much larger than the usual p < .05. A second consideration would be to examine the data from “natural experiments.” For example, Wyoming did not “lock down” until yesterday, but all neighboring similar states locked down much earlier. Denmark versus Sweden would comprise a similar comparison. Such overall mortality nation-to-nation comparisons could also help conquer the sloppy definition problem. Conquest of this first obstacle is essential for a good process. I don't think this first obstacle is difficult to defeat, but it involves silencing the current NIH head who says that we should keep the controls until a vaccine is available. The second obstacle is more psychological. If the feds lifted the control measures, CNN would devote hours to coverage of Jones et al. who died of covid-19 and whose grieving relatives insist that Jones would have lived if only Trump had kept the controls in place. Persons who die with a positive test for covid-19 whether or not there were co-morbidities will get lots of face time on CNN. This makes ending the controls politically very dangerous. All the epidemiologists in the world who testify that the death rate after controls are lifted is no greater than the regular flu death rate will be ignored. To combat this I suggest some Dawesian process by parading lots of anecdotes in front of TV cameras. These are people who went back to work and are grateful for that, people who got covid-19 after going back to work and recovered with no problem, people who tested positive but never showed any symptoms, etc. Another aspect of the process would be to allow state variations in the lifting of the controls. Packed subways in New York City are risky, but there are no subways in Wyoming. I'd use a state's overall monthly mortality data to make this decision. Response 10: I'm part of the regional task force that has to take decisions on when and how restart the economy (Region: Trentino, State: Italy). I'm the only psychologist, the rest are economists, sociologists, virologists. The process is simple: the politicians asked a group of "experts" to give them "advice", then they are going to make the political decisions. We are working using scenarios: from the most optimistic one to the least optimistic ones, incorporating all the available clinical, economic, psychological information into the scenarios. My part is to predict how citizens, workers and consumers will behave in each scenario. As a result of the scenarios we are also giving advice on which sectors and economic activities will suffer the most, so that the politicians will have a list to prioritize economic support. Response 11: I would suggest something like an "adaptive policy with monitoring" strategy : 1. measure the demands (random sampling in the population to evaluate the spread) 2. measure the offers (available means to treat the cases like oxygen masks etc.) 3. derive a stream of policy interventions from severe (like now) to lean (like recommendation to wear masks in the public). 4. continuously measure 1. and 2. and adapt 3. accordingly by moving a step up or down. This just as a quick reply without any application examples for this concrete decision domain, but it looks like many have to say something on this topic now. My personal view would be to put a stronger concentration on the vulnerable. There is a strong discussion in Germany on what follows next. Keen to see what these steps (promised to be announced till latest tomorrow) will be. Response 12: I believe the normative protocols- similar to handling the pandemics- already exist regarding the decision on re-opening economies. So, I think the question is not what should be (should have been) done- most countries , notably the US, already have (or, had) the normative decision protocols that ought to be followed. The real question is a) why these evidence-based protocols have not been activated when the decision-makers knew that the pandemics is coming , b) why these normative scripts are not been publicly discussed for further input? (In general, the protocols call for controlling health aspects of the pandemics - such as massive testing to find and isolate cases etc- before re-opening economies). Response 13: My sense is that a better process would be to avoid the "fallacies of one" and have the decision-makers first EXPLICITY identify at least Three key goals for the decision, then identify at least three uncertain futures (uncertainties) and then at least three key (explicit) decision options. I would suggest 4, and no more than 5, if the fifth is a default wait option. These ideas are variations of what is covered in my article on debiasing with Jack and Kathie in HBR (2015). More elaboration on those ideas can be found in a companion book chapter. Included in those publications are some of the reasons for these simple suggestions for decision improvement. I would be happy to expand upon these ideas if you want. Response 14: To me I would use a Tetlock value tradeoff model and ask myself a simple question; (1I which sacrifice am I more willing to make: loss of economy and ensuing loss of life vs., (2) loss of life from COVID. I would then establish a hierarchy between the two based and commit to the least-worst choice. Sent from my iPad - excuse typos and brevity ---------------------------------------------------------------------- ------------------------------------------------------------ -SECURITY/CONFIDENTIALITY WARNING- This message and any attachments are intended solely for the individual or entity to which they are addressed. This communication may contain information that is privileged, confidential, or exempt from disclosure under applicable law (e.g., personal health information, research data, financial information). Because this e-mail has been sent without encryption, individuals other than the intended recipient may be able to view the information, forward it to others or tamper with the information without the knowledge or consent of the sender. If you are not the intended recipient, or the employee or person responsible for delivering the message to the intended recipient, any dissemination, distribution or copying of the communication is strictly prohibited. If you received the communication in error, please notify the sender immediately by replying to this message and deleting the message and any accompanying files from your system. If, due to the security risks, you do not wish to receive further communications via e-mail, please reply to this message and inform the sender that you do not wish to receive further e-mail from the sender. (LCP301) ------------------------------------------------------------ ######################################################################## To unsubscribe from the EVIDENCE-BASED-HEALTH list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=EVIDENCE-BASED-HEALTH&A=1