The Imperial College of Overestimation and the Modeling Data That Locked You Down
Our modeling data shows you're going to have to give up your rights and freedoms.
The phrase "modeling data shows..." seems to have become synonymous with news coverage about COVID-19. Newscasters often use the phrase before spewing out the doomsday situation that's sure to come sometime in the near future. Astronomical hospitalizations, mortality and sickness are typical forecasts constructed by these models. But who are these modelers? How do they arrive at these apocalyptic conclusions? And why do we instill so much faith in them as if they were Moses announcing the 10 commandments from the mountain side?
Well for that we have to go back to the origination of the pandemic, in the beginning during the early months of 2020 COVID had just recently been discovered in China and was beginning to make its globetrotting trip around the world. China had taken extreme measures in order to contain the virus like limiting people's travel, closing businesses and quarantining the sick and suspected to be sick in field hospitals and hotels. The world wasn't surprised when China was able to enact such strict social measures on their citizens since under normal times the government maintains a tight grip of control around its population. Regardless even with such strict measures of quarantine and containment the virus managed to spread outside of China. The first Western nation to fall victim to the new plague was Italy when the virus struck in early March 2020. Prime Minister Giuseppe Conte announced nationwide restrictions that applied to all of Italy's citizens. This was the first time a nationwide lockdown had been attempted, even China had only placed restrictions on Hubei province which contained Wuhan within its borders.
At the time of the announcement Italy had confirmed 9172 cases of Sars-COV-2 with 463 deaths attributed to the disease1. Giuseppe's lockdown included restrictions on travel, closure of schools and universities, cancelations of jail visitations and inmate day programs, all sports events and outdoor gatherings were forbidden and bars were subject to a 6pm curfew. Giuseppe attempted to play to Italians sense of civic duty to convince citizens to comply with the restrictions "We all have to renounce something for the good of Italy." He's quoted as saying during the lockdown announcement.
Strategies like this to get people to comply with mandates would be covered in a paper published later that month by SAGE (Scientific Advisory Group for Emergencies). S.A.G.E is a British organization that works under the countries parliament. The paper was titled "Options for Increasing Adherence to Social Distancing Measures"2 it covered guidelines and strategies for democracies to employ in order to get their citizenry to comply with government mandates. Aside from playing to people's civic duty and "responsibility to others" the paper also highlighted such options like persuasion.
One mechanism for persuasion the paper suggested was to enhance the "perceived threat" of the virus in the less vulnerable population, a quote from the SAGE paper is as follows "A substantial number of people still do not feel sufficiently personally threatened; it could be that they are reassured by the low death rate in their demographic group, although levels of concern may be rising. Having a good understanding of the risk has been found to be positively associated with adoption of COVID-19 social distancing measures in Hong Kong. The perceived level of personal threat needs to be increased among those who are complacent," Basically younger people were realizing that the mortality rate amongst their demographic was relatively low and therefore they were less at risk so given that information they were more likely to disobey social distancing measures. To fix this the government had to make them perceive the virus as a larger threat. The strategies outlined in "Options for Increasing Adherence to Social Distancing Measures" would be used extensively in the coming years to get the citizens of democratic nations to adhere to lockdown measures. Nevertheless the lockdown in Italy was successful and a majority of the nation complied with the draconian policies. This showed that these types of government lead lockdowns as seen in China could be replicated at least to some degree in the west.
Now that Italy was burdened with Sars-COV-2 it was only a matter of time before the virus would reach other European countries in the same magnitude. However in Great Britain Prime Minister Boris Johnson had a much different attitude towards the virus at the time. Boris originally had every intention of letting the virus run its course through Great Britain with the end goal of achieving herd immunity within the population. However this strategy changed when the Imperial College published their 20 page report titled "Report 9: Impact of Non-Pharmaceutical Interventions (NPIs) to Reduce COVID-19 Mortality and Healthcare Demand"3 which outlined the modeling of the virus’s spread, hospitalization and mortality rate for both the U.K and U.S. The report was published March 16th, 2020 and by March 23rd the U.K drastically shifted from their current "herd immunity" strategy in favor of a lockdown based one, largely due to the college's predictions.4
Now before I go into detail about "Report 9" it's important to provide some backstory about the Imperial College. The Imperial College is a college located in Great Britain with affiliations to the CDC (Center for Disease Control). At the time of the COVID paper the epidemiological modeling team was headed by Neil Ferguson a theoretical physicist who decided earlier in his career to apply his mathematical skills to virology by predicting viral spread and creating models. COVID wasn't the first time him or the Imperial College had developed epidemiological models for viral spread.
They had also developed models in the past for Mad Cow disease and H5N1 (Bird flu). In the case of Mad Cow disease the college had predicted that 100,000 to 150,000 British citizens would succumb to the virus5, less than 180 Britons died as a result of Mad Cow6. The college predicted as many as 200 million deaths due to H5N17 between Fergusons prediction in 2005 and May 6th 2022 a total of 465 deaths worldwide have been attributed to H5N18.
In fact overestimation in scientific modeling had eventually led the U.K Parliament to go on record saying "We examined this issue using the swine flu case study and had misgivings about the Government's communication of what it termed "reasonable worst case scenarios", that is, the worst situation that might reasonably happen. While such scenarios are useful for organizations preparing for, and responding to, emergencies, use of such scenarios led to sensationalized media reporting about the projected deaths from swine flu. We concluded that the Government must establish the concept of "most probable scenarios" with the public, in all future emergencies."9 Unfortunately this advice was thrown out the window when the Imperial College published its COVID-19 report.
Report 9 outlined two distinctive strategies that Great Britain and the U.S could take to handle the virus. Those two strategies where mitigation in which transmission would not be interrupted completely but slowed so that the burden on the health care system would be more manageable because hospitalizations would be spread over time rather than a fast and large surge, the other strategy and the one promoted by the Imperial College was suppression in which the spread of the virus is interrupted and the aim of it is to bring case numbers down instead of letting them slowly rise as in the mitigation strategy.
The main quantifiable difference between these strategies is the R-rate, what R-rate is a measure of how many people on average an infected person will spread the virus too. The mitigation strategy slows the R-rate whereas the objective of the suppression strategy is to get the R-rate below 1 which will than lead to a depletion in cases. Now as you can imagine the suppression strategy requires more stringent measures or N.P.I's (Non Pharmaceutical Interventions) as the paper refers to them and also for longer times than in the mitigation strategy.
At the time of the paper COVID's R-rate was estimated to be 2.4, the college based this number off data obtained from Wuhan but ran simulations for an R-rate value of 2.0-2.6. However other data about COVID wasn't so readily available since the virus was so new at the time and because of this a lot of input data into the modeling programs was assumed rather than actually quantified. What tangible numbers that could be scrounged up in regards to COVID from the outbreaks in Wuhan and Italy were used but a lot of assumptions were made. Assumptions including incubation period, rate of infection, the difference in spread ability between symptomatic and asymptomatic carriers, re-infection and so on. To be fair these numbers were likely sourced from similar viruses like influenza but both the college and media should have done a better job at making the public aware that such assumptions were being made because as the New York Post reported April 5th "Virus Models are not crystal balls".10
Another problem with the colleges paper was that as the paper itself states "We do not consider the ethical or economical implications of either strategy (suppression or mitigation) here" (bracketed portion added to provide context) which again to be fair was not the job of the epidemiologists, it was instead the duty of the politicians charged to enact policy based on the modeling data however it seems as though law makers acted like the scientists in the sense that they too seemed to not consider the ethical or economical consequences of lockdown.
The thing that seemed to scare Britain into draconian law and what the messaging early in the pandemic by the mainstream media was that if left unchecked the virus would overwhelm our healthcare system, ICU's would fill up and as a result and patients would go without care. It was made clear in the colleges report that even with the most aggressive mitigation strategies both general ward and ICU beds would be grossly exceeded.
New York state and more specifically New York City was the first place in the United States that the virus hit heavily and where the college had predicted an 8-fold increase in hospital capacity. In response to the virus NYC more than doubled their ICU capacity from 1600 beds to 3500 beds by April 9th, 2020.11 They did this in part by setting up a temporary field hospital in the Jacob K. Javits convention center as well as docking a US navy hospital ship the USNS Comfort in its port.
After the virus burned through New York the society of critical care medicine did a study of COVID's impact on the state. The study titled "United States Resource Availability for COVID-19"12 published May 12th, 2020 mostly focused on nationwide ventilator availability but found that the Javits center and USNS Comfort ship where underutilized during the peak of the epidemic in NYC. A paragraph in the report by the society reads as follows:
"The IMHE and MRC GIDA (Global Infectious Disease Analysis [Imperial College]) projection data, as broadly presented in the media, convey a high degree of certainty, particularly regarding the peaks of illness and death. However, many stasticians, epidemiologists, and clinicians have questioned the value and certitude of the projections, especially when so many factors of COVID-19 and societies reactions to both the illness and compliance with governmental recommendations were unknown. These commentaries cast doubt on the core premises of the protection. Collectively the commentaries suggest that such models be used primarily for short- rather than long-term planning. Many discrepancies were noted between the projections and data-driven realities gleaned from contact with COVID-19. For example, in metropolitan New York, there were gross overestimations of the need for hospital and ICU beds and ventilators. These projections led to a large number of additional hospital beds being built and a big hospital ship being brought to New York City. Both of these high-cost and large-scale undertakings were underutilized. Moreover, the uncertainties of the usefulness of projections have been reinforced by the frequent revisions by the IHME and MRC GIDA." (Bracketed portion added). I feel it's important to keep this paragraph in its entirety in the article because it illustrates the stark contrast between the modeling predictions and reality of COVID-19.
Therefore the timeline in summary is as follows; Italy locks down in early March, the first democratic nation to do so, the large compliance of its citizens to government mandates shows that lockdown policies are possible in the Western world. March 16th the Imperial College releases its report, modeling the spread of COVID-19 in the U.K and U.S stating that even with the most aggressive mitigation strategies (case isolation, home quarantine and social distancing of the elderly) hospital capacity will be surpassed eight fold. This causes the U.K to drastically switch its COVID-19 mitigation plan from obtaining herd immunity to lockdown based policy this after a week of the report becoming public the U.K announces lockdown on March 23rd. The virus then continues to spread and makes landfall in the U.S via New York. New York City doubles its ICU capacity by April 9th in response to the virus. Come May 12th the Society of Critical Care Medicine publishes a study analyzing the situation in New York through the lens of ventilator availability and finds that the resources deployed to New York like the field hospital in the Javits center and the Navy hospital ship where underutilized.
That's really the part to take away from this is that the modeling predicted an 8-fold over capacity hospitalization limit and NYC only doubled their capacity and with only double the resources available they were underused during the surge of the virus. This shows the large discrepancy between modeling and real world data, especially with an organization that has a track record for over estimating the severity of viruses.
In the end modeling should be done and its predictions should be used to influence, not make government policy. Also politicians and the media should have done a much better job at making the public aware of the assumptions that are made in these models. Furthermore the public should also be updated on the discrepancies seen between real world contact with the virus and what the modeling predicted. Leaders should have stayed level headed when presented with these papers and not have acted on a panic trigger by taking the predictions as gospel. Lastly a free press should have criticized lockdown policy and held leaders accountable for why they were willing to take such drastic measures based on essentially an educated guess. So next time you hear a newscaster say "modeling data shows...." take the prediction for what it is; an educated guess.
Footnotes:
1) Jason Horowitz, “Italy Announces Restrictions over Entire Country in Attempt to Halt Coronavirus,” New York Times, March 9, 2020, https://www.nytimes.com/2020/03/09/world/europe/italy-lockdown-coronavirus.html
2) “Options for Increasing Adherence to Social Distancing Measures,” SAGE, March 22, 2020, https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/882722/25-options-for-increasing-adherence-to-social-distancing-measures-22032020.pdf
3) Neil M. Ferguson et al., “Report 9: Impact of Non-Pharmaceutical Interventions (NPIs) to Reduce COVID-19 Mortality and Healthcare Demand,” Imperial College COVID-19 Response Team, March 16, 2020, https://www.imperial.ac.uk/media/imperial-college/medicine/sph/ide/gida-fellowships/Imperial-College-COVID19-NPI-modelling-16-03-2020.pdf
4) Bill Bostock, “How ‘Professor Lockdown’ Helped Save Tens of Thousands of Lives Worldwide—and Carried COVID-19 into Downing Street,” Business Insider, April 25, 2020, https://www.businessinsider.com/neil-ferguson-transformed-uk-
covid-response-oxford-challenge-imperial-model-2020-4
5) Kate Wong, “ ‘Mad Cow’ Sheep in Britain Could Increase the Human Death Toll,” Scientific American, January 10, 2002, https://www.scientificamerican.com/article/mad-cow-sheep-in-britain/
6) “Variant Creutzfeldt-Jakob Disease Current Data (February 2015),” University of Edinburgh, February 13, 2015, available at the Wayback Machine at https://web.archive.org/web/20150226031911/http://www.cjd.ed.ac.uk/documents/worldfigs.pdf
7) James Sturcke, “Bird Flu Pandemic ‘Could Kill 150M,” The Guardian, September 30, 2005, https://www.theguardian.com/world/2005/sep/30/birdflu.jamessturcke
8) Avian Influenza(H5N1) – United States of America, World Health Organization, May 6 2022, https://www.who.int/emergencies/disease-outbreak-news/item/2022-E000111
9) "Scientific Advice and Evidence in Emergencies—Science and Technology Committee,” Parliament of the United Kingdom, March 2, 2011, https://publications.parliament.uk/pa/cm201011/cmselect/cmsctech/498/49803.htm
10) Post Editorial Board, "Virus Models aren't crystal balls and other commentary", NYPost, April 5, 2020, https://nypost.com/2020/04/05/virus-models-arent-crystal-balls-and-other-commentary/
11) Lydia Ramsey Pflanzer and Jeremy Berke, "Converted Operating Rooms and Shuffled patient's: How NYC scrambled to turn 1600 ICU beds into 3500 to care for the sickest Coronavirus patients.", Business Insider , April 9th 2020, https://www.businessinsider.com/coronavirus-nyc-more-than-doubled-its-icu-capacity-in-weeks-2020-4?op=1
Neil A. Halpern and Kay See Tan, “United States Resource Availability for COVID-19,” Society of Critical Care Medicine, May 12, 2020, https://sccm.org/getattachment/Blog/March-2020/United-States-Resource-Availability-for-COVID-19/United-States-Resource-Availability-for-COVID-19.pdf?lang=en-US
*I just want to take this opportunity to give a shout out to Alex Berenson and his book Pandemia. That book lead me to a lot of this source material as well making me aware of the Imperial college.*
*This was edited by me (Darren Gabrylewicz) May 29th 2022, the edits consisted of a general grammar and spelling and footnote clean-up*