SAGE Models Are GIGO – Garbage In, Garbage Out
Wed 12:05 pm +01:00, 24 Feb 2021
Erstwhile sceptic Christopher Snowdon may have crossed the aisle of late to become a lockdown enthusiast and Toby’s sparring partner – they’re in action again today, head-to-head on talkRADIO, more details tomorrow – but he hasn’t entirely lost his critical faculties when it comes to SAGE BS. He regards the vaccines as the way out of the crisis, and he is withering about the latest modelling from SAGE, which he thinks makes far too many pessimistic assumptions regarding the vaccination programme. He starts with the team from Warwick.
In their best case scenario, lockdown reduces R to 0.8 and 3 million vaccines are delivered each week from February (with 1 million a week delivered in January). Lockdown ends on 22 February and all NPIs are dropped by July. This means that everybody who wants a vaccine has had one (or, indeed, two) by the summer. The authors nevertheless predict that England will see 2,000 deaths per day in August. If the vaccines don’t work as well as expected, this rises to nearly 5,000 per day.
I am only an interested amateur and am happy to be put straight, but WTF?!? Last August, when there was no vaccine and minimal NPIs, England had about eight deaths a day. At the height of the winter second wave, it had 1,238 deaths (January 19th). 5,000 deaths per day is more than 70 per million. Even the worst hit countries such as Belgium and Czechia never got above 30 per million at the height of their epidemics. And yet these guys think that it could far exceed that in Britain in the summer after the vaccines have been fully rolled out. Even in their best case scenario, there would still be 1,000 deaths a day.
When your model gives you such an implausible result, you have to question your assumptions. So what are they?
According to the brief text, the authors expect all these deaths to come about because some people will refuse to take the vaccine and some people who take the vaccine won’t be protected.
“Uptake: Throughout we assume 95% uptake in care homes, 85% in the general population above 50 and 75% in adults below 50 for the first dose. This drops to 75% for the over 50s and 66% for the under 50s for the second dose.”
People aged under 50 are almost irrelevant in terms of Covid mortality so the figure to focus on is 85%. Here are their assumptions about protection against symptomatic disease:
“Efficacy: We sub-divide into the effects of protection against symptoms (disease efficacy) and reduction in transmission – we assume that transmission blocking acts by stopping infection. Disease efficacy is taken as 70% and 88% after dose 1 rising to 88% and 94% after dose 2 for the Oxford and Pfizer vaccine respectively. Transmission efficacy is taken to be 48% rising to 60% for both. Protection is lagged by 14 days after the dose is delivered.”
This roughly reflects what the trials have shown us about these vaccines.
The authors don’t provide a figure for the total number of deaths in their projected third wave, but it looks around twice as bad as the second wave which has killed about 60,000 people in England and will probably end up killing around 75,000. So, as a very rough estimate, they’re suggesting there will be 150,000 deaths after everybody who wants a vaccine has had one. That’s more than all the deaths we’ve had already. Unsurprisingly, the authors call for the government to hold back on relaxing lockdown, although quite what this would achieve is unclear: the lesson of the study is that COVID-19 will get us all eventually.
Imperial have also done some modelling. This study was produced on 14 January, the day after Warwick’s. It makes exactly the same assumptions about transmission efficacy and disease efficacy. Unlike Warwick, they assume vaccine uptake is 85% across all age groups. They assume that NPIs are gradually lifted on the first day of each month but do not say which ones. As with Warwick, all NPIs are lifted by 1 July.
The assumptions are therefore very similar and the conclusions are only slightly less gloomy. Like Warwick, they predict a massive summer wave and 130,800 dead even under their most optimistic scenario.
He spies two problems.
Firstly, both models underestimate how many people will take the vaccine. So far, 91% of the 80+ cohort have taken it and 96% of those aged 75-79 have taken it. That’s a lot more than 85% assumed in these models.
Secondly, and more importantly, the estimates of disease efficacy (the reduction in risk of getting symptomatic disease) are roughy correct, but the authors seem to have overlooked the crucial point about the AstraZeneca vaccine which is that there were ‘no severe cases and no hospitalisations’ in the trials. The vaccine seems to be 100% effective in preventing death from COVID-19. The Pfizer vaccine is 95% effective in preventing symptomatic disease altogether, so the number of deaths that would occur among the cohort who take it would, presumably, be very low (in Israel, there were 4 severe cases and no deaths among 523,000 vaccinated people).
It is incredible that neither study factors in the effect of the vaccines on severe disease and mortality. You can forgive them for not predicting that so many people would take up the vaccines, but we’ve known about the AZ vaccine’s ability to prevent severe disease and death since November.
Worth reading in full. He expands his argument in this Twitter thread. And also in the Telegraph.
As sceptics we would want to add a few more problems to the list. We’d ask why seasonality is ignored and mass deaths are predicted in the height of summer, despite almost zero Covid last year with few restrictions and no vaccines. We’d wonder why prior infections, innate immunity and cross T-cell immunity from other coronaviruses are ignored, and why the IFR is so high compared to those estimated by the WHO and CDC.
So many poor assumptions. As the Americans say, Garbage In, Garbage Out.
Stop Press: Dr Anthony Fauci, head of the National Institute of Allergy and Infectious Diseases in America, declared on Monday during a White House COVID-19 press briefing that people who are vaccinated against COVID-19 still shouldn’t dine indoors at restaurants or go to cinemas “because of the safety of society”. This is despite the positive cases in the country continuing to decline. They do seem to be missing the point of the vaccine, don’t they?
Stop Press 2: CRG chair Mark Harper has tweeted his questions about the dubious modelling assumptions.
The Most Chilling Sentence in the Roadmap
Steve Sieff from GreenBandRedBand has got in touch to say he thinks we overlooked the worst part of the roadmap. He explains here.
I am one of the three who has read the roadmap in full. For me the most chilling paragraph is one you didn’t quote (emphasis mine).
64. For these reasons, a significant proportion of the population could still be infected, either because they have not been vaccinated or because the vaccine is not effective for them. This is illustrated by Figure 7. This could mean that some measures to limit transmission are still needed after all adults have been offered a vaccine. These could include guidance such as “hands, face, space”, maintaining the Test, Trace and Isolate system and controls at the border (see chapter 4). The extent to which such measures will be required after all adults have been vaccinated is still unknown. As set out in chapter 3, the Government is exploring what measures may be required.
I also note that although the document makes several references to the various harms caused by the restrictions, it does not include within those harms the basic objection that all of the measures taken reduce our ability to live freely. Even for the ‘lucky’ individuals who have not been physically or mentally affected by the ongoing measures, restrictions on freedom should still be seen as a last resort where no other answer is available. One of the stated aims of the government in the roadmap is “to restore freedoms sustainably, equitably and as quickly as possible” but in the context of the whole document it feels very much like that is the very bottom of the priority list.
So on the plus side we have the idea that Zero Covid is not realistic. That is a big relief for sure. But on the negative we have a painfully slow re-opening, none of which is guaranteed, with the most far-reaching changes being subject to further reviews. No doubt these reviews will be carried out by… SAGE.
There is also an increase in use of face coverings by one of the groups least likely to be affected (schoolchildren), the clear acknowledgement that vaccine passports for international travel will be required and the concession that some form of immunity certificate might well be used domestically. We also have a long term plan based heavily on continued testing of asymptomatic people with the aim of heavy enforcement of local outbreaks. Possibly also the longer term continuation of distancing and masks to some extent. So I cannot help but read this document as a map to the dreaded ‘new normal’ and a nail in the coffin of pre-Covid freedoms. Covid will be endemic and comparable to flu, but we will be expected – on pain of criminal sanction – to continue with measures which exceed any reasonable assessment of the risk which will remain.
One other small point. As you are aware, there is still no legislation obliging schools to close. It was Government advice during the first lockdown and the same approach was taken with the current lockdown. The wording of the roadmap hints that the face masks in schools changes may also be by way of recommendation rather than obligation. It may not make any difference to schools who feel obliged to comply, but it would be marginally better than making them mandatory by law (emphasis mine).
92. The Government also recommends that the use of face coverings in Higher Education, Further Education and secondary schools is extended for a limited period to all indoor environments – including classrooms – unless 2m social distancing can be maintained. Face coverings are now also recommended in early years and primary schools for staff and adult visitors in situations where social distancing between adults is not possible, for example, when moving around in corridors and communal areas.
https://lockdownsceptics.org/
