Vallance vs Vallance vs SAGE: Why does doubling time matter?

One thing that all the protagonists (myself included) agree on is that SAGE’s estimate of doubling time in mid-March was of critical importance. Vallance testified forcefully to this effect in 2020:

When the SAGE sub-group on modelling, SPI-M, saw that the doubling time had gone down to three days, which was in the middle of March, that was when the advice SAGE issued was that the remainder of the measures should be introduced as soon as possible.

And then confirmed in this exchange with a committee member:

Sir Patrick Vallance: Knowledge of the three-day doubling rate became evident during the week before. 

Q1080 [ed – this number has changed from the previous time I posted about this] Graham Stringer: Did it immediately affect the recommendations on what to do?  

Sir Patrick Vallance: It absolutely affected the recommendations on what to do, which was that the remaining measures should be implemented as soon as possible. I think that was the advice given.

and again:

Sir Patrick Vallance: The advice changed because the doubling rate of the epidemic was seen to be down to three days instead of six or seven days. We did not explicitly say how many weeks we were behind Italy as a reason to change; it was the doubling time, and the realisation that, on the basis of the data, we were further ahead in the epidemic than had been thought by the modelling groups up until that time.

But why does the doubling time matter so much, and is the difference between 3 days and 5 day really important? The point of this post is to answer these questions.

I’ll start by repeating a couple of plots I first made way back in April 2020 when it was just dawning on me what a colossal cock-up SAGE had made of it. These graphs are generated by a simple SEIR model that I’ve shown to reasonably replicate more sophisticated ones. In the below, the blue “calibrated” line uses data up to 14 March to estimate doubling time, which comes out at 3 days. The red “uncalibrated” line uses parameters from Ferguson’s March 16 paper, which has a doubling time of 5 days.

So, one obvious point is that the blue curve does a much better job of predicting what was going to happen (ie, the magenta x, which were not used to calibrate either model). But that’s not the point of this particular post. Rather, it’s just that the predictions for 3d vs 5d doubling are radically different. Here is the longer-term view, now without the logarithmic scaling on the axis:

Note the “we are here” point is mid-March, when the red and blue curves are visually indistinguishable.

There are several reasons why the doubling time matters. Firstly, the estimate of current pandemic size is based on historic data. Edmunds specifically talks of a 12 day delay from infection to illness, to testing, to finally test data reporting. So 100 cases reportedtoday means 100 infections 12 days ago, and that’s 4 doublings at 3d doubling, meaning 1,600 cases now. If the doubling time was only 5d, then the 12 day delay is just over 2 doublings, and we wouldn’t be at 1,600 cases for another 8 days (4 doublings is 20 days, so that means 12 just past and another 8 in the future). So one immediate consequence of a change in perceived doubling time is that we’ve lost just over a week in terms of pandemic progression. (These calculations all ignore the proportion of infections that are undetected, which can reasonably assume to be roughly constant and thus not affect the argument.) 

Secondly, the 3d doubling means the pandemic comes much sooner, and (thirdly) reaches a much higher peak, as shown in the 2nd graph above. What was expected some time over the summer, is now happening next month, and it’s going to be a lot worse than expected – getting on for twice as many cases per day at peak.

Finally, the more rapid doubling implies a higher R0-number, and this makes it harder to control. With R0=2.4 (Ferguson’s 16 March number), a reduction of contacts to 40% of normal would control the virus, because 2.4 x 0.4 = 0.96 which is less than 1. Whereas a 3d doubling implies a rather higher R0, let’s say 3.2. This then requires much stiffer action, because 3.2 x 0.4 = 1.28 which is still greater than 1. A reduction to 30% of previous contact levels to get the R number below 1 (3.2 x 0.3 = 0.96 as before) is obviously much tougher to achieve especially given that family members, essential work etc mean we can’t really isolate perfectly. (Quibbling over the exact value of R0 to use doesn’t invalidate the general point that a higher R0 is harder to control.)

So, changing the estimate of doubling time from 5d to 3d would be a real “oh shit” moment for anyone involved in pandemic planning. It means (a) we’ve instantaneously lost 8 days of lead-time, (b) the pandemic peak is going to be coming a month sooner than expected, (c) the peak is going to be almost twice as big as expected, and (d) the control measures we were hoping to use are much less likely to be adequate. Any plan predicated on a 5d doubling time would immediately have to be revisited in the most extreme and urgent manner.

I hope I have convinced my reader that whatever plans were in place in early March under the assumption of a 5 day doubling, the new understanding that the doubling time was instead 3d would cause an abrupt and substantial change of perspective. This, from a scientific perspective, is inevitable and obvious.

Vallance vs Vallance vs SAGE: Introduction

Ok I’m going to do a bit of analysis of Vallance’s evidence to the UK Covid-19 Inquiry, focussing specifically on the events of the mid-March 2020 period up to the imposition of the first “lockdown” on 23rd March. On Monday 20th Nov 2023 he was interviewed by the Inquiry and also provided some written testimony. This is broadly speaking a more detailed version of the testimony he provided back in July 2020 to the House of Commons Science and Technology Committee (which I blogged about at the time) but appears significantly inconsistent with the documentary evidence provided by the minutes of SAGE meetings and other records of that period. If you already agree with me that Vallance misled the S&TC with his testimony in 2020 then you might not find this very interesting, but I think I might as well go over it again as there’s a lot more testimony to consider (including other participants in SAGE).

I’ll break it up into sections in order to make it digestible, and also to avoid me going round in ever decreasing circles. To start with, let’s consider some background concepts…

UK Covid-19 Inquiry: Module 2

The currently ongoing Module 2 describes its aims thusly:

This module will look at, and make recommendations upon, the UK’s core political and administrative decision-making in relation to the Covid-19 pandemic between early January 2020 until February 2022, when the remaining Covid restrictions were lifted. It will pay particular scrutiny to the decisions taken by the Prime Minister and the Cabinet, as advised by the Civil Service, senior political, scientific and medical advisers, and relevant Cabinet sub-committees, between early January and late March 2020, when the first national lockdown was imposed.

I’m particularly interested in this period, as it’s the time that the expert scientific analysis and advice from SAGE was so woefully inadequate. I’ve blogged about this at length, but just to recap, the scientists were mistakenly thinking that the doubling time of the pandemic was about 5-6 days (various numbers appear in the SAGE minutes) and that we shouldn’t take too stringent measures as there was a genuine risk that by doing so we’d put the pandemic off to the following winter when it would add to the normal seasonal pressures on the NHS. They were quite anxious that we should get through it over the summer of 2020 instead.

Vallance misled the Science and Technology Select Committee a while ago about this, claiming that SAGE had recommended lockdown on the 16th or 18th of March. This is contradicted by the minutes of those meetings, and even if you try to argue that the minutes may not be completely definitive on that, it is also contradicted by his accompanying statement that their change of heart was due to correcting their estimate of the doubling time (to 3 days), which the SAGE minutes document very precisely to the 23rd March. He’s due to give evidence to the UK Covid-19 Inquiry on Monday, so I await with interest to see whether he will correct the record or also mislead them.

This error was not just an inconsequential comment in a committee that no-one cares about, but has been widely reflected in press comment. For example, the usually excellent Lewis Goodall on Xitter:

Doesn’t seem to me that what @uksciencechief said earlier has received as much attention as it should, in the day’s tsunami of news. In saying that SAGE advised the govt to lockdown a week earlier than they did, yet more space is opening up between them. pic.twitter.com/cKuWXlh7vH

— Lewis Goodall (@lewis_goodall) July 16, 2020

UK Covid-19 Inquiry

 I can see I’m going to have to go over all this again. It’s not a task I really face with much enthusiasm, but it doesn’t seem like anyone else is prepared to do it. To say I’m disappointed at the revisionism, sleight-of-hand and downright misleading testimony from several senior scientists to the UK Covid-19 Inquiry would be an understatement. I had naively hoped there might be some element of humility, introspection and self-reflection concerning their errors at the start of the outbreak, but I’ve seen no hint of this. (If anyone wants to reassure me that lessons have been learnt internally, then I’m all ears, but would want to see evidence of this.)

Unfortunately, neither the inquisitors themselves, nor the journalists following the process, nor the array of commentators eagerly quoting the juicy messages, seem to have the will or perhaps the scientific skills to unpick the story. That’s not to say it is hugely complicated, but a basic understanding of the underlying mathematics is vital for piecing together how it all played out, and why. And it’s very clear that most people start out with an agenda and go looking for support, rather than really being interested in understanding the truth. The scientists are delighted to have found a route to blaming the politicians, and the politicians are too focussed on knifing each other to question what the scientists are now claiming the history to be.

That’s not to say I’m perfect (far from it), but on this particular topic, I happen to be correct. A more difficult question, is whether anyone else cares. Anyway, on with the show….

Retired

As you may have noticed, there hasn’t been a lot of science getting done here recently. 

The basic reason for this is that we’ve decided to retire and close down Blue Skies Research Ltd. We set it up about 10 years ago, when we returned from Japan, and have had a lot of fun continuing our research in a private setting but over the last few years have been gradually winding down the research activity and increasing the other-than-research activity and want to focus on the latter from now on.

There’s a paper in the works with paper charges still to pay so the company isn’t completely shut down yet. We aren’t looking for new projects but if something exciting comes up, we might change our minds.

To be honest, we haven’t been particularly inspired by new ideas for a while and simply don’t have any burning climate science questions that we need to answer. After all, we have worked out what Equilibrium Climate Sensitivity is (actually we worked it out in 2006, but everyone else took 15 years to catch up). There are lots of other scientists quite capable of taking the field wherever they choose to, and we look forward to seeing where they go!

No comment necessary

 

Chess

Many years ago, I played chess as a schoolboy. Not all that brilliantly, but good enough for the school team which played in various competitions. This fell by the wayside when I went to university, and I'd never had the time or energy to re-start though kept on playing against my uncle when we met. A couple of years ago during covid lockdowns I started playing on-line on chess.com, and then more recently someone started a chess club in Settle where a small bunch of us have been playing fairly informal and quick games. Last weekend was my first proper over-the-board competition, at the very conveniently located Ilkley Chess Festival. I'd naively assumed this would be a local event for local people, but my opponents came from all over, hailing from Portsmouth, Nottingham, Shrewsbury, and even Scarborough. There were also some Scots on the entry list that I didn't meet.

I've blogged the event on the chess.com site (here and here) as that allows for embedding of games. Spoiler alert: after losing the first game, I won the next 4, ending in 4th place. In the “Intermediate” section, which means under-1750 rated. (I don't have a current rating for OTB chess, so had to guess which section to enter. At school I was about 1450.)

Someone was taking pictures, so here is a picture of the main hall:

and here I am, about to win my 3rd game:

BlueSkiesResearch.org.uk: EGU 2022 – how cold was the LGM (again)?

I haven’t blogged in ages but have actually done a bit of work. Specifically, I eventually wrote up my new reconstruction of the Last Glacial Maximum. We did this back in 2012/3 (see here) but since then there have been lots more model simulations, and then in 2020 Jessica Tierney published a new compilation and analysis of sea surface temperature proxy data. She also produced her own estimate of the LGM temperature anomaly based on this data set, coming up with -6.1±0.4C which seemed both very cold and very precise compared to our own previous estimate of -4.0±0.8C (both ranges at 95% probability).

We thought there were quite possibly some problems with her result, but weren’t a priori sure how important a factor this might have been, so that was an extra motivation to revisit our own work.

It took a while, mostly because I was trying to incrementally improve our previous method (multivariate pattern scaling) and it took a long time to get round to realising that what I really wanted was an Ensemble Kalman Filter, which is what Tierney et al (TEA) had already used. However, they used an ensemble made by sampling internal variability of a single model (CESM1-2) and a few different sets of boundary conditions (18ka and 21ka for LGM, 0 and 3ka for the pre-industrial), whereas I’m using the PMIP meta-ensemble of PMIP2, PMIP3, and PMIP4 models.

OK, being honest, that was part of the reason, the other part was general procrastination and laziness. Once I could see where it was going, tidying up the details for publication was a bit boring. But it got done, and the paper is currently in review at CPD. Our new headline result is -4.5±1.7C, so slightly colder and much more uncertain than our previous result, but nowhere near as cold as TEA.

I submitted an abstract for the EGU meeting which is on again right now. It’s fully blended in-person and on-line now, which is a fabulous step forwards that I’ve been agitating for from the sidelines for a while. They used to say it was impossible, but covid forced their hand somewhat with two years of virtual meetings, and now they have worked out how to blend it. A few teething niggles but it’s working pretty well, at least for us as virtual attendees. Talks are very short so rather than go over the whole reconstruction again (I’ve presented early versions previously) I focussed just on one question: why is our result so different from Tierney et al? While I hadn’t set out specifically to critique that work, the reviewers seemed keen to explore, so I’ve recently done a bit more digging into our result. My presentation can be found via this link, I think.

One might assume a major reason might be that the new TEA proxy data set was substantially colder than what went before, but we didn’t find that to be the case. In fact many of the gridded data points coincide physically with the MARGO SST data set which we had previously used, and the average value over these locations was only 0.3C colder in TEA than MARGO (though there was a substantial RMS difference between the points, which is interesting in itself as it suggests that these temperature estimates may still be rather uncertain). A modest cooling of 0.3 in the mean for these SST points might be expected to translate to about 0.5 or so for surface air temperature globally, not close to to the 2.1C difference seen between our 2013 result and their 2020 paper. Also, our results are very similar when we switch between using MARGO and TEA and both together. So, we don’t believe the new TEA data are substantially different from what went before.

What is really different between TEA and our new work is the priors we used.

Here is a figure summarising our main analysis, which follows the Ensemble Kalman Filter approach, which means we have a prior ensemble of model simulations (lower blue dots, summarised in the blue gaussian curve above) each of which is updated by nudging towards observations, generating the posterior ensemble of upper red dots and red curve. I’ve highlighted one model in green, which is CESM1-2. Under this plot I have pasted bits of a figure from Tierney et al which shows their prior and posterior 95% ranges. I lined up the scales carefully. You can see that the middle of their ensembles, which are entirely based on CESM1-2, are really quite close to what we get with the CESM1-2 model (the big dots in their ranges are the median of their distributions, which obviously aren’t quite gaussian). Their calculation isn’t identical to what we get with CESM1-2, because it’s a different model simulation, with different forcing, we are using different data and there are various other differences in the details of our calculation. But it’s close.

Here is a terrible animated gif. It isn’t that fuzzy in the full presentation. What it shows is the latitudinal temperatures (anomalies relative to pre-industrial) of our posterior ensemble of reconstructions (thin black lines, thick line showing the mean), with the CESM-derived member highlighted in green, and Tierney et al’s mean estimate added in purple. The structural similarity between those two lines is striking.

A simple calculation also shows that the global temperature field of our CESM-derived sample is closer to their mean in the RMS difference sense, than any other of our ensemble members. Clearly, there’s a strong imprint of the underlying model even after the nudge towards the data sets.

So, this is why we think their result is largely down to their choice of prior. While we have a solution that looks like their mean estimate, this lies close to the edge of our range. The reason they don’t have any solutions that look like the bulk of our results is simply that they excluded them a priori. It’s nothing to do with their new data or their analysis method.

We’ve been warning against the use of single model ensembles to represent uncertainty in climate change for a full decade now, it’s disappointing that the message doesn’t seem to have got through.

Marmalade Training Camp

A trip to Scotland last weekend to learn the ancient art of Victorian Marmalade Making from marmalade sensei, the Mother in Law. It turned our to be less art and more chemistry!  I still don't quite understand how it worked, but it did. Maybe it is actually magic. It was great weather for the project; continuous rain for 3 days. 

Step 1. Get Seville oranges, and the same mass of lime and lemons. These kind of oranges are mostly pith and pips, taste very bitter, and can only be found in January, although not only in Scotland. Wash and remove the ends, and any nasty ones.

 

 

Step 2. Juice fruits! An acceptable diversion from Victorian tradition is to use an electric juicer. The juice goes into the juice pot, the pith and pips into the pith and pits pot, and the shells of rind go to the slicer. The slicer is a large heavy metal thing that clamps to the table, a handle is turned and sliced peel comes out of the bottom. A non-Victorian alternative to the slicer is unknown.

Step 3. Add water to the pith and pips bowl and to the rind. pints of water = 1.1 x weight of fruit in lbs, with about 0.1 going into the pith.

Step 4. Soaking the fruit is neither here not there as far as the chemistry/magic is concerned, apparently. But by now you will be tired, so you can take a break ... overnight if you like.

Don't forget your cat!

Step 5. Find cauldron! Put rind-marinade into cauldron.

Step 6. Manufacture a bag from cloth and string that contains the pith and pips, and suspend in cauldron. This bag contains the magical carbohydrate pectin which is required to make the marmalade set. Bring to boil and cook for an hour (the internet suggests 2-3 hours for bright, tender marmalade. The internet might be wrong.). Apparently the acid from the fruit helps get the pectin out, but I don't understand this, becuase the juice is not yet added at this stage.

Step 7. Turn off the heat. Extract bag from cauldron and squeeze it hard to get out all the pectin. 

Step 8. Add juice.

Step 9. Add sugar

Step 10. Add more sugar

Steps 11-13. Add yet more sugar. About 1.6x weight of fruit in total!!!!

Step 14. Bring slowly to a rolling boil. 

 


Step 15. Excitedly test every 5 seconds to see if it is done yet. It is done when it sets. This is the magic/chemistry bit. Pectin and acid and heated up sugar and do something or other that makes - jelly. But this isn't the same as caramalisation that you use to make toffee, which is more like burning sugar. In fact you want as little caramalisation as possible, because marmalade shouldn't taste like toffee. This is why the internet says boil for 15-20mins. The internet also says too much boiling at this stage make the rind tough. It was more like an hour for us, but our marmalade is still pretty and the rind very nice. Maybe internet people want the rind to melt in their mouths or something weird?

Anyway, you can test by cooling a small spoonful on a plate and when it starts to set it is done, or use a Victorian thermometer. Not sure what the markings on the thermometer engraved by ancestors mean, but when the brass holder gets all sticky with globs of marmalade, it is done.

Step 16. Remove from heat and quickly fill up all your jars (which, hopefully, appear beside you by magic) and screw the lids on ASAP.  

 

Optional Step. Next day, if some of your jars are not screw top, or they are screw top but the button on the lid didn't go down as the marmalade cooled, or you don't have lids... melt paraffin wax (in a jug in boiling water) and pour over the top and slap on some kind of lid! Marmalade will stay good for ... 3 years or so?

Step 17. Eat. Yum yum!

 

Omicron

It occurred to me that the talk of perhaps bringing in restrictions some time in the future was probably poorly timed, in that we are probably pretty close to the peak right now and if action is going to be worthwhile, it needs to be pretty much immediate. Having made a few comments to that end on twitter, I thought I should check out my intuition with some calculations. So here they are.

My starting point is that the Omicron variant represented 22% of tests on the 11th Dec (link) and we had about 40k positive tests on that day (link - but see additional note at bottom of post) meaning 9k tested cases which I will assume represents 18k real infections (ie 50% of infections are actually observed) and furthermore I'll assume that these infections happened on the 8th as it must take a little while to feel ill and get tested. 

I'm using a doubling time of about 2 days with an underlying R0 number of 6, and another assumption I'm making is that the population is about 50% immune. I'm ignoring the Delta infection which is small in comparison and carries on largely in parallel with Omicron.

So I initialise the model to hit 18k infections on the 8th, and ran it forwards. This is what I get with no action at all, just the natural infection profile of an uncontrolled epidemic:

32 million infections in total, with a daily peak of 2.7 million on the 27th.

If instead we were to introduce severe restrictions now, such that the underlying R0 dropped from 6 to 1.5, the epidemic would be much smaller:

A daily max of about 430k infections and only 4 million in total. Note that the underlying R0 dropping to 1.5 means the effective R value drops to about 0.75 as the population is half immune.

However the Govt seems to be slowly meandering towards the possibility of some restrictions in about a week. If we were to say Boxing Day instead, then we get:

The daily max here is 2.4 million, with the total about 16 million. So even this delayed action does cut the epidemic in half, by shutting it down rapidly from the peak. That's a bit better than my intuition had suggested to me.

The details of these calculations are sensitive to the timing of the peak of course, which depends on all the assumptions I've made. What is not in doubt is that every day makes quite a big difference to the outcome.

Edit: In the time it took me to write this post, the number of cases by specimen date on the 11th has been updated to 46k!