Wednesday, April 15, 2020

BlueSkiesResearch.org.uk: Model calibration, nowcasting, and operational prediction of the COVID-19 pandemic

Title as post. Yes, this is us dipping our toes into epidemiology. Turns out that calibrating a simple model with observational data is much the same whether it’s paleoclimate or epidemics. The maths and the methods are much the same. In fact this one is a particularly easy one as the model is embarrassingly linear (once you take the logarithm of the time series). I’ve been posting my analyses on Twitter and the other blog, but since this is a real paper with words and figures and references and stuff, it can go here too (plus, I can upload a pdf here unlike blogspot).

We have been doing a very straightforward MCMC calibration of a simple SEIR model (equivalent of energy balance box model in climate science, pretty much). The basic concept is to use the model to invert the time series of reported deaths back through the time series of underlying infections in order to discover the model parameters such as the famous reproductive rate R. It’s actually rather simple and I am still bemused by the fact that none of the experts (in the UK at least) are doing this. I mean what on earth are mathematical epidemiologists actually for, if not this sort of thing? They should have been all over this like a rash. The exponential trend in the data is a key diagnostic of the epidemic and the experts didn’t even bother with the most elementary calibration of this in their predictions that our entire policy is based on. It’s absolutely nuts. It’s as if someone ran a simulation with a climate model and presented the prediction without any basic check on whether it reproduced the recent warming. You’d get laughed out of the room if you tried that at any conference I was attending. By me if no-one else (no, really, I wouldn’t be the only one).

Anyway, the basic result is that the method works like a charm and we can reliably deduce the changes in R due to imposed controls, and it looks increasing clear that it’s been less than 1 in the UK for several weeks now, while the experts are still talking about the peak being a couple of weeks away. The whole experience is just…so strange.

Anyway, I did try talking politely to some of the experts but just got brushed off which may partly explain the tone in the manuscript. Or maybe that’s just me 🙂

The paper has been submitted to medrxiv but who knows what they will make of it. My experiences when I have poked my nose into other peoples’ fields has not usually be a very encouraging one so I’m half expecting them to reject it anyway. So be it.

Here is today’s forecast to encourage you to read the paper.

uk_14_apr

17 comments:

PeteB said...

As you've said, today might be a bit of an outlier - weekend & bank holiday effect for 4 days from Good Friday - Easter Monday

Frank said...

Hi James, in Germany we have a similiar situation. A today released "pandemic bulletin" (in German) of the "Robert Koch Institut" ( it's the official department of the German gouvernment!)https://www.rki.de/DE/Content/Infekt/EpidBull/Archiv/2020/Ausgaben/17_20_SARS-CoV2_vorab.pdf?__blob=publicationFile they show in Fig. 4 the "R estimation" and conclude it's near 1 as of yesterday. Are you able do use the German data to produce a more reliable R-estimation?
Thanks in advance

KenH said...

Do you think you will look at the US?

lucia said...

Cool!!!

Steve Reynolds said...

I'm curious how for Sweden, 'R0 = 2.5 ± 0.3' can be estimated accurately with such sparse data (appears to be only about 7 deaths prior to imposing controls).

Nathan said...

Projected 720, had 761... That's pretty good for your model.. unfortunate for the people...

crandles said...

Typo top of page 12 "piblic domain".

(Well you said you wanted to encourage people to read it.)

crandles said...

Suppose I should do a better job, there is also:

page 8 "wth"
Fig 2b "for UK" perhaps should be 'for the UK'?

page 11 "These results demonstrates"
Page 11 "We are have"

crandles said...

Can you also predict where China's long tail of new daily new cases goes? I guess that is a harder problem that depends a lot on what international travel is allowed and whether contact tracing is sufficient to prevent internal transmission restarting?

crandles said...

>"I'm curious how for Sweden, 'R0 = 2.5 ± 0.3' can be estimated accurately with such sparse data (appears to be only about 7 deaths prior to imposing controls)."

12540 current confirmed cases is rather more than a handful/7. Of course, this comes with its own problems; it depends not only on disease prevalence but also on how much testing done and when. If to first order, testing ramps up with disease prevalence then confirmed cases give a fairly good idea but there is room for such an assumption to be wrong so I have no idea how to calculate the level of uncertainty in this case.

Still R0 likely to be similar to similar countries with similar lack of controls so why do you think the error bounds need to be much wider?

Witold said...

Very interesting paper. I have recommended it to my colleagues, who are in a broad community advising government in Poland. By the way - on medrxiv.org they state that they will not consider papers that have been published elsewhere - including your own site. Maybe you should pull down paper from the blog for the screening period. I have pulled mine down from the site, where I try to estimate the number of cases worldwide if testing was performed to a German standard.

Jim Hunt said...

Daily deceased up to 847 today.

Not to mention John Conway, and Sir John Houghton, and Norman "Bite yer legs" Hunter.

crandles said...

Weekly cycle in new cases seems to be looking likely in a few countries? Also in deaths? Presumably this is in reporting effects rather than actual. Wondering if recognizing this might help with future predictions.

Meanwhile 761 861 847 888 is well inside your error bounds but OTOH doesn't seem to be showing the downward trend you seemed so confident about. On third hand, still below 980 peak.

Phil said...

Spreading is mostly before symptoms.

https://www.nature.com/articles/s41591-020-0869-5

Frank said...

A little o/t, however congrates for the today released paper on ECS!!

lucia said...

BTW: A blog visitor pulled in Illinois data and ran your model

http://rankexploits.com/musings/wp-content/uploads/2020/04/KennethsGraph.png

There could be some slight dating issues, but I'm happy to see a prediction with R<1 for after Governor Pritzker's "stay at home" order.

ChangeWithin said...

Excellent effort! Based on current data, the data follows the upper bound of prediction. How can the model be improved, or assumptions revised to for more reliable prediction for say the next 30 days?