...is the title of an interesting TV programme that was on BBC 4 last night. It is quite amazing that they dared to show such a maths/stats/science-heavy program at prime time, albeit on a minor channel, so I will start by commending them for that (the inevitable grumbles follow later). The three numbers they featured were the 0.85C warming since 1880s, 95% confidence that anthropogenic influence had caused most of this, and the 1 trillion tonnes of carbon that would take us to about 2C warming. I think it was originally planned to be three 30 minute programmes, but they ran it all together as one long piece, which seemed to work well to me.
I think they told the stories in an engaging manner, there was also lots of interesting historical stuff about how our understanding of the climate system has developed, which was mostly very well done and would probably have been even more interesting had I not already known it! But of course I was hardly the target audience.
In fact one of the researchers making the program contacted me last year to talk about Bayesian vs frequentist approaches to detection and attribution, specifically the IPCC's statement attributing most of the warming of the last century to anthropogenic effects. Unfortunately I wasn't able to be very encouraging about the idea of explaining the differences between Bayesian vs frequentist approaches to the general public, after all most climate scientists struggle with this question as is demonstrated by the IPCC's misrepresentation of D&A results! I've written on this (really must update my web pages, that link won't last for ever...or will it?) but the argument has little traction even in the climate science community because most people are quite content to continue in their comfortably-erroneous way.
Anyway, the Bayesian thing didn't make it into the transmitted programme, which I was neither surprised nor disappointed about, as I really can't see how to present it in such a way that the general public would get anything out of it. And the traditional misrepresentation of the probability of observations more extreme than observed given the null, as the probability of the null given the observations, was heavily featured (that's basically where the 95% comes from). Sigh. But what I really want to grumble about most strongly was the garbled and nonsensical representation of Kalman filtering in the first section, which, contrary to the claims in the programme, is not a method to check observations against each other and has not been used for temperature data homogenisation. The Kalman filter is actually used for updating a model prediction with new observations, and this is how it was used for space navigation. That is, based on current estimates of velocity and position at time t1, the equations of motion are used to predict the new position and velocity at subsequent time t2, and then imperfect observations of the position at t2 are used to update the estimates of position and velocity, and so on ad infinitum.
Ok, pedants may observe that NCEP has pioneered the use of an ensemble Kalman filter for its 20th century reanalysis project, but this is somewhat tangential to climate change and their results, interesting as they are, have their own homogenisation problems and are are hardly central to the debate on global warming. Ironically, Doug McNeall (who was involved as a scientific consultant, I'm not blaming him for anything in particular though) tweeted a link to the wikipedia page on Kalman filtering, which is a much better resource for anyone interested in learning more about the topic. Anyway, I'm really baffled as to where this bit came from - maybe they just couldn't resist a link to “rocket science” :-) Or did someone think “filter” might be related to filtering out bad data? Well, it isn't.
The “pixel sticks” were very clever, but I don't really think a line graph is improved by drawing it on wobbly axes, expecially if a straight line trend is then drawn through the data! I wonder if Doug will feature that on his Better Figures blog :-) And as for the presenters spending most of their time walking away from the camera...I'm probably sounding like a grumpy old man so I'd better stop. As I said, I think it was pretty good overall, but if you want a mathematical/statistical program that really doesn't make any concessions to dumbing down, and that does cover climate change (and Bayesian statistics) on occasion, I strongly recommend “More or Less” on Radio 4.
Update: Oh, this is interesting. It's a blog post about the programme from the mathematician (Norman Fenton) who presented the 95% section. Turns out he is actually a Bayesian who clearly understands how that number is tarnished by prosecutor's fallacy, and he argues that the scientific debate would be improved by a greater use of Bayesian methods!
Update: Oh, this is interesting. It's a blog post about the programme from the mathematician (Norman Fenton) who presented the 95% section. Turns out he is actually a Bayesian who clearly understands how that number is tarnished by prosecutor's fallacy, and he argues that the scientific debate would be improved by a greater use of Bayesian methods!
35 comments:
I liked the history bits, but I mostly spent my time hoping the presenters were going to walk into lampposts as they never looked where they were strutting. All the going backwards and striking off at right angles, and camerawork into the sunlight was far too obvious to be anything other than distracting. I can imagine it being shown in a few years time as rubbish TV of the past that we used to think was cool. And as for the pixel sticks: please, Sony, put motion correction in those things! Also, why a nubile young woman and two indistinguishably crusty old men presenters. However, the young Kalman looked relatively sexy, as mathematicians go. Wikipedia says he is still alive and 84!
I was sound asleep after an hour, so could it have been too long, just maybe?
I guess the temptation of rocket science was just too big. The formulation about the link between homogenization and the Kalman Filter was also very vague. So I guess they knew themselves it was not rocket science.
For the many errors in the details. The big picture was very accurate. Much better than normal in mass media. But if you make a snow for geeks, the details should also be right.
They also contacted me when they were still searching for a format. I would not have minded if they had asked me again a bit later. Next to Kalman there were a few other small and avoidable things I am not sure about.
"crusty old men"
Hey wait, those guys are my age!!!!
Although I can take solace in at least sharing the hair color of the one in the middle.
How's that kitty, BTW?
Is there an easy way to explain how one would apply the prosecutor's fallacy in this context (to a physicist who's understanding of statistics should be better than it is actually)? I understand it in the normal context (at least I think I do) - if a suspect, for example, matches a blood type that is shared by 5% of the population, you can't claim that that means there's a 95% chance that they're guilty. You need to consider the entire population in order to determine the chance of them being guilty (at least, I think this is right, but I often gets these the wrong way around).
In the case of the 95% attribution study, the hypothesis is that less than 50% of the warming is anthropogenic, and we then find that there is a less than 5% chance that this can explain the data, hence we reject the hypothesis, accepting that more than 50% of the warming is anthropogenic.
What I'm kind of struggling to get is quite how the prosecutor's fallacy would influence this. If we have all the possible data and all the possible models, does it make much of a difference? Is it simply that we can't invert "less than 5% chance that less than 50% of the warming was anthropogenic" to "more than 95% chance that more than 50% of the warming was anthropogenic"?
As the target demographic for the show (broadly), I thought as a whole it did a pretty decent job of presenting the numbers in such a way that the average Joe could walk away from the show and have a little bit more confidence (in his ignorance) that, actually, climate science is making a reasonable fist of things.
The keys here were; focussing on the relationship between observations and models; explaining how (if not quite accurately) you get from data to projection, and showing, time and time again, graphs which all pointed the same way (with a small interlude in the middle where there was a flatline).
The point being that our intake is often peripheral to the facts: These were not climate scientists, they were mathematicians, therefore intrinsically cleverer, less biased and confident in the supremacy of numbers over hypotheses. Score plus for credibility. The graphs almost all pointed upwards: our limited recall picks up the message from the shiny bits and dumps the detail; score plus for background message. And the show, by and large, didn't proselytise, patronise or dumb down: score plus for reasonably bright viewers buying the message.
Next week, I'm looking forward to Wittgenstein presenting 'when weasels attack!' on the Discovery channel...
ATTP, I agree that in some cases the fallacy doesn't matter hugely, in that a more formal Bayesian analysis might well come up with similar numbers, especially when one takes into account the likelihood that scientists will end up settling for a convenient and nice-sounding number like 95%.
But when you think about it carefully it doesn't quite add up, and it certainly doesn't add up when you look at a small subset of the evidence and argue as a result that we have low confidence of an anthropogenic effect...as I argue with respect to ocean warming in the linked paper. Apparently we only have moderate confidence that we have warmed the ocean, even though we have measured it both warming and expanding (similarly to expected from models), and we are very confident that we have warmed the atmosphere directly above it! That's just logically incoherent.
But getting back to the basic fallacy, the statement "there is less than 5% chance that an unforced system would warm by even half as much as observed" is simply not equivalent to "there is a 95% chance that our forcing has caused more than half of the warming". It is the logical equivalent of me rolling a die 20 times, getting a 6 8 times and arguing "there's less than a 5% chance that un unbiased die will have got a 6 more than 6 times, therefore there's a 95% chance that some bias caused at least 2 of the 6s". (think I've got the probabilities right there)
Ie, you are claiming that there is a 95% probability of some significant bias. That's a statement of Bayesian belief, that cannot arise from (correctly interpreted) frequentist statistics).
Thanks for this - and the link to my article. Indeed when I was originally approached about the programme it was about explaining the Bayesian approach generally.
James, thanks, very useful. What you mention about the oceans is interesting. I hadn't picked up on that and, I agree, it doesn't really make sense.
I think I now see the issue and I have probably (and am not alone I think) been expressing this incorrectly. I shall endeavour to be more careful :-)
> Why a nubile young woman and two indistinguishably crusty old men presenters.
If James was available, this is a shame. If Jules was too, there is no excuse.
Nubile may refer to an even younger age, BTW.
***
Why not present bayesianism using your favorite example that it will rain tomorrow, James?
I get the impression that rather than comparing climate model projections with actual measured observations, the real issue for testing anthropgenic effects is to compare models which do take into account the rise in human-generated CO2 with models that do not take into account human-generated CO2
In other words, what are the chances that a model (or ensemble) that ignores man-made effects can do a better job at matching actuals.
I am not sure how a model can "ignore" CO2, though. Would it be by assuming different laws of physics, or assuming the value is constant, or simply by not calculating any effects of CO2 at all?
David, the standard test is to run the model with no increase in atmospheric CO2. This is easily done and the results show no warming at all recently - rather, a small cooling due to small natural influences. The physical laws in the model are not changed, CO2 is provided as an input to the simulation and this is technically straightforward to change.
neverendingaudit, that example of rain doesn't really relate to the detection/attribution problem, which is what the program makers were after. And it's really a rather subtle distinction that arguably doesn't matter a great deal in this instance, cos we all "know" that we are warming the planet anyway, so who cares about the details of the stats...
Maybe when traditional D&A starts to fail widely (as I argued some time ago was inevitable) then people will start to fish around for alternatives.
Isn't the ocean thing just a matter of not having direct measurements for the deep bits?
This winter's continuing bizarre weather pattern over the U.S. may turn out to be that widespread failure.
SE Brazil this August could also do the job.
Um, James, not sure if this question is answered elsewhere on your blog, but from this post I (sort of) get the point about how you think the attribution statements go wrong; but what I don't know is whether there is a simple-ish way of stating what your suggested Bayesian approach concludes on the matter of attribution...
I am not a scientist, but watched the repeat on Thursday night.
I was slightly confused by the climate 'models' which were talked about.
In the show they would show the temperate charts from about 1950 IIRC and they compared this to climate 'models' and what they would have expected the climate to be.
This seemed clear to me. But am I correct in thinking that the 'models' were not created in 1950, but decades later? In the years 2000 and onwards?
I don't quite see how you can make a model for something in say 2005, and then claim it predicted something in 1960. Any help?
John, you are right and this is indeed misleading wording, it's a phrasing that climate scientists have got so used to using that it probably got used without too much thought. It is a somewhat open question how much the models are created to reproduce what we know to have happened, but note also that they are based on sound physical principles too (as I was just discussing here in comments). Note also that the earliest proper models date from 1984 and this one (lower end of post) did generate a pretty good forecast in the true sense.
Steve, I don't know exactly how different a Bayesian approach would be, I haven't seen one recently. Another thing it can account for much more clearly is how good we think the models are, ie the frequentist approach basically assumes that the model simulates the correct pattern of response to each forcing (they do check this a bit, but this check does not form part of the formal calculation). A bayesian approach would allow for a more coherent accounting of how good we think the models are in that sense. But actually, as I argued in this piece, D&A isn't really the right question anyway, we just want to estimate the (past and future) contribution to climate change.
I was appalled the BBC is turning stuff like this out. I suppose I'm a sceptic but prepared to be convinced - this was not the way to do it. A few jolly clever scientists basically saying its all very complicated but in simple terms you might understand - such as football which we know you uneducated ones follow - we have worked it all out and this is what's happening. Any humility to suggest that models may have errors or assumptions; that we don't fully understand the mechanics of climate - the dismissal of sun activity as if it was as certain as what day it was tomorrow was a classic - or that scientists have been wrong before was missing. It came over like some government information film - terrible.
The problem, Pluto, is that they are damned if they do include uncertainty discussions, because they'll be taken out of context by deniers and ordinary people will, unless everything is very carefully phrased, come away from it still thinking that things are not well understood, whereas they are fairly well understood, at least enough to say that action a decade ago would be a good thing.
What I find most worrying Guthrie is you're not being ironic. Those ordinary people are a pest aren't they; best keep things simple for them to make sure they come up with the right answer.
Pluto, of course in a 75 min programme they have to simplify a bit. If you want the longer version, read the (freely available) IPCC WG1 report. If you want the really long version, spend a few years reading the primary scientific literature that it summarises.
I'm guessing you do neither of those and stick to septic blogs for your "information" though.
James, I'm a sceptic yes but I live on the planet too, and as I said I'm prepared to be convinced. I watched the full 75mins to be persuaded; but it was the usual - its all very complicated you're just going to have to believe us - something most people are rightly suspicious of. The sceptics arguments are pretty consistent and well know - if the science is so good why haven't they be swept away? Why doesn't the BBC produce a programme where their evidence is destroyed step by step? (instead of scoffing at them) Why can't we see a Pacific island that's shrunk due to rising sea levels, or the increasing price of skiing holidays because of less snow or declining penguin numbers? I think that this is why the argument is being, if not lost, certainly eroded. Governments were prepared to spend money when there was plenty - because voters liked the idea (when they had money) now there is less money people aren't prepared to spend on a 'crisis' they don't really believe in. If they want to change this the 'warmists' need to stop being so smug and condescending and show why they're right - instead of just telling people they are.
James pointed you to the WG1 report, Pluto.
You do realize how old and how transparent that "science points to the need for solutions I'd rather not spend my money on, therefore the science must be wrong" is? And how when you use it people might think interacting with you is a complete waste of time?
"The sceptics (sic) arguments are pretty consistent and well know (sic) - if the science is so good why haven't they be (sic) swept away?"
Ahahahahahahahaha.
*ahem*
As a general matter the standard denialist arguments you refer to were destroyed years ago. The refutations aren't hard to find. Skeptical Science is the place to start.
James - I'm actually saying "the science shows a lot of estimates and models which tend to play down the error and uncertainty around them- but no smoking gun - until that changes I would prefer to spend money on the global economy lifting people out of poverty and disease" Keep up your sneering (you can spell sceptic with a 'c' or a 'k' - but no doubt there is only one opinion as far as you're concerned) the great unwashed are wandering away - and will progressively force their elected representatives to spend their money on something else.
Yep, nobody cares about the global poors as much as RWNJs.
Pluto, I judge that you aren't a genuine s[ck]eptic. If you were, you wouldn't demand to be spoon-fed. If you wanted to understand the scientific case for AGW, you'd at least read the WG1 report, and maybe even dig into its sources. It's really not hard, if you're willing to make an effort rather than just waiting for the TV show!
But really, your claim to scepticism is fatally undermined by this:
"The sceptics arguments are pretty consistent and well know - if the science is so good why haven't they be swept away?"
Pluto, doesn't it matter to you that the pseudo-sceptics' arguments are wrong, even if they are "pretty consistent" and haven't been "swept away"? Their consistently wrong arguments have been swept away again and again, that just hasn't stopped the pseudo-sceptics from repeating them, which is why they're well known! If you were a genuine sceptic, you'd consider the possibility that you're being fed wrong information for cynical purposes, and ask yourself what those purposes might be. That's how we know you're not one.
Has any such programme ever changed any mind? If not what use they?
(1) Education isn't about changing minds. (2) There are surely many who vaguely accept the consensus view but still appreciate the opportunity to learn something more about the science underpinning it.
Hmmm...surely anybody with an interest have educated themselves already. Actually I'd expect viewing figures to be minimal on the 4th channel of the BBC.
Add the inevitable oversimplifications and I just can't see a target audience who would make use of the broadcast.
I suspect it's just box ticking on the way to Paris. I'm constantly reminded of Father Ted in Speed 3, where faced with an insoluble problem the priests go and celebrate Mass. They're aware it's useless but do it anyway because that's the only thing they can think of.
Likewise all sorts of people trying to do something wrt Paris will put a lot of effort into useless actions they're very professional about. In this case it's Bbc producers. Same is happening at the Guardian where they'll write a tons of articles which won't move the debate, simply because writing articles is what the Guardian is about.
Another comic situation along the same lines is when Calvin gets stuck in the toilet area of the school during the food-themed play dressed like an onion and when the time of his line comes, he delivers it perfectly without an audience.
Well I agree it will have a small audience - but that's an argument against BBC4 not the programme in particular. As I said, I expect there are many who generally accept the consensus view without having had sufficient interet to investigate in detail. I think they will have appreciated the programme.
It is hardly surprising that sceptics didn't like the content, or even the fact that the programme was broadcast.
For the record I have not watched it, never will and have read little about it. As for the content you weren't content yourself so it's hardly a matter of skepticism.
Anyway we've had James Cameron last year, Naomi Oreskes in 2015, the bbc doing its bit, and not a possible inch of progress out of any of that.
It's like an introduction to Chinese all in Chinese. Those who can follow don't need it. Time filling, nothing else.
I think you're not far off the pace Maurizio; with some innovative analogies. Rightly or wrongly I think the 'warmist' case is passed its zenith (which was probably inconvenient truth) - in part I think for the reasons you give. I wrote on here interested to know what is the argument to the person sat at home who is a sceptic or should I say agnostic. All there is just the usual "its in the IPCC" - with its graphs, adjustments, averages and models; and insults of course. You could never sell anything to anyone with such an argument be they a man in Starbucks or the CEO of an international company. Which is probably why nobody apart from governments spending someone else's money are doing anything - and then not much more then attending conferences and signing up to flexible targets. Look out the window its still snowing, that pacific island the marines fought for 70 years ago is still there unchanged, polar bears aren't dying, some glaciers are expanding, wasn't there such a thing as a medieval warming period? I'm not saying these things therefore prove that GW isn't happening; but if there is no answer to them other then look at this 100 page report (you idiot) which lots of scientists agree with - the argument (rightly or wrongly) will continue its graceful decline.
Following their extirpation from California a century or more ago, Steller's sea lions have now started to appear again. Those would be worth seeing.
Hi there does anyone know if this programme is available to view anywhere? @ethicalteam
Try YouTube.
Post a Comment