Friday, December 11, 2015

The benefits of an informed prior

Since Bayesian priors seem to have come up on a couple of blogs recently...

A few weeks ago I got a rather spammy email about a “climate change challenge” from someone I'd not come across before. Looking at his “publication list”, that is perhaps not so surprising (the most recent ACP submission listed there was rejected without review, so cannot be found on their web site). Anyway, it's just a typical kooky site full of claims about how everyone else is wrong apart from the author who cannot get his groundbreaking theories published, move along nothing to see etc...

Shortly afterwards, Doug Keenan announced a “contest” wherein all and sundry were challenged to identify from a large set of random time series which of them had been generated by a trendless random process, and which were generated by one with a known trend. Of course the trivial trick underlying his game is to make the trend small relative to the inherent variability of the random time series. For a simple example, consider if I generate a set A consisting of 500 samples from N(0,1) and a set B of 500 samples from N(1,1). If I publish the 1000 values, no-one could possibly hope to identify correctly which were from A and which were from B, because any value like 0.3 or 0.8 could easily have come from either set. If I'd used N(0,0.1) and N(1,0.1) for the two sets, on the other hand, it would have been rather different...

Amusingly, “I'm a genius time series analyst” Keenan bungled his calculations, as is documented in the comments here. His original set of trendless and trended series were sufficiently well separated that a successful partitioning might have been possible, at least with a bit of luck. Of course, his “$100,000” prize was “based on the honor system” so any entrant's chance of collecting would have to factor in their opinion of how honourable he is. I suggest that his juxtaposition of “My name would be mud if I reneged”, with the fact that he actually has reneged on his challenge by withdrawing the original set of samples and replacing it with another, might be usefully considered as evidence on the matter.

As for the Bayesian prior, that's what got me to the same answer as Andrew Gelman without the need to do all the calculations...


6 comments:

Steve Bloom said...

Reminds of the Monckton "millennium puzzle" episode, although Monckers didn't have a way to back out and so apparently had to sell his ancestral estate. Galileos, the lot of them.

KenH said...

N(1.0.1)?

James Annan said...

Thanks, silly typo corrected (to N(1,0.1))

Nick Barnes said...

Monckton didn't actually have to sell the estate; that was just nonsense for the press. [I happen to know one of the prize-winners.]

Steve Bloom said...

Interesting, Nick. Do you know how he came out of it financially?

JohnMashey said...

Regarding http://whyclimatechanges.com/about/climate-publications/, that was the same guy who had a booth at AGU, with banner, glossy handouts, and his book. He's a retired USGS geologist who did a lot of work on volcanoes.
See What Really Causes Global Warming?: Greenhouse Gases or Ozone Depletion?
and use Look Inside to quickly peruse the Table of Contents.