Comments on James' Empty Blog: Once more unto the breach dear friends, once more....

With regards to the OP title:http://www.nearingzer...

2008-06-09T22:14:00.000+09:00

With regards to the OP title:

http://www.nearingzero.net/screen_res/nz089.jpg

Yoram;"It is a procedure which, if practiced consi...

2008-06-07T06:57:00.000+09:00

Yoram;

"It is a procedure which, if practiced consistently, guarantees a low rate of mistakes (i.e., of ruling out the true value of an unknown parameter)."

That makes sense. If a researcher generated 95% confidence intervals throughout their work, they expect to have excluded true values ~5% of the time.

> what is a sane way to interpret confidence inter...

2008-06-06T14:23:00.000+09:00

> what is a sane way to interpret confidence intervals?

It is a procedure which, if practiced consistently, guarantees a low rate of mistakes (i.e., of ruling out the true value of an unknown parameter).

The important thing to note here is that the CI is the procedure, rather than the result of the procedure in a specific case. About the result in a specific case nothing can be said.

A CI is therefore a commitment by the practitioner to do things in a certain way - a commitment that is undertaken before the data is observed.

Crandles, if priors are personal beliefs, then con...

2008-06-04T07:02:00.000+09:00

Crandles, if priors are personal beliefs, then constructing a prior by a Delphi process might be a good way to depersonalize it. Otherwise why is your choice better than mine?

James, I do exaggerate a bit, but the point about where to draw the line remains.

Thank you.

2008-06-04T04:07:00.000+09:00

Thank you.

Blogged here. It is obviously wrong, and happens t...

2008-06-03T10:27:00.000+09:00

Blogged here. It is obviously wrong, and happens to have been published in the same special issue (guest editor: P Chylek) that spawned the Schwartz nonsense. I'm not aware of any attempt at a peer-reviewed comment on it, but it will certainly not influence the field.

What is your take on Limits on climate sensitivity...

2008-06-03T10:15:00.000+09:00

What is your take on

Limits on climate sensitivity derived from recent satellite and surface observations

which states "We find that the climate sensitivity is reduced by at least a factor of 2 when direct and indirect effects of decreasing aerosols are included, compared to the case where the radiative forcing is ascribed only to increases in atmospheric concentrations of carbon dioxide."

I am assuming I don't read climatologese all that well...

Eli>"Given that, is there only an arbitrary way of...

2008-06-03T01:01:00.000+09:00

Eli
>"Given that, is there only an arbitrary way of separating the two (1979, 1980, 1981, etc??).

If so prior construction becomes an art rather than an algorithm"

Priors are meant to be personal beliefs that differ from person to person. So even if two people agreed to use expert opinion to 1979 their priors would be different.

You are welcome to have a go at it using a different split.

Anyway yes it is more of an art of extracting your own beliefs than an algorithm.

Does this mean that you 'can easily argue that assigning uniform probability from 0 to 100 C is ok'?

Simple answer - No.
If I said that I believed there was a 99% chance that aliens would come and steal the moon tomorrow would this make that a reasonable expectation? I think you would dismiss me as mad much more readily than you would accept that as a reasonable belief.

Where precisely to draw the line between reasonable beliefs and unreasonable ones may not be very clear. However if there is enough of a gap then the precise position does not need to be determined. I doubt you would disagree with me saying any belief that the odds are greater than 25% is an unreasonable belief (other than to suggest a lower percentage could be substituted).

Uniform probability from 0 to 100 C
implies a 50% probability of sensitivity over 50C. That seems crazy to me. Could any intelligent and well informed to 1979 person reasonably believe that the chance of sensitivity being over 50C is greater (let alone 5 times greater) than the chance of sensitivity being between 0 and 10C?

"what is a sane way to interpret confidence interv...

2008-06-02T15:17:00.000+09:00

"what is a sane way to interpret confidence intervals?"

I don't know. Well, I know the technically correct interpretation (eg an interval generated according to a random process such that p% of intervals so generated will contain the true parameter value), but I also know that almost everyone misinterprets them as probability intervals in practice. There is a huge discussion on the Wikipedia page about this (here on down). The technically correct interpretation seems rather useless in practice...and the term "confidence" seems to sometimes be used as a con-trick in full knowledge that the unwary will indeed interpret it incorrectly as "probability". So in general I am not a fan of them. although they do have the advantage of simplicity.

Eli,

If you are going to persist in claiming that a uniform prior represents "ignorance", you are going to have to address the question of how one can be ignorant about x but knowledgeable about 1/x, or vice-versa. That doesn't correspond to any plausible definition of "ignorant" IMO (in either technical or common usage).

Crandles, my point is that the amount of informati...

2008-06-02T01:05:00.000+09:00

Crandles, my point is that the amount of information in the prior effects the result. Given that, is there only an arbitrary way of separating the two (1979, 1980, 1981, etc??).

If so prior construction becomes an art rather than an algorithm and one can easily argue that assigning uniform probability from 0 to 100 C is ok, even if the oceans boil (priors are supposed to be ignorant!). My preference would be to separate models and observational information, using the former to build the prior.

James...beginners question...what is a sane way to...

2008-06-01T22:42:00.000+09:00

James...
beginners question...
what is a sane way to interpret confidence intervals?

Eg. Statistical Analysis, Kachigan p. 141;

"In other words, we are 95% sure that the true mean weight of the ball-bearings, had we measured every single one of the day's production, would be somewhere between 149.21 and 150.39 grams. More technically, we are 95% sure that the procedure for creating the obtained confidence interval would produce an interval encompassing the population mean. For the actual interval produced, the true mean either is or is not in it, so strictly speaking we cannot say there is a .95 probability that the mean falls in the interval. This is a moot point from a practical standpoint, but has importance in a theoretical context in which no quantitative probability-type statement is allowable for a specific interval.We will adhere to the more practical view that "95% sure" or "95% confident" are meaningful common sense statements with respect to specific intervals, and are as legitimate as statements which invoke "odds" or phrases such as "substantial assurance" to circumvent the probability issue."

... in other words, is "confidence" used as a hand-waving way of saying "probability", because saying "probability" would be wrong?
... if it is likely (but unstated) that a person intended to convey a probability when they wrote a confidence interval... should the interval be interpreted as a confused frequentist statement, or as bayesian with uniform prior (regardless of what one thinks the individual "intended").
... if the frequentist frame does not give a location for the population parameter, how can a frequentist frame be useful when interpreting confidence intervals?

Martin is going in the right direction. Consider ...

2008-06-01T07:21:00.000+09:00

Martin is going in the right direction. Consider just the 'endings' in Jared Diamond's Collapse, not to mention the (other) ones in books by historians, pre-historians and archaeologists.

It is rather akin to an avalanche. but alas, without the physical laws governing avalanche behavior.

Indeed just consider the current world food crisis brought about (mostly) by bad policies.

James: double counting, by implicitly using observ...

2008-05-31T19:32:00.000+09:00

James: double counting, by implicitly using observational info anyway? Or by multiple models sharing physics, or even code, in sub-components?

James and Steven, CO2 doubling on its own very unlikely, I agree. But combined with peak oil and other ecological stresses for a 10B population?

Martin,That's effectively what the original NAS re...

2008-05-31T18:39:00.000+09:00

Martin,

That's effectively what the original NAS report (Charney) did with the two main models around at the time (Hansen and GFDL). So it's certainly not an unreasonable idea! One still needs to think about what sort of tail to add outside the range of models, though, as it would not (IMO) be reasonable to claim a priori that S cannot like outside that range. And using modern models brings up the question of double counting, which the 1979 NAS report avoids much more straightforwardly.

Steven, there is no reasonable scenario I can think of in which 2xCO2 causes the end of civilisation (unless society is so brittle that any minor disruption triggers a nuclear holocaust, in which case the next flu pandemic or peak oil will do the job first). In general though, your question points towards issues of nonlinear utility - this is a standard economic issue but one that we preferred not to touch on in this paper.

One thought that has crossed my mind: would the di...

2008-05-31T12:27:00.000+09:00

One thought that has crossed my mind: would the distribution of sensitivities from the 22 or so models included in the AR4 report constitute a valid prior? These models are supposed to be based on the physics only, untuned to observations.

James: great paper. I have one worry, though. If t...

2008-05-30T23:56:00.000+09:00

James: great paper. I have one worry, though. If there's a chance of a 100% world GDP loss (which would represent human extinction or the collapse of civilization or whatever), then from a utilitarian point of view it won't do to take the expected value, as it seems like the effects of a 100% loss would be permanent in a way that other losses probably wouldn't be.

So my question to you is -- do you have any idea how to estimate the probability of a doubling of CO2 causing permanent collapse/extinction? I'd put it at significantly less than 1% but I'm not sure how much less exactly.

Martin & James Annan --- Thank you.

2008-05-30T04:25:00.000+09:00

Martin & James Annan --- Thank you.

Well, it seems that Chris has answered most points...

2008-05-29T21:46:00.000+09:00

Well, it seems that Chris has answered most points pretty well - thanks!

Yoram, it is not in dispute that one can generate an arbitrarily large range of possible posteriors by selecting extreme enough priors. The question is what sort of priors are reasonable, and no-one, not even the most extreme advocate of uniform priors, has every suggested that 370C is a sensible value. But even in that case (lognormal in L) the posterior does not actually appear to be that unreasonable. This seems to me like a strong demonstration of robustness, not a show of arbitrariness and methodological weakness.

I don't know what you found to disagree with me about confidence intervals - the range you present in your later comment is precisely the 95% probability interval that Forster and Gregory presented (for this, they explicitly assumed a uniform prior in L, noting that this choice was unconventional) and your point about wider intervals extending to L=0 is precisely what I meant with my statement "However, some might go to infinity "and beyond", especially at the higher confidence levels." Of course a confidence interval will just be routinely misinterpreted as a probability interval anyway...

David, there could br a case for using Arrhenius' 6C, but note that this was based on a calculation that is known to have substantial inaccuracies (and does not need doubling and redoubling). The canonical 3C can be viewed as more careful and credible version of the same calculation. But there is also the issue Chris emphasises of what information is contained in the prior versus likelihood - anyone arguing for a really wide and high prior based on ancient history will have to also consider how to take account of all that we have learnt in the meantime, not just the last couple of decades of some satellite observations that I used.

Yoram,Ah yes, I misread your formula. What you do ...

2008-05-29T20:46:00.000+09:00

Yoram,

Ah yes, I misread your formula. What you do is precisely right.

Still, the centre of the normal distribution at 2.3 maps to 0.43, well away from the middle of the CI for S. For what it's worth.

Martin,For confidence intervals you don't need to ...

2008-05-29T13:59:00.000+09:00

Martin,

For confidence intervals you don't need to make assumptions about the distributions of the unknown parameters: they don't have any.

I used the FG measurement of L (with its associated Normal uncertainty) as it appears in the A&H paper to generate a CI for L. One convenient property of CIs is that they allow easy transformation of the parameters.

The CI for S = 1/L is derived as:

CI_S = { 1 / l : l \in CI_L }.

Yoram, yes, if you assume that S is gaussian. More...

2008-05-29T13:15:00.000+09:00

Yoram, yes, if you assume that S is gaussian. More sensible IMHO is to assume L gaussian, and work from there (but then you have to first convert those numbers which are apparently stated for S :-( )

It will become asymmetric but that would be realistic.

It seems hard to avoid the conclusion that using a...

2008-05-29T13:05:00.000+09:00

It seems hard to avoid the conclusion that using a confidence interval is much more sensible than arguing about what counts as prior information.

A 95% CI for S is
S \in 1 / (2.3 +- 0.7 * 1.96) = [0.27, 1.08].

David, I seem to remember that Arrhenius did inclu...

2008-05-29T12:05:00.000+09:00

David, I seem to remember that Arrhenius did include water vapour in his 6C estimate (and by the way found also the polar amplification in his original paper; fantastic).

As for the oceans, they don't affect sensitivity only the time scale on which it becomes visible (as it is defined as equilibrium sensitivity).

crandles is right that if the prior is based on state of knowledge 1979, then it must rule out runaway (or even very large S).

>"Well yes, I would agree. For a prior this seems ...

2008-05-29T06:19:00.000+09:00

>"Well yes, I would agree. For a prior this seems reasonable: I know one very Earth-like planet that has suffered runaway feedback. Actually two, counting Snowball Earth. Where do you philosophically draw the line between prior and observed?
"

James has made clear in the paper that he has used a prior from expert opinion as at 1979 and updated with data entirely post 1979.

Now you might want to argue that to arrive at that prior you need a more ignorant prior and update it with knowledge to 1979. But this really isn't necessary - James is arguing that the precise details of his prior doesn't matter too much as long as something plausible is used. So the need to go back to an ignorant prior and expert knowledge to 1979 isn't necessary.

A sensitivity of over 90C would presumably lead to oceans boiling. I suspect a sensitivity of 40C would be enough to restrict life to only near the poles several times in Earth's history and we would have known about that happening by 1979. This should be ruled out at the 2.5% level.

I don't see much wrong with a median of 3.7C but a 2.5% chance of S greater than 370C is crazy unless you are trying to get back to a prior with only knowledge to 1850 or earlier or something (James has indicated the split between data used to update and all other knowledge doesn't need to be timewise.)

What is more important to ask is how well scientists in 1979 could have predicted the warming rate over the next 30 years for the greenhouse gas levels that have actually existed. If they could have done very well, would this imply some double counting of the (predicted/actual) data to limit the remaining uncertainty?

Eli, why ask about 'minimum amount of information' when it is clear that all information must be used either in the prior or in the data used to update else you end up with something that isn't a credible probability distribution?

Drawing the line between prior and observed: Good...

2008-05-29T06:17:00.000+09:00

Drawing the line between prior and observed: Good question.

The Bayesian literature makes a fairly big thing out of "background information". So I think we are entitled to use ancient physics; Tyndall's measurements, Arrhenius' calculation, Stefan's Law; thermo if needed.

So here is a guess at an uninformed prior. Arrhenius said climate sensitivity to just a doubling of CO2 is 6 K (hope I have that right). From Tyndall, water vapor doubles that (hope I have that right). But then warming the oceans doubles that again (hope I have that right). So S = 24 K, around which I'll most pessimistically draw a Cauchy distribution, being quite uncertain about these values.

Fire away!