Monday, October 26, 2009

Reply to comment

Our reply to the comment I mentioned recently is now up on CPD, so go and have a look if you're bored enough.

I suppose I ought to point out the discussion to a few relevant researchers who might have some interesting points to make.

8 comments:

crandles said...

Bottom of page C869 (or 4 of 10)

"This should therefore provide a useful update to existing
studies, which suggests that the main result of AH06 could now more reasonably be
considered too narrow rather than too broad (eg see the discussion in AH09)."

Is that backwards with you meaning

'AH06 could now more reasonably be
considered too broad rather than the too narrow that the comment asserts (eg see the discussion in AH09).

William M. Connolley said...

Now I see how this will work out. Frame et al. will publish a "corrected" procedure which will be essentially the same as yours but will fix the Finnish comments. From that point on everyone can reference them not you :-)

Hank Roberts said...

The very last bit of your Comment seems to be missing a bit:

"Fig. 2. Posterior pdfs arising from different calculations. LGM (red) and 20C (blue) indicate
results using single constraints, AH06 method (mauve) combines these constraints using a
univariate independent ap

C875"

James Annan said...

Chris, Thanks, I'll give you a little more time to spot any more errors and then post a correction :-)

Belette, the reality is better than you imagined it....I hope all will be revealed eventually.

Hank, yes I noticed that it didn't like the figure captions (truncated, and it didn't process the LaTeX), so I put them in the main text too.

crandles said...

>"The implication of this approach is that all knowledge of the climate system is thrown
away at the outset, other than that arising from the specific limited set of observations
which formed the focus of the study. In the previous literature, this approach was
invariably taken without any discussion or justification. The Comment does not discuss
how reasonable this approach is, nor why it should be considered more credible than
the approach of AH06
In contrast to the procedure of simply throwing away large quantities of data, the approximation
of independence (conditional on x), made in AH06, may result in a bias
in the result towards either too narrow or too broad an outcome."

OK you have put in some effort to show:
"The conclusion to
be drawn here is that the approximation of AH06 made may either underestimate or
overestimate the uncertainty compared to the full calculation.that the approximation of AH06 made may either underestimate or
overestimate the uncertainty compared to the full calculation."

It was an overestimate of the uncertainty in the example.

It is obvious that the combination of those seems two is good enough. But strictly that conclusion on its own might not be good enough if it is possible that it could vastly underestimate the uncertainty.

If they want to comment then the onus should be on them to show that they believe it underestimates by more than the amount to make it worse than throwing out all the other data.

However, I am just wondering if it is possible to get a feel for the expected relative sizes of the change attributable to the data ruling out sensitivities compared to the potential for change via nuisance parameters and model error.

If the technique is a poor one then would it be possible for the uncertainty introduced via parameters and model error could swamp the effect of data ruling out sensitivities? Is there any way to know if this could be happening?

James Annan said...

Chris,

I suspect that the independence assumption is a little optimistic (which is why we added the discussion about extra observations), but it's hard to be sure. I am (genuinely!) more interested in others having a go than refining our own calculations, partly as "a man convinced against his will, is of the same opinion still" and partly because a wider range of views is intrinsically more robust.

However, AH09 does make it all rather moot, I think, because without the uniform prior the long tail doesn't really arise in the first place. Alternative calculations would still be welcome here though...

crandles said...

4 days left with no further posts -disappointing I guess.

"the reality is better than you imagined it....I hope all will be revealed eventually"

Intriguing. Now what would be 'better' or is that 'more unfair or airbrushed out'?

Perhaps certain people will write: 'Although a Baysian approach has been suggested, this was criticised by Henriksson et el. so we prefer the approach of ....'

Then everyone copies that without bothering to go off and look at the Annan & Hargreaves vs Henriksson et el. argument.

James Annan said...

Chris,

The original author just asked for an extension (I've not heard if it was granted, but expect it will be).

(BTW I switched moderation on to all old posts as I've been getting a lot of spam on them recently.)