Subjectivity, objectivity and belief in probabilistic estimation

As I've discussed before (eg here and here) there is a certain amount of confusion over what subjective and objective really mean, in the context of Bayesian estimation (such as estimating climate sensitivity S).

There is a good discussion of these issues in Jonty Rougier's forthcoming paper, which I could do worse than to simply quote from at length (it's in Section 2 of his paper):

The first thing to understand about probability in this context is that there is no such thing as ‘the’ probability. To ask for ‘the’ probability is to commit a category error. A probability is a numerical summary of a person’s state of knowledge about a proposition: it is inherently subjective (i.e., it relates to the mind of a subject). Therefore probability takes the possessive article, not the definite one: better to say your probability.

Some readers will be concerned about this characterisation of probability. There appears to be a syllogism that runs “Science is objective, your type of probability is subjective, objective and subjective are antonyms, therefore your type of probability has no place in science.” It comes up in discussions with scientists often enough to warrant a brief comment. The error is to confuse two meanings of ‘subjective’. ‘Objective’ in this context may be taken to mean disinterested, or uninfluenced by personal prejudice: obviously a hallmark of good science. There is a meaning of ‘subjective’ which is antonymous to this: emanating from a person’s emotions or prejudices. But in dictionaries this is the second meaning. The first meaning of ‘subjective’ is relating to the mind of the subject, and this is the appropriate sense when probability is used to describe uncertainty: uncertainty is a property of the mind.

A scientist’s prediction will be perforce subjective, but he should aim to be objective as well, by making a disinterested appraisal of the probabilities he attaches to events—this is not paradoxical. Objectivity is not always easy to achieve. For example, if a climate scientist thought too little attention was being given to a certain type of future climate catastrophe, he might be tempted to overstate his probability of the event, in order to attract attention. For the policymaker, though, it is not just what a scientist thinks that is important, but also the extent to which that scientist can justify his assessment. Even though probabilities are subjective statements, not all such statements are demonstrably valid, and of those that are, not all are authoritative. Valid statements are those that are consistent with the probability calculus, the axioms of which were clarified by Kolmogorov in the 1930s.

However, scientists routinely talk about the probability, eg "the probability of S>6". I'm sure I've done it myself. It is a much more natural way for a scientist to write than to talk about "my probability", and I suspect that many of them would be squeamish to talk openly about their own beliefs, partly for the reasons Jonty elaborates above. OTOH, Bayesian probability only exists as a belief. One obvious way of squaring this circle is to interpret the scientist's statement as meaning "for anyone who accepts the stated (and unstated) premises underlying this calculation, their personal probability would be ...". Or, in more formal terms, A implies B. Indeed most papers can be reduced to this form, where A is a set of assumptions, observations and approximations, and B are the conclusions.

While this might clarify what scientists mean by "the probability", it merely shifts the real question one stage further back: does the author of the paper actually accept these premises (A) and therefore the conclusions (B) themselves? I think it is reasonable to expect and assume that they do, unless there are clear statements to the contrary in the paper. For example, we tried to make it clear that we do not believe P(S>4.5)=5%, for example, but instead consider this probability to be an overestimate (given a high/low bet at 19:1 odds, I'd choose the low side). OTOH, the authors of various other probabilistic estimates have sometimes hidden behind circumlocutions that make it hard to know to what extent they really believe their premises (and therefore conclusions). (I don't really want to pick on anyone in particular here, I think we've all done it to a certain extent.) I'd like to see greater clarity on this point in the future. By all means write a paper showing that A implies B, but if you don't actually believe either A or B, you ought to say so!

The main purpose of this comment is to lay the groundwork for a subsequent one I'm planning...