The risks of financial managers

The following question is a slightly reworded version of a real question in a real financial management company's risk questionnaire that was provided to someone locally. I've tried to be fair to the financial company while making their question a bit less vague, they actually had two similar questions which cover this issue in slightly different ways.

"You have the choice of placing your investment in one the following 5 portfolios, ranging from low to high risk. For each portfolio, you can assume the return over each consecutive year (edit: was 5 years) takes one of two possible values, with 50% probability of each outcome. Which portfolio would you prefer for your investment?

A: 50% chance of either +11% or -7%

B: 50% chance of either +17% or -10%

C: 50% chance of either +25% or -16%

D: 50% chance of either +37% or -27%

E: 50% chance of either +58% or -48%"

So, which option(s) do you like, and why?

BlueSkiesResearch.org.uk: How confident should you have been about confidence intervals?

OK, it’s answer time for these questions (also here on this blog). First, a little background. This is the paper, or rather, here it is to download. The questions were asked of over 100 psychology researchers and 400 students and virtually none of them got all the answers right, with more wrong than right answers overall.

The questions were modelled on a paper by Gigerenzer who had done a similar investigation into the misinterpretation of p-values arising in null hypothesis significance testing. Confidence intervals are often recommended as an improvement over p-values, but as this research shows, they are just as prone to misinterpretation.

Some of my commenters argued that one or two of the questions were a a bit unclear or otherwise unsatisfactory, but the instructions were quite clear and the point was not whether one might think the statement probably right, but whether it could be deduced as correct from the stated experimental result. I do have my own doubts about statement 5, as I suspect that some scientists would assert that “We can be 95% confident” is exactly synonymous with “I have a 95% confidence interval”. That’s a confidence trick, of course, but that’s what confidence intervals are anyway. No untrained member of the public could ever guess what a confidence interval is.

Anyway, the answer, for those who have not yet guessed, is that all of the statements were false, broadly speaking because they were making probabilistic statements about the parameter of interest, which simply cannot be deduced from a frequentist confidence interval. Under repetition of an experiment, 95% of confidence intervals will contain the parameter of interest (assuming they are correctly constructed and all auxiliary hypotheses are true) but that doesn’t mean that, ONCE YOU HAVE CREATED A SPECIFIC INTERVAL, the parameter has a 95% probability of lying in that specific range.

In reading around the topic, I found one paper which had an example which is similar to my own favourite. We can generate valid confidence intervals for an unknown parameter with the following procedure: with probability 0.95, say “the whole number line”, otherwise say “the empty set”. If you repeat this many times, the long-run coverage frequency tends to 0.95, as 95% of the intervals do include the true parameter value. However, for a given example, we can state with absolute certainty whether the parameter is either in or outside the interval, so we will never be able to say, once we have generated an interval, that there is 95% probability that the parameter lies inside that interval.

(Someone is now going to raise the issue of Schrödinger’s interval, where the interval is calculated automatically, and sealed in a box. Yes, in this situation we can place 95% probability on that specific interval containing the parameter, but it’s not the situation we usually have where someone has published a confidence interval, and it’s not the situation in the quiz).

And how about my readers? These questions were asked on both blogs (here and here) and also on twitter, gleaning a handful of replies in all places. Votes here and on twitter were majority wrong (and no-one got them all right), interestingly all three of the commenters on the Empty Blog were basically correct though two of them gave slightly ambiguous replies but I think their intent was right. Maybe helps that I’ve been going on about this for years there

Bentham marathon

I've always intended to limit myself to one marathon a year, as I reckon that's quite enough time and effort to be devoted to serious running. But a couple of years ago I did the 3 peaks race when it was scheduled 4 weeks after Manchester marathon and that worked out ok, so when local road club Bentham Beagles announced they were putting on a marathon 6 weeks after Manchester, it felt like it would be a bit rude to not turn up.

The event was being arranged by a couple there who wanted to mark their 100th marathon with a local event (yes, there really are people who do 100 marathons as some sort of hobby/challenge). The route promised to be extremely hilly, heading first due south over the fells and down to a section of footpath around Stocks Reservoir, before returning over Bowland Knotts - both main climbs reaching altitudes of well over 400m, with plenty of smaller bumps to negotiate and numerous "arrowed" sections where the gradient exceeds 14% and in one place 20%. In fact the total climb of around 1000m comfortably exceeds that of the famous Snowdonia marathon (which goes round Snowdon, not up it). Not quite what Manchester training had prepared me for but I did manage a couple of runs over parts of the course in preparation so had some idea what I was letting myself in for. 

Post-Manchester resting was going ok and the blisters had healed, but then the three weeks leading up to the Bentham race were spent travelling first to Stockholm and then London, finally returning home around 10pm Friday night with the race starting at 9am on Saturday. Not quite ideal pre-race preparation, but never mind. I was determined to make it a fun run rather than going flat out, as it was never going to be a fast time. The on-line registration system provided a list of entries, and with a limit of 100 runners it wasn't too hard to do some stalking and work out that there quite probably wasn't going to be anyone properly fast (which means faster than me, of course). 

I guessed that a few of them might well set off a bit ambitiously at the start, however. Therefore a plan was hatched to try to go as easily as possible while keeping in touch with the leaders for the first half, before potentially pushing on a bit harder in the second half. Sure enough, a few people did charge off ahead but not ridiculously so, stretching out to a lead of up to a minute as we started up the first main climb. I think I was about 6th at one point, but soon enough a couple of the early leaders started to fall back and shortly afterwards I found myself running alongside one other guy who seemed quite experienced (he informed me that he had been pretty good in decades past with a ~28 min 10k to his name). We slowly reeled in the leaders, eventually forming a group of three for the last steep descent to the half-way mark.

The long-time leader stopped for a break at the reservoir (turned out to have foot problems) but Mr 28min was still going well and I didn't want to leave it to the last 10k in case he still had a turn of speed! (For context, my 10k PB is outside 37 mins, anything under 29 mins would be one of the leading times nationally.) So as we hit the second big hill I put a bit of effort in and was pleased to find he didn't respond. 3 miles later I had a quick snack at the top of Bowland Knotts and he was nowhere to be seen. I had time for a couple of pics on the way down...

jules had cycled out to meet me around the 20 mile mark and told me I had a decent gap so I just kept going at a sensible pace hoping my legs wouldn't fall off. I haven't done such a long or hilly run since the three peaks in 2017 so wasn't sure how the last few miles would go. Turned out just about ok and I finished in a time of 3:24, almost 9 mins clear of second.

The allocation of number was purely alphabetical!

Pic: Andrew Swales

Almost always road races are really just a time trial for me with the aim being to go as fast as possible over the distance so it was fun to be able to actually "race" for once without worrying about the time. And the event was very well organised with plenty of well-stocked refreshment stops. Obviously running 99 marathons previously meant the organisers knew what needed to be done!

Here's the strava log:

BlueSkiesResearch.org.uk: How confident are you about confidence intervals?

Found a fun little quiz somewhere, which I thought some of my readers might like to take. My aim is not to embarrass people who may get some answers wrong – in testing, the vast majority of all respondents (including researchers who reported substantial  experience) were found to make mistakes. My hypothesis is that my readers are rather more intelligent than average  Please answer in comments but work out your answers before reading what others have said, so as not to be unduly influenced by them.

I will summarise and explain the quiz when enough have answered…

---

A researcher undertakes an experiment and reports “the 95% confidence interval for the mean ranges from 0.1 to 0.4”

Please mark each of the statements below as “true” or “false”. False means that the statement does not follow logically from the quoted result. Also note that all, several, or none of the statements may be correct:

1. The probability that the true mean is greater than 0 is at least 95%.

2. The probability that the true mean equals 0 is smaller than 5%.

3. The “null hypothesis” that the true mean equals 0 is likely to be incorrect.

4. There is a 95% probability that the true mean lies between 0.1 and 0.4.

5. We can be 95% confident that the true mean lies between 0.1 and 0.4.

6. If we were to repeat the experiment over and over, then 95% of the time the true mean falls between 0.1 and 0.4.

BlueSkiesResearch.org.uk: Stockholm

Just had a couple of weeks in Stockholm, courtesy of Thorsten Mauritsen at MISU. who we had previously visited in Hamburg. Lots of science will be forthcoming but we are too busy doing it to write about it for now

For the moment, I will just note that Thorsten is Danish, previously working in Germany but now in Sweden, we had discussions with his British and French group members, the Head of Department is Spanish. Discussions in the canteen seemed to be mostly English in a variety of accents (including the Dutch student who had considered coming to the UK but who had been dissuaded by the obvious reason), mixed with a range of unidentifiable Scandinavian languages – presumably mostly, if not all, Swedish. Theresa May and Jeremy Corbyn would be horrified to hear of such an outrageous situation and I’m relieved that they are doing their level best to ensure that no Brits will risk encountering such a terrible situation again.

(Actually, to be honest I am relieved that their level best is so pitiful that we aren’t actually going to leave the EU. But I’m still disgusted that they are so scared at the thought of people living, working and studying in different countries that they are completely fixated on the idea of preventing us from doing so.)

Haga parkrun was close to our hotel, and by strange quirk of fate on Sunday morning I ran a route which quite closely approximates a lap of the upcoming Stockholm marathon. I hadn't even known there was a marathon. Some were out practising for the famous Stockholm ski marathon too.

Stockholm has a lot of islands, and as a result, there’s a lot of coastline and water.

On our last night we had dinner in a Michelin-starred restaurant which was an interesting experience. However this photo below is just the little castle on the top of Kastellholmen which may be used as some sort of conference centre I think.

Oyster Roulette

 If you've ever lost at Oyster Roulette you might not be inclined to try it here. Or are they just being precautionary/truthful?

But, how nice that a full norovirus recovery pack it is included among the condiments... 

Salt, sugar, handwipes.

BlueSkiesResearch.org.uk: Steffen nonsense

Been pondering whether it was worth bother blogging this but I haven’t written for a while and in the end I decided the title was too good a pun to pass on (I never claimed to have high standards).

The paper "Trajectories of the Earth System in the Anthropocene" had entirely passed me by when it came out, though it did seem to attract a bit of press coverage eg with the BBC saying

Researchers believe we could soon cross a threshold leading to boiling hot temperatures and towering seas in the centuries to come.
Even if countries succeed in meeting their CO2 targets, we could still lurch on to this “irreversible pathway”.
Their study shows it could happen if global temperatures rise by 2C.
An international team of climate researchers, writing in the journal, Proceedings of the National Academy of Sciences, says the warming expected in the next few decades could turn some of the Earth’s natural forces – that currently protect us – into our enemies.

and continues in a similar vein quoting an author

“What we are saying is that when we reach 2 degrees of warming, we may be at a point where we hand over the control mechanism to Planet Earth herself,” co-author Prof Johan Rockström, from the Stockholm Resilience Centre, told BBC News.
“We are the ones in control right now, but once we go past 2 degrees, we see that the Earth system tips over from being a friend to a foe. We totally hand over our fate to an Earth system that starts rolling out of equilibrium.”

Like I said, I had missed this, and it was only an odd set of circumstances that led me to read it, about which more below. But first, the paper itself. The illustrious set of authors postulate that once the global temperatures reach about 2C above pre-industrial, a set of positive feedbacks will kick in such that the temperature will continue to rise to about 5C above pre-industrial, even without any further emissions and direct human-induced warming. Ie, once we are going past +2C, we won’t be able to stabilise at any intermediate temperature below +5C.

The paper itself is open access at PNAS. The abstract is slightly more circumspect, claiming only that they "explore the risk":

We explore the risk that self-reinforcing feedbacks could push the Earth System toward a planetary threshold that, if crossed, could prevent stabilization of the climate at intermediate temperature rises and cause continued warming on a "Hothouse Earth" pathway even as human emissions are reduced. Crossing the threshold would lead to a much higher global average temperature than any interglacial in the past 1.2 million years and to sea levels significantly higher than at any time in the Holocene.

(the paper fleshes out these words in numerical terms).

The paper lists a number of possible positive carbon cycle feedbacks and quantifies them as summing to a little under half a degree of additional warming (Table 1 in the paper). The authors then wave their hands, say it could all get much worse, and with one bound Jack was free. End of paper. I went through it again to see what I’d missed, and I really hadn’t. It is just make-believe, they don’t "explore the risk" at all, they just assert it is significant. There’s a couple of nice schematic graphics about tipping points too.

The mildly interesting part is what led me to read the paper at all, 6 months after missing its original publication. An editor contacted me a little while ago to ask if I’d write half a of debate (to form a book chapter) over whether exceeding 2C of warming would lock us onto a trajectory for a much warmer hothouse earth. I was charged with arguing the sceptical side of that claim. I was initially a bit baffled by the proposal as I had not (at that point) thought anyone had claimed anything to the contrary, but it soon became clear what it was all about. I said I’d be happy to oblige, but it turns out that my intended opponents, being two of the co-authors on the paper itself, were not prepared to defend it in those terms.

For posterity

Should have been quicker off the mark with the screenshot, cycling legend Gethin Butler pipped me last weekend at Brighton. This is the runbritainrankings list of UK marathon performances this year so far, in the male V50 category.

Of course the top guys tend to run London which is at the end of the month, so I'll be bumped a fair way down the order shortly. Not to mention the other various marathons through the rest of the year. I'll probably be around 20th eventually.

[jules' pics] Durham Cathedral

--
Posted By Blogger to jules' pics at 4/12/2019 04:52:00 PM

Unsubscribe from new post emails for this blog.

2:47:15

Another year, another Manchester marathon, another perfect day with perfect cool, still, cloudy weather and perfect organisation, another PB, shaving 89 seconds off the time I managed last year. A lengthier blog is posted here for those who are interested in these things. It was a good run for me and although I did dream of 2:45 (and set off aiming for that time) I wasn't quite up to it in the end. Finished 3rd in the V50 category (out of about 800 or so) which was a little stronger than in previous years as the event is growing in size and reputation so the field is strengthening. I was actually more than 40 places lower in 167th place overall despite the faster time.

Soon: off to Stockholm to finish off some work we've been very lackadaisical over for the past couple of years. That will be worth some blog posts when it's done. Hopefully we'll still be in the EU or else the travel might be...interesting.

Oh, this marathon result means one of my predictions has failed. Well, I only gave myself a 60% chance, so this was only a marginally less favoured outcome. The remaining 5 all seem well on track.