Found a fun little quiz somewhere, which I thought some of my readers might like to take. My aim is not to embarrass people who may get some answers wrong – in testing, the vast majority of all respondents (including researchers who reported substantial experience) were found to make mistakes. My hypothesis is that my readers are rather more intelligent than average 
Please answer in comments but work out your answers before reading what others have said, so as not to be unduly influenced by them.
I will summarise and explain the quiz when enough have answered… Please answer in comments but work out your answers before reading what others have said, so as not to be unduly influenced by them.
A researcher undertakes an experiment and reports “the 95% confidence interval for the mean ranges from 0.1 to 0.4”
Please mark each of the statements below as “true” or “false”. False means that the statement does not follow logically from the quoted result. Also note that all, several, or none of the statements may be correct:
1. The probability that the true mean is greater than 0 is at least 95%.
2. The probability that the true mean equals 0 is smaller than 5%.
3. The “null hypothesis” that the true mean equals 0 is likely to be incorrect.
4. There is a 95% probability that the true mean lies between 0.1 and 0.4.
5. We can be 95% confident that the true mean lies between 0.1 and 0.4.
6. If we were to repeat the experiment over and over, then 95% of the time the true mean falls between 0.1 and 0.4.
6 comments:
We don't know what the "so called" true mean is from the given information, we only know this particular sample confidence interval. I'll say false to all six statements. You can call me statistically stupid at the 95% confidence level later though.
I want all of these to be true. So I guess I want credibility intervals.
The true mean is presumably a single number and therefore it has to be 0% or 100% chance of being some particular number or within a range. So I am tempted to label 1, 2, 4 and 6 as 'not even wrong' while 3, and 5 are just wrong. However by talking about 95% probability this appears to give the statements the appearance of being a: Given what we know, the probability is. However, there just isn't enough information: There are infinite number of sets the numbers could be drawn from and no means of indentifying a credibility interval for just a few of them nor a means of drawing a representative sample. Not sure if 'False meaning does not follow logically' sufficiently captures this.
3 and 5 just seem false from just treating confidence interval as credible interval.
I suspect there may be several other issues that I haven't even got close to indicating.
I have a post on your other site in moderation.
The paper you mentioned over there had a reply and a response to the reply from the original authors ,,,
Interpreting confidence intervals: A comment on Hoekstra, Morey, Rouder, and Wagenmakers (2014)
https://link.springer.com/article/10.3758/s13423-015-0859-7
Continued misinterpretation of confidence intervals: response to Miller and Ulrich
https://link.springer.com/article/10.3758/s13423-015-0955-8
Both articles are open access.
One more 2018 invited paper from three of the original authors ...
Improving the interpretation of confidence and credible intervals
https://iase-web.org/icots/10/proceedings/pdfs/ICOTS10_8A2.pdf
A 2016 paper (referenced in above article) ...
The Interpretation of Scholars' Interpretations of Confidence Intervals: Criticism, Replication, and Extension of Hoekstra et al. (2014)
https://www.frontiersin.org/articles/10.3389/fpsyg.2016.01042/full
Finally, a May 2019 paper ...
CONCEPTUAL KNOWLEDGE OF CONFIDENCE INTERVALS IN PSYCHOLOGY UNDERGRADUATE AND GRADUATE STUDENTS
https://iase-web.org/documents/SERJ/SERJ18(1)_Crooks.pdf
(read the abstract as I can't cut and paste it right now for some odd reason)
Thanks for links
I see that reply does include the quibble I made with the wording of question 5. Doesn't change the fact that the use of the term "confident" there will mislead any innocent reader who isn't aware it is being used in a technical sense that does not coincide with common english usage.
Post a Comment