tag:blogger.com,1999:blog-715339341803133734.post6336201821866312032..comments2017-03-18T13:40:54.923-05:00Comments on Maximum Entropy: The Fundamental Confidence FallacyTom Campbell-Rickettshttp://www.blogger.com/profile/07387943617652130729noreply@blogger.comBlogger4125tag:blogger.com,1999:blog-715339341803133734.post-86794972109192789442015-07-21T14:46:05.840-05:002015-07-21T14:46:05.840-05:00Maybe I didn't explain myself clearly. I don&#...Maybe I didn't explain myself clearly. I don't think we need to marginalize over probability models and confidence procedures.<br /><br />From your paper:<br /><br />"A X% confidence interval for a parameter theta is an interval (L, U) generated by an algorithm that in repeated sampling has an X% probability of containing the true value of theta."<br /><br />Thus, if Professor Bumbledorf conducts 100 experiments, and reports a (valid) 95% CI for each, and I draw one of them at random from a hat, then 95 times out of 100, I expect to get one that contains the true value of theta - there is a 95% probability that it will contain the true value (the Bernoulli urn rule). After a single measurement, but without seeing his data, I'm in the same position of randomly sampling from the hat. All I have to go on is the CI and its associated properties. Regardless of the probability model and the shape of the posterior, the integral from L to U is thus 0.95, by definition.<br />Tom Campbell-Rickettshttp://www.blogger.com/profile/07387943617652130729noreply@blogger.comtag:blogger.com,1999:blog-715339341803133734.post-68793841217812179412015-07-21T14:08:13.928-05:002015-07-21T14:08:13.928-05:00I don't feel that "probability is somethi...I don't feel that "probability is something out there". What I'm skeptical of is that there is a meaningful way of marginalizing over the possible probability models and confidence procedures, and hence I'm skeptical that any one is compelled to adopt probability assignment. There are, after all, an uncountable number of probability models, and for each of these probability models there are an uncountable number of 95% confidence procedures. "Many" of these are trivial, having 0 probability of containing the true value. These can't be described in any sort of "space" that I'm aware of. In typical scenarios where the principle of indifference is applied, there are natural symmetries or invariances in the problem that allow one to apply the principle. I don't see that here, but maybe I'm missing something obvious.Richard Moreyhttp://www.blogger.com/profile/11319149283079163004noreply@blogger.comtag:blogger.com,1999:blog-715339341803133734.post-80080045803463345442015-07-21T10:39:29.472-05:002015-07-21T10:39:29.472-05:00Hi Richard
Many thanks for your comment. Strictly...Hi Richard<br /><br />Many thanks for your comment. Strictly, you are correct that we cannot say that a probability assignment follows logically, without specifying a probability model. As there was no probability model prescribed in the survey question, however, I was free to supply my own, and as there was no information supplied to adjust my probability estimate either above or below the 95% confidence reported, I shot down the middle, as indifference dictates. The situation is analogous to the submarine example, where instead of being supplied the separation between a pair of bubbles, we are only aware of the calculated confidence interval, and we have to determine the desired probability - all we have to go on is the defined 'long-run' behaviour of confidence intervals.<br /><br />You say that "under some conditions the statements might be true, but inferring this would require information that was not stated in the problem." Maybe I misunderstand your meaning, but this strikes me as strange. It sounds as if you feel that a probability is something out there, waiting to be discovered, as soon as the required evidence comes to light. <br /><br />My view is that probability theory is the machine we use for quantifying how much we know, in the presence of missing information. As long as a question is meaningfully constructed, there can be no situation in which there is not enough information to form probability estimate. Otherwise, how would we ever accumulate enough knowledge to get started?<br /><br /> Thanks again for your comment.Tom Campbell-Rickettshttp://www.blogger.com/profile/07387943617652130729noreply@blogger.comtag:blogger.com,1999:blog-715339341803133734.post-64868777246847036042015-07-21T07:34:46.079-05:002015-07-21T07:34:46.079-05:00Hi Tom, I just saw your blog post here. Thanks for...Hi Tom, I just saw your blog post here. Thanks for the summary. I would like to object to your characterisation of the Hoekstra et al survey: note that in the survey, the participants were asked to note which of the responses *logically follow* from the information given. In the sense of the paper, "false" is "this statement does is false in the sense that I cannot infer the statement from the information about the CI." It is certainly true that none of the statements logically follow. It is also certainly true that under some conditions the statements might be true, but inferring this would require information that was not stated in the problem. Best, Richard MoreyRichard Moreyhttp://www.blogger.com/profile/11319149283079163004noreply@blogger.com