Munich Center for Mathematical Philosophy (MCMP)

Breadcrumb Navigation


Talk (Work in Progress): Michał Tomasz Godziszewski (MCMP/LMU)

Location: Ludwigstr. 31, ground floor, Room 021.

13.02.2020 12:00  – 14:00 


The Brier Score and Elimination Counterexamples


In the accuracy-first epistemology program, the usual thing used to assess the accuracy of the degrees of belief is to use scoring rules - functions that measure the quality of a probability-estimate for a given event, with lower scores signifying probabilities that are closer to the event's status (1 if it occurs, 0 otherwise). Which of the rules should we choose? D. Fallis and P. Lewis (2016)) argue that the Brier Score, at least in the partition version, is not a good tool for measuring the value of an agent's belief function. The reason is that conditionalization is supposedly always of epistemic benefit to the agent, yet there are cases in which, according to the Brier Score, the inaccuracy of a belief function increases after conditionalization cases like this shall be called 'elimination counterexamples'. Both the logarithmic and spherical measures are immune to
elimination counterexamples. So, should we care? Is this really so bad? Is this the fault of the Brier score? Is the situation similar in the context of Boolean algebras instead of partitions? As L. Wroński suggested (2017), a natural thing to do in an inaccuracy-first framework is to point out that conditionalization minimizes expected inaccuracy. In all examples the following is true: there is a different world $w$ such that, werethat world the actual one, the inaccuracy would decrease. A real flaw of the Brier score would be the following: if it allowed situations in which after conditionalization the inaccuracy increased whatever the actual world was. For the case of partitions, we know this cannot happen, and so we have proven "sanity check'' theorem for conditionalization coupled with the Brier Score: There are no probabilistic mass vectors $p$ and $q$ such that $q$ is obtained from $p$ via conditionalization and $B(p) < B(q)$, regardless of the choice of the actual world $w$. What is more, as worked out during private communication with B. Fitelson, D. Fallis and P. Lewis, we can give necessary and sufficient conditions for the Brier score rising over partitions. During the talk I wish to prove the results, put it in the appropriate philosophical context and discuss the arguments that use the elimination counterexamples in the debates on scoring rules.