A Dice Game
I thought it might be useful to illustrate some aspects of the system described in section 4 here in a very simple game, which I will describe below. It is very limited in what it can show, especially given that it only involves two participants. It is definitely not meant to capture all facets of the problem, but it can be a starting point for building intuition.
The game is essentially a competition between a dice expert A and a dice expert B. A table placed in the middle of a room has an open top box on it. The box contains a six-sided die. However, instead of the usual dots it has “yes” or “no” printed on each side. We do not know how many sides have “yes” and how many have “no.” We must defer to experts A and B to tell us.
Expert A has developed a Grand Unified Theory of Dice Construction (GUD-A). Sophisticated reasoning based on their theory has convinced expert A that 2 out of 6 of the sides say “yes.”
Expert B has a competing Grand Unified Theory of Dice Construction (GUD-B) in which they are equally confident. Expert B has determined, on the basis of their theory, that 5 out of 6 of the sides of the dice say “yes.”
Both expert A and expert B would like to convince us of the correctness of their GUD. To do so, they run a sequence of “experiments.” Each experiment consists of shaking the box around for a while, looking at the die, and announcing whether a “yes” or a “no” rolled up.
But neither expert A nor expert B is allowed to personally run the experiment. Instead, they must bring in an outsider to perform each roll. This “roller” does each experiment and announces a “yes” or “no” result based on what they see.
Now, there may be various reasons to be wary of a roller. They may have weak arms and not shake the box hard enough, or they may have bad eyesight that prevents them from accurately reading the top of the dice. They might just be untrustworthy. One of the players might suspect the roller of being biased in favor of the other player. So either one or both of the players might end up refusing to acknowledge the announced result of an experiment. If that happens, |V| = 0 and, according to the reward distribution algorithm, neither player receives any reward points on that experiment. Thus, the experts are motivated to consult with one another before each roll to ensure that they agree that the chosen roller is acceptable.
If one of the players (player A, for example) decides that they do not approve of how the game is being played, then that player may choose to leave the game entirely. In that case, player B may continue to play alone. But with a single player, 
Say that the players can play up to 50 dice rolls (or “experiments”). Assuming the players are driven by a self-interested desire to accumulate as many reward points as possible, can an outsider tell whether it is GUD-A or GUD-B that is the correct dice theory just by looking at how many reward points the players accumulate?
If both players and the roller are absolutely honest and reliable, then it will be fairly obvious. But we can ask what happens if there are occasional disagreements. By refusing to acknowledge outcomes that go against their own predictions, a player can narrow the difference between their total reward points and the reward points of their competitor. However, by doing this they also lower the total number of reward points distributed to the group.
The idea is that, when individuals try to accumulate the largest possible number of reward points, the group should hopefully bring clarity to the essential underlying questions. The dice game above is an extremely stripped down example of the type of scenario we might use to stress-test that idea. The fact that the players need to share a critical level of trust in each other and in the roller is what makes it more than just a simple dice rolling bet.
The biggest limitation of the above example is that there are only two players. It is straightforward, however, to extend it to many dice-betters.
Organized Skepticism 
Comments are closed.