An opinionated primer on Causal Decision Theory (CDT) and Evidential Decision Theory (EDT)

The core distinction between the two theories lies in how they form probability distributions over outcomes. Causal Decision Theory restricts this to physical consequences, ignoring correlations that do not involve direct causation. Evidential Decision Theory conditions on the action itself, treating the action as evidence about the state of the world, including correlated events. This difference becomes critical when the decision-maker's action is correlated with another agent's action without a causal link.

In the motivating example, where an agent faces a copy of themselves, CDT leads to defection because the agent cannot causally influence the copy's prior decision. EDT leads to cooperation because choosing to cooperate serves as strong evidence that the copy also chose to cooperate. The author argues that this evidential reasoning yields a higher expected utility in this specific setup, challenging the default assumption that causal reasoning is always optimal.

The article extends this argument to the 'Smoking Lesion' scenario, claiming EDT solves it correctly by recognizing the correlation between smoking and cancer risk without assuming causation. It also introduces the 'Adversarial Offer' concept, suggesting that a sophisticated agent can exploit CDT agents by making offers that are only beneficial if the agent defects, thereby extracting value from their causal reasoning limitations.

For software engineers building multi-agent systems, this distinction matters when agents share code, weights, or training data. If two AI instances are functionally identical, their decisions are correlated. Applying CDT in such environments may result in both agents defecting in cooperative scenarios, whereas EDT would align their incentives to cooperate. The post suggests that decision-making modules in such systems might need to account for these evidential links to maximize collective utility.