Markovian Dependence in Utility Theory with Whole Product Sets

This paper takes the notion of Markovian dependence, so well-known in the study of stochastic processes, and uses it in a study of utility theory for sequential processes. Markovian dependence in the utility sense is defined on the basis of indifference between special types of gambles for several sequential processes. Theorems that state how the basic utility function may be broken up into an additive form under Markovian dependence are presented. In addition, the notion of stationary transition value mechanisms is explored in the context of completely homogeneous sequential decision processes.