An algorithm to calculate transient distributions of cumulative rate and impulse based reward

Abstract

Markov reward models have been used to solve a wide variety of problems. In these models, reward rates are associated to the states of a continuous time Markov chain, and impulse rewards are associated to transitions of the chain. Rate based rewards are gained per unit time in the associated state, while impulse rewards are gained instantaneously each time certain transitions occur. We develop an efficient algorithm to calculate the distribution of the total accumulated reward over a given interval of time when both rate and impulse rewards are present. As special cases, we obtain an algorithm to handle models for which only rate rewards occur and another algorithm for the case when only impulse rewards are present. The development is based purely on probabilistic arguments, and the recursions obtained are simple and have a low computational cost. Copyright © 1998 by Marcel Dekker, Inc.