Home / Statistics / Stochastic Processes / Discrete Markov Process in Continuous time

# Discrete Markov Process in Continuous time

Discrete Markov Process in Continuous time

Interesting Quote: “It is a truth very certain that when it is not in our power to determine what is true, we ought to follow what is most probable.”

– Descartes

## Introduction of Discrete Markov Process in Continuous time:

In addition to specifying a mechanism for changes of state, it is also necessary to specify when the changes occur for a continuous time process. Therefore, every continuous time process has associated with a process consisting those instants of time when the transition take place. This is called associated point process. The transition time will be denoted by t0 =0, t1 , t2 ,……and the intervals between consecutive times by Zn= tn -tn-1 , n= 0,1,2,3…. These are called inter-arrival time of a process. Such Markov process at which state values are discrete whereas time to evolve the event is continuous are called Discrete Markov process in continuous time.

## Poisson Process:

It is a type of counting process in which random variable counts something over a specified time interval. It is used to model many phenomena like

• The no. of phone call received per hour by an office.
• The no. of customers in a restaurant in a particular hour.
• The incidence of deaths in a small town with a reasonably stable population.
• The clicks emitted by GM counter to record the detection of radioactive particles

## Example of Poisson Processes

An illustrative example of the Poisson process is that of fishing. Let the random variable X(t) denote the number of fish caught in the time interval [0, t]. Suppose that the number of fish available is very large, that the enthusiast stands no better chance of catching fish than the rest of us, and that as many fish are likely pick up at one instant of time as at another.

Under these “ideal” conditions, the process {X(t); t≥ 0} may be considered to be a Poisson process. This example serves to point up the Markov property (the chance of catching a fish does not depend upon the number caught) and the “no premium for waiting” property, which is the most distinctive property possessed by the Poisson process. It means that the fisherman who has just arrived at the pier has as good a chance of catching a fish in the next instant of time as he who has been waiting for a bite for four hours without success.

## Properties of Poisson Process

• The sum of two independent Poisson process is a Poisson process.
• The difference of two Poisson process is not a Poisson process.
• In case of Poisson process, the time taken by the n events to occur, have the same distribution as the order statistics corresponding to n independent random variables uniformly distributed on the (0, t).

## Pure Birth Process

In many processes of practical importance, the appearance of the new individual (i.e., birth) or the disappearance of existing individuals (i.e., death) depends upon some degree at least on the present population size. In the Poisson process the parameter λ remains constant irrespective of the population size. A more general process can be obtained by making the parameter λ dependent on the population size ‘n’ at time t.

## Pure/Simple/Linear Death Process

### Simple Death Process

The simple birth process considered a monotonically increasing population size.  Here, we define a converse process in which the population is monotonically decreasing. Let X(t)= the population size t = time and P(t) ={𝑋(𝑡) = 𝑛} be the probability that there are n individuals at time t. Suppose, 𝜇𝑛 = 𝑛𝜇 as death rate (linearly related to population size.

In general

## Application:

Birth-death processes have been used extensively in main applications including evolutionary biology, ecology, population genetics, epidemiology, and queuing theory