GATE Statistics Stochastic Processes Syllabus

Stochastic Processes syllabus — full chapter and topic list

Expand any chapter to see its topics and sub-topics. This is the whole examinable outline for Stochastic Processes in GATE Statistics, not a summary of it.

1. 1

   Markov chains

   5 topics

   • Finite State Space
   • Countable State Space
   • Classification of States
   • Limiting Behavior of n-step Transition Probabilities
   • Stationary Distribution
2. 2

   Poisson Process

   1 topic

   • Basic Properties
3. 3

   Birth-and-Death Process

   1 topic

   • Basic Properties
4. 4

   Pure-Birth Process

   1 topic

   • Basic Properties
5. 5

   Pure-Death Process

   1 topic

   • Basic Properties
6. 6

   Brownian Motion

   1 topic

   • Basic Properties

Stochastic Processes flashcards for GATE Statistics

18 of 50 cards from the Stochastic Processes deck — real questions with worked answers.

1. What is a stochastic process?

   A stochastic process is a collection of random variables $\{X_t : t \in T\}$ defined on a common probability space, indexed by a parameter $t$ (usually time). It models the evolution of a random system over time.

2. Define a discrete-time Markov chain and state the Markov property.

   A discrete-time Markov chain is a process $\{X_n\}_{n \geq 0}$ on a state space $S$ satisfying the Markov (memoryless) property: $$P(X_{n+1}=j \mid X_n=i, X_{n-1}=i_{n-1},\dots,X_0=i_0) = P(X_{n+1}=j \mid X_n=i).$$ The future depends on the past only through the present.

3. What does it mean for a Markov chain to be time-homogeneous?

   A Markov chain is time-homogeneous if the one-step transition probabilities do not depend on $n$: $$P(X_{n+1}=j \mid X_n=i)=p_{ij} \quad \text{for all } n.$$

4. What is the transition probability matrix $P$ of a Markov chain, and what properties must its entries satisfy?

   $P=(p_{ij})$ with $p_{ij}=P(X_{n+1}=j \mid X_n=i)$. It is a stochastic matrix: $p_{ij}\geq 0$ and each row sums to one, $\sum_{j} p_{ij}=1$ for all $i$.

5. State the Chapman–Kolmogorov equations for $n$-step transition probabilities.

   $$p_{ij}^{(m+n)} = \sum_{k \in S} p_{ik}^{(m)}\, p_{kj}^{(n)}.$$ In matrix form $P^{(m+n)} = P^{(m)} P^{(n)}$, hence the $n$-step matrix is $P^{(n)} = P^{n}$.

6. How is the $n$-step transition probability $p_{ij}^{(n)}$ obtained from the transition matrix?

   It is the $(i,j)$ entry of the $n$-th matrix power: $p_{ij}^{(n)} = (P^{n})_{ij}$, with $P^{0}=I$ so $p_{ij}^{(0)}=\delta_{ij}$.

7. If $\pi^{(0)}$ is the initial distribution (row vector), what is the distribution after $n$ steps?

   $$\pi^{(n)} = \pi^{(0)} P^{n}.$$ The marginal distribution at time $n$ is the initial row vector times the $n$-th power of $P$.

8. What does it mean for state $j$ to be accessible from state $i$ (notation $i \to j$)?

   State $j$ is accessible from $i$ if there exists $n \geq 0$ with $p_{ij}^{(n)} > 0$; i.e. it is possible to reach $j$ from $i$ in some number of steps.

9. When do two states communicate, and what kind of relation is communication?

   States $i$ and $j$ communicate ($i \leftrightarrow j$) if $i \to j$ and $j \to i$. Communication is an equivalence relation (reflexive, symmetric, transitive), partitioning $S$ into communicating classes.

10. What is an irreducible Markov chain?

    A Markov chain is irreducible if all states communicate with one another, i.e. there is a single communicating class equal to the whole state space $S$.

11. Define a recurrent state and a transient state.

    Let $f_i = P(\text{return to } i \mid X_0=i)$. State $i$ is recurrent if $f_i = 1$ (return is certain) and transient if $f_i < 1$ (positive probability of never returning).

12. Give the criterion for recurrence/transience in terms of the sum $\sum_n p_{ii}^{(n)}$.

    State $i$ is recurrent iff $\sum_{n=0}^{\infty} p_{ii}^{(n)} = \infty$, and transient iff $\sum_{n=0}^{\infty} p_{ii}^{(n)} < \infty$. For a transient state the expected number of returns is finite.

13. Distinguish positive recurrent from null recurrent states.

    Let $m_i = E[\text{return time to } i \mid X_0=i]$. A recurrent state is positive recurrent if $m_i < \infty$ and null recurrent if $m_i = \infty$ (return is certain but expected return time is infinite).

14. Are recurrence, transience, positive/null recurrence class properties?

    Yes. They are class properties: all states in a communicating class are of the same type. If one state in a class is recurrent (resp. transient, positive/null recurrent), all states in that class are too.

15. Define the period of a state $i$.

    The period of $i$ is $d(i)=\gcd\{n \geq 1 : p_{ii}^{(n)} > 0\}$. If $d(i)=1$ the state is aperiodic; periodicity is a class property.

16. What is an aperiodic Markov chain?

    A chain is aperiodic if every state has period $1$. For an irreducible chain, it suffices that one state be aperiodic (e.g. some $p_{ii}>0$).

17. Define an ergodic state and an ergodic Markov chain.

    A state is ergodic if it is positive recurrent and aperiodic. A chain is ergodic if it is irreducible, positive recurrent, and aperiodic.

18. What is an absorbing state?

    A state $i$ is absorbing if $p_{ii}=1$ (once entered, the chain never leaves). An absorbing state forms a communicating class by itself and is recurrent.

See more Stochastic Processes flashcards →

Planning Stochastic Processes for GATE Statistics

Stochastic Processes (GATE Statistics) FAQ