You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
Don't understand Stats Prob: Markov chain has transition matrix...
- Thread starter Joystar77
- Start date
After reading the reference given by Denis,A Markov chain has transition matrix
\(\displaystyle \begin{pmatrix}1/3 & 0 & 2/3 & 0 & 0 \\
0 & 1/2 & 0 & 1/2 & 0 \\
3/4 & 0 & 1/4 & 0 & 0 \\
0 & 3/4 & 0 & 1/4 & 0 \\
1/3 & 0 & 1/3 & 0 & 1/3 \end{pmatrix}\)
Analyze the state space (reducibility, periodicity, recurrence, etc), and discuss the
chain's long run behavior.
I notice that your matrix does NOT have columns that sum to 1, so that sum of elements in the state vector is not conserved. Is it possible you have entered the Transpose of the transition matirx? What definitions are you using?Denis said:
What I was going to suggest is that you calculate the first few powers of the matrix and look for patterns.
Hello, Joystar77!
\(\displaystyle \text{Let }\,X \,=\,\begin{bmatrix}a\\b\\c\\d\\e \end{bmatrix}\:\text{ where }\:AX \:=\:X\:\text{ and }\:a+b+c+d+e \:=\:1\)
\(\displaystyle \text{We have: }\;\begin{bmatrix}\frac{1}{3}&0&\frac{2}{3} &0&0 \\ 0&\frac{1}{2}&0&\frac{1}{2}&0 \\ \frac{3}{4}&0&\frac{1}{4}&0&0 \\ 0&\frac{3}{4} &0&\frac{1}{4}&0 \\ \frac{1}{3}&0&\frac{1}{3}&0& \frac{1}{3} \end{bmatrix} \begin{bmatrix}a\\b\\c\\d\\e\end{bmatrix} \;=\;\begin{bmatrix}a\\b\\c\\d\\e \end{bmatrix}\)
. . . \(\displaystyle \begin{bmatrix}\frac{1}{3}a + \frac{2}{3}c &=& a \\ \frac{1}{2}b + \frac{1}{2}d &=& b \\ \frac{3}{4}a + \frac{1}{4}c &=& c \\ \frac{3}{4}b + \frac{1}{4}d &=& d \\ \frac{1}{3}a + \frac{1}{3}c + \frac{1}{3}e &=& 3 \end{bmatrix}\)
. . . . \(\displaystyle a +b+c+d+e \;=\;1\)
\(\displaystyle \text{Solve the system: }\:\begin{Bmatrix}a &=& \frac{1}{6} \\ b &=& \frac{1}{4} \\ c &=& \frac{1}{6} \\ d &=& \frac{1}{4} \\ e &=& \frac{1}{6} \end{Bmatrix}\)
\(\displaystyle \text{I'll let }you\text{ interpret this result.}\)
\(\displaystyle \text{A Markov chain has transition matrix: }\;A \;=\;\begin{bmatrix}\frac{1}{3}&0&\frac{2}{3} &0&0 \\ 0&\frac{1}{2}&0&\frac{1}{2}&0 \\ \frac{3}{4}&0&\frac{1}{4}&0&0 \\ 0&\frac{3}{4} &0&\frac{1}{4}&0 \\ \frac{1}{3}&0&\frac{1}{3}&0& \frac{1}{3} \end{bmatrix}\)
\(\displaystyle \text{Discuss the chain's long run behavior.}\)
\(\displaystyle \text{Let }\,X \,=\,\begin{bmatrix}a\\b\\c\\d\\e \end{bmatrix}\:\text{ where }\:AX \:=\:X\:\text{ and }\:a+b+c+d+e \:=\:1\)
\(\displaystyle \text{We have: }\;\begin{bmatrix}\frac{1}{3}&0&\frac{2}{3} &0&0 \\ 0&\frac{1}{2}&0&\frac{1}{2}&0 \\ \frac{3}{4}&0&\frac{1}{4}&0&0 \\ 0&\frac{3}{4} &0&\frac{1}{4}&0 \\ \frac{1}{3}&0&\frac{1}{3}&0& \frac{1}{3} \end{bmatrix} \begin{bmatrix}a\\b\\c\\d\\e\end{bmatrix} \;=\;\begin{bmatrix}a\\b\\c\\d\\e \end{bmatrix}\)
. . . \(\displaystyle \begin{bmatrix}\frac{1}{3}a + \frac{2}{3}c &=& a \\ \frac{1}{2}b + \frac{1}{2}d &=& b \\ \frac{3}{4}a + \frac{1}{4}c &=& c \\ \frac{3}{4}b + \frac{1}{4}d &=& d \\ \frac{1}{3}a + \frac{1}{3}c + \frac{1}{3}e &=& 3 \end{bmatrix}\)
. . . . \(\displaystyle a +b+c+d+e \;=\;1\)
\(\displaystyle \text{Solve the system: }\:\begin{Bmatrix}a &=& \frac{1}{6} \\ b &=& \frac{1}{4} \\ c &=& \frac{1}{6} \\ d &=& \frac{1}{4} \\ e &=& \frac{1}{6} \end{Bmatrix}\)
\(\displaystyle \text{I'll let }you\text{ interpret this result.}\)