Expected value of Markov chain after 3rd step

[Win_odd Dhamnekar]

A Markov chain [imath] \{ X_n, n \geqslant 0\}[/imath] with states 0, 1, 2 has the transition probability matrix [math] P =\begin{bmatrix} \frac12 & \frac13 & \frac16 \\ 0 & \frac13 & \frac23 \\ \frac12 & 0 & \frac12 \end{bmatrix}[/math]If [imath]P\{X_0 = 0\} = P\{X_0 =1\} =\frac14,[/imath] find [imath]E[X_3][/imath]

How to answer this question correctly?

 

[BigBeachBanana]

A Markov chain [imath] \{ X_n, n \geqslant 0\}[/imath] with states 0, 1, 2 has the transition probability matrix [math] P =\begin{bmatrix} \frac12 & \frac13 & \frac16 \\ 0 & \frac13 & \frac23 \\ \frac12 & 0 & \frac12 \end{bmatrix}[/math]If [imath]P\{X_0 = 0\} = P\{X_0 =1\} =\frac14,[/imath] find [imath]E[X_3][/imath]

How to answer this question correctly?

Perhaps my stochastic is rusty but I'm having trouble interpreting the meaning of [imath]E[X_3][/imath].
Often when we find the expected value of discrete Markov Chains, we're looking for the number of steps to transition from one state to another. Based on my interpretation, your question is incomplete.

 

[Win_odd Dhamnekar]

Perhaps my stochastic is rusty but I'm having trouble interpreting the meaning of [imath]E[X_3][/imath].
Often when we find the expected value of discrete Markov Chains, we're looking for the number of steps to transition from one state to another. Based on my interpretation, your question is incomplete.

But Author provided the answer to this question as follows. Would you explain that?
[math]E[X_3] = P(X_3=1) + 2P( X_3 =2) = \frac14P^3_{01} + \frac14P^3_{11} + \frac12P^3_{21} + 2 \left[ \frac14P^3_{02} + \frac14P^3_{12} +\frac12P^3_{22} \right][/math]

 

[BigBeachBanana]

But Author provided the answer to this question as follows. Would you explain that?
[math]E[X_3] = P(X_3=1) + 2P( X_3 =2) = \frac14P^3_{01} + \frac14P^3_{11} + \frac12P^3_{21} + 2 \left[ \frac14P^3_{02} + \frac14P^3_{12} +\frac12P^3_{22} \right][/math]

Now I understand, but I think it's a question that shouldn't be asked for discrete Markov Chain because it's not guaranteed you'll have a whole number as an answer, where your answer must be either 0,1, or 2.
[imath]E[X_3][/imath] is the expected state after 3 transitions.
First, define the distribution for [imath]X_3[/imath]:
[math]X_3 = \begin{cases} 0 &\text{with } \Pr(X_3=0)\\ 1 &\text{with } \Pr(X_3=1)\\ 2 &\text{with } \Pr(X_3=2)\\ \end{cases}[/math]Then [imath]E[X_3]=\cancel{0\cdot \Pr(X_3=0)}+ 1\cdot \Pr(X_3=1) + 2\cdot \Pr(X_3=2)[/imath]
Note that it's not necessary to compute [imath]\Pr(X_3 = 0)[/imath] since it's getting multiply by 0.

Now, find [imath]\Pr(X_3 =1)[/imath]. In words, the probability that after 3 transitions, we're in state 1. Moreover, we could've started in either states 0, 1, or 2. Thus:
[math]\Pr(X_3=1)= \frac{1}{4}p_{01}^{(3)} + \frac{1}{4} p_{11}^{(3)} + \frac{1}{2}p_{21}^{(3)}[/math]Similarly,
[math]\Pr(X_3=2)= \frac{1}{4}p_{02}^{(3)} + \frac{1}{4}p_{12}^{(3)} + \frac{1}{2}p_{22}^{(3)}[/math]
Note: I'm putting the superscript [imath](3)[/imath] in parenthesis to differentiate between exponents and numbers of steps. i.e. [imath]p_{01}^3 \neq p_{01}^{(3)} [/imath]
Putting all the parts together will give you the provided solution.

 

[BigBeachBanana]

The more I think about how the expected value was computed, it makes utterly no sense. The state (0, 1, and 2) is a categorical variable i.e. it is one that has two or more categories, but there is no intrinsic ordering to the categories. Applying mathematical operations to categorical variables does not provide any meaningful insight.

Suppose I define the states 0 = Healthy, 1= Disabled, 2= Dead.
How do you compute this?
\(\displaystyle E[X_3]=\text{Healthy}\cdot \Pr(X_3=\text{Healthy}) + \text{Disabled}\cdot\Pr(X_3=\text{Disabled}) +\text{Dead}\cdot \Pr(X_3=\text{Dead})\)

The second point is that looking at this:
[imath]E[X_3]=\cancel{0\cdot \Pr(X_3=0)}+ 1\cdot \Pr(X_3=1) + 2\cdot \Pr(X_3=2)[/imath]

State 0 would never be an answer because it's only interpolating between 1 & 2, where [imath]\Pr(X_3=1) = \Pr(X_3=2) \neq 0[/imath].

The provided solution is utter nonsense.

 

[Win_odd Dhamnekar]

The more I think about how the expected value was computed, it makes utterly no sense. The state (0, 1, and 2) is a categorical variable i.e. it is one that has two or more categories, but there is no intrinsic ordering to the categories. Applying mathematical operations to categorical variables does not provide any meaningful insight.

Suppose I define the states 0 = Healthy, 1= Disabled, 2= Dead.
How do you compute this?
\(\displaystyle E[X_3]=\text{Healthy}\cdot \Pr(X_3=\text{Healthy}) + \text{Disabled}\cdot\Pr(X_3=\text{Disabled}) +\text{Dead}\cdot \Pr(X_3=\text{Dead})\)

The second point is that looking at this:
[imath]E[X_3]=\cancel{0\cdot \Pr(X_3=0)}+ 1\cdot \Pr(X_3=1) + 2\cdot \Pr(X_3=2)[/imath]

State 0 would never be an answer because it's only interpolating between 1 & 2, where [imath]\Pr(X_3=1) = \Pr(X_3=2) \neq 0[/imath].

The provided solution is utter nonsense.

This problem is taken from the following book. I think this book does not contain any nonsensical study material.


[math]E[X_3] =0.963[/math]